Are we there yet?

Potential Failure Assessment table

The aim of NPD project management is to develop the product within time and budget constraints, while also assuring that the design work is complete. There are risks from a NPD team not recognising an over- or under-design. Releasing a new product late and over budget, due to a tendency of making endless low value-adding refinements to the design, is practically just as bad a releasing a sub-standard design too early.

How do we know when our design work is sufficiently done?

I advocate using a design inventory tool akin to Failure Mode and Effect Analysis (FMEA). Traditional FMEA defines the effect and root cause, which is helpful when it comes to quantifying a failure impact. I don’t think the tool needs to go down into such level of details. Instead, I have simplified the approach to FMEA, making it more visual and easier to review. The resulting Potential Failure Assessment (PFA) tool is used as a ‘design inventory’ that needs filling up. The PFA tool is started early and is maintained as an on-going working document throughout the NPD project, where the PFA is revisited at every design review opportunity.

The PFA tool has two elements: 1) a fault tree visualising the failure modes and their root causes – the latter being highlighted in the grey text boxes here; and 2) an evaluation table. The two elements, together, make up an simplified FMEA. By dividing the FMEA in this way we reduce the size and complexity of our frequently revisited assessment table. The assessment table lists and considers solely the root causes (the greyed boxes in the fault tree below), which makes the approach overall more manageable – compared to a full and busier FMEA table.

Respiratory Nasal Prong

Nasal interface for a respiratory care ventilation,
relating to the PFA example below.

Fault tree for nasal prong

Example PFA fault tree for a nasal interface.

Whereas the traditional FMEA tends to be aimed at making a technical system failure-free, we use the PFA tool here to manage a wider range of objectives. Remember, the Kano model tells that a strategy aimed solely at preventing failure is not a recipe for creating true satisfaction. Achieving what Kano terms “excitement attributes” require that we think about failure as something that goes beyond the merely technical needs. When defining failure in our PFA we should set the bar wider and higher than simply ‘quality assuring’ the technical system.

In thinking of failure as a “shortfall in meeting standard or expectation” and further thinking of success as being the opposite, namely “no shortfall in meeting standard and expectation”, it enables us to manage both dissatisfiers and satisfiers within the one tool. The PFA does not only ask the FMEA question: “What could potentially go wrong”?  It further addresses: “What will it take to do it right”? For example, we have included an assessment of the potential failure of “Market rejection”, which is simply the reciprocal of market acceptance. The “Does not excite” root cause is integrated into the single PFA design document together with the more conventional technical ‘quality assurance’ root cause types, such as “Interface leak” and “Gas tube kink”. This gives us a single focal point for managing and designing out all types of failure modes. If we look at the PFA in another way it can be said to be a kind of inventory for value creation.

PFA assessment table

Example PFA assessment table for a nasal interface.

The PFA scores severity and likelihood of occurrence on a scale of 0 to 10. The aim is to make the resulting risk score as low as practically possible. Anything with a score above 10 is generally intolerable, where the product cannot be released until the underlying cause is resolved or controlled. By “generally” we refer to the fact that in some instances, such as for critical medical devices, the market regulators tolerate higher risk devices in a trade-off for their potentially positive benefits to society. However, this is not the situation for the type of device in this case study and we therefore aim to have all the risks scores reduced to 10 or below.

The PFA focuses the design activity to some extend on creating countermeasures, to eliminate or reduce the potential failures. The PFA proves a highly valuable tool, forcing the designers to evaluate and re-evaluate the strength of their design decisions and to balance them across the domains, such as engineering, production and marketing. When a design solution is considered then we can simultaneously assess the impact of its tolerance requirements on the production process. When making an engineering cost reduction decision we can simultaneously assess its impact on market acceptance.

Once we have adequately countered the potential for all root causes, when the ‘result’ scores for each the failure modes are reduced to 10 or less, then we have in effect the successfully filled the inventory of value creation. The design work is complete.

The PFA tool integrates with QFD, in helping to verify that design solutions matches the importance ranked ‘technical requirements’. It is highly appropriate in medical device design, where the principle design aim is overall “safety and efficacy”. The medical device design dossier must, by a mandatory regulatory requirement, contain a formal documented design risk assessment and also a usability evaluation. The formal documents format must be suitable for third party examination. Basically, had we not used the PFA tool here then we would have had to produce something very similar anyway. We have labelled each failure root cause in the table, with a suffix that enables us sorting and extracting those specific to the design (D), process (P), use related (U) and marketing (M). This enables us to easily extract the domain specific assessments, such as for the mandatory design risk assessment and for the usability assessment.