What is the harm in that?

Medical Device (MD) and In-vitro Diagnostics (IVD) risk analysis utilizes ISO 14971 as the “go to” standard for risk management. One element of the risk management process, and the point of this blog, is establishing harm severities. Establishing a specific (as opposed to a generic) device harm table is often overlooked or done “on the fly” during a product’s risk analysis session.  This can lead to suboptimal results and inconsistencies.  To create some motivation for a pragmatic solution let’s first take a brief step back and look at one common scenario: not fully utilizing a cross functional team during development or in establishing process aids prior.  Although Research & Development (R&D) staff are chartered with product development and sustaining, we know there are many other departments/disciplines needed for achieving market realization.  Below is a depiction of a “core team”.  

Resourcing a complete interdisciplinary cross functional team can be a challenge (large or small companies). In the highly regulated MD & IVD industry not having the balance can cause many roles to be taken on by those who do not have the appropriate knowledge and experience. In development, R&D is often tasked to fill in gaps and the team becomes rather like a fiddler crab with a dominate appendage rendering less significance to other cross functional contributions. It is important to understand the need for expert contribution in other areas and compensate in some manner if there are constraints to achieving this. Let’s look at a means of achieving this balance in the area of identifying device specific patient harms.

MD and IVD products work within a specific domain of the broad healthcare diagnostics and therapy fields. Each domain carries with it, its own profile of risks. We know that risk assessment is both an art and a science. We humans do our best at estimating likelihoods of harms, effectiveness of hazard mitigations, and harm severities. 

 For many years I developed and led projects in a fiddler crab culture and witnessed engineers arguing over harm severities for particular hazards. Hello? What do engineers sitting in some conference room know about clinical harms? As the common scenario goes, let’s take a vote, how many think the generic harm is a “3”? How many think the generic harm is a “2”? Ok the “2’s” have it! This was absurd. We have a medical director whose job it is to work with the product in the clinic and who is a trained MD. Why isn’t he/she involved?!?!?! Well for a couple reasons; 1) very busy and 2) the fiddler grab mentality to product development.  At other organizations I worked for, the medical director was a part time consultant, or the clinical staff are unavailable since they are out in the field 99% of the time (often with a front row seat to patient harms). 

As an aside, at one organization, after much resistance, I did get the founding medical doctor (emergency & military rough and tumble kind of a guy having seen a lot in his career) to sit in on a detailed task by task risk assessment. Once he understood the gist, the scales fell from his eyes, and he said “do you mean all these devices I use in the emergency room go through this type of risk analysis? I had no idea!” – and he was soon on board because he saw how risk analysis is an extension of providing patients the best care. Wonder if some other top executives could benefit here as well?

So, what do you do when the medical director or clinical resource is too busy or not explicitly chartered by the organization to contribute to the “down in the trenches” risk assessment? Well, you do the next best thing, you have them help create what I call a “handy-dandy decoder ring” aka “specific harms table” for your company’s product/domain. To fix the dysfunctional generic voting approach, my regulatory cross functional team member and I were able to finally corral our medical director and created a specific harms table. It made a huge difference in the risk management process. No more debates, confidence in assigning the right level of severity, efficient and effective risk analysis process. Let’s delve more into how this can work.

If we look at ISO/TR 24971, the guidance to the ISO 14971 standard provides a nice table of severity levels (one for 3 levels, another for 5 levels). The five-level table is duplicated below.

Table 1: Generic 5-Level Harm Severity Table

If you merely apply this table as your harm severity table, you will end up debating harms for various hazards (i.e. show of hands, how many say “2”?, how many say “3”?). This is especially true with potential hazards resulting in “delayed procedure” or “information that could lead to a misdiagnosis”. These hazardous situations are very domain sensitive and hence so are their associated potential harms. A delay in acquiring a signal for a life support device like an automated external defibrillator (AED) has a different severity level than a delay in acquiring a signal from wearable health information watch. This is why you need the handy-dandy decoder domain specific harm table. This is where the cross functional (clinical) resources can belly up to the bar and really contribute!

Let’s return to ISO/TR 24971 and look at 5.5.4 that states: Severity levels are chosen and justified by the manufacturer based on the harms that could result for a particular medical device. The severity levels should be defined with sufficient specificity, so that the correct level of severity can be assigned to each harm identified in the risk analysis.”

ISO/TR 24971 Section H.2.4 further elaborates on the identification of potential harm in the light of intended use. Intended use has a significant impact on harm severity: misdiagnosis, inappropriate therapies, repercussions from false positive or false negatives etc. 

So, the intended use of the particular medical device informs harm severity. Do we want to leave this up to an R&D team to debate or can we establish this up front and greatly facilitate an efficient and effective risk analysis?  In obtaining clinical cross functional involvement, it is best if a couple areas can be explicitly declared as foundational to the harms table these are: context and direct vs secondary (or tertiary) effects.

For context, a thorough specific product type hazard analysis should exist. Table C.1 from ISO 14971:2019 and Annex A of TIR 24971:2020 provide comprehensive hazards that the team should assess relative to the specific product. Identifying the hazards that exist relative to the device is crucial to subsequently identify specific harms related to the device. IEC 62366-1 (Usability) and Cybersecurity (TIR 57 etc.) should also be comprehended.  Clearly documenting the use scenario, patient condition and user profiles being considered can be helpful. It adds context and rationale transparency.

Regarding direct versus secondary (or even tertiary) effects it is helpful to separate direct harms from subsequent harms so that severity scores can more appropriately be ascribed to the situation and thus help establish meaningful priorities. If many things can potentially result in death and they get assigned accordingly, then every risk is of the highest priority and the team won’t be able to focus on what really are the highest priority.  For example, a severe burn could lead to a serious infection resulting in death.  Referring to Table 1 above, do you assign catastrophic (5) to the harm because it could in some cases lead to death or do you assign Serious/Major (3) because the direct harm is a severe burn? This is grey and perhaps an organizational policy of basing severity on direct with imminent (high likelihood) secondary harms but not tertiary may make sense. Hence, a shock sufficient to cause heart fibrillation which can readily lead to death receives a catastrophic (5). But a bacterial infection that may lead to serious injury and ultimately to death receives a Serious Major (3) and not a Critical (4) or even further removed a Catastrophic (5). If you leave this kind of analysis for the “heat of the moment project under a tight schedule” risk analysis you will end up with debates, inconsistency, and suboptimal results.

Creating device specific harms table as a part of your risk assessment process can greatly reduce these issues. Having top management review and approve further establishes the policy and culture the team works under. The company specific harms table is applicable to all the manufacturer’s products of that domain (other tables can be for other domains). This makes it very efficient for other teams as they don’t have to continually come up with a list of harms. The harms become known and understood by the cross-functional development teams and thus jump start future risk analysis exercises.

Given you have your clinical team involved in developing such a specific harm table and you’ve established/documented some harm guidance policies, you still need to work through domain specific areas – especially in the grey areas (basically fill in the whole table range specific to the product).

Let’s take our ECG example at a high level and look at two different products to illustrate what we mean by specific harms for the device type.

This may be a bit of an extreme example, but the point is that these two different devices for the same hazardous situation (delay of case due to intermittent ECG) have vastly different potential harm severities. These harm severity differences include the intended use, the condition of the patient, the user, perhaps direct versus secondary effects etc. (BTW: if you notice many of these additional challenging areas pertain not so much to safety as they do to essential performance – be sure your harms table comprehends it! Essential Performance ) These are not areas to be debated during a product risk analysis. If the manufacture business is AED’s or Diagnostic ECG Instruments, they know their domains and capturing the potential harms up front provides the culture and the team comfort in assigning severity as they work through the risks.

In summary, by creating a specific harms table, the practical working scenario of the risk assessment team becomes:

1. Team identifies the hazards that can lead to a delay of case, in our example.
2. Look at the harms table to see what the harm severity is, in our case it’s an inconvenience.
3. Move on to the next hazard.
4. Well, this next hazardous situation can lead to misinformation what is the harm in that ….?
   

Qserve has a large pool of interdisciplinary resources (clinical, quality, regulatory, engineering) to help with establishing specific harms tables and risk analysis processes. We take a comprehensive approach to risks including usability, cybersecurity and particulars to software and AI risks. We offer project level support, training, and process support. Our desire is to help fill in gaps and provide the necessary balance to achieve global market realization in the highly regulated MD and IVD environments.