CAPA

Please FDA – Retraining is NOT the Answer!

The FDA has recently issued a draft Q&A Guidance Document on “A Risk-Based Approach to Monitoring of Clinical Investigations”. Definitely worth taking a look. There are 8 questions and answers. Two that caught my eye:

Q2. “Should sponsors monitor only risks that are important and likely to occur?”

The answer mentions that sponsors should also “consider monitoring risks that are less likely to occur but could have a significant impact on the investigation quality.” These are the High Impact, Low Probability events that I talked about in this post. The simple model of calculating risk by multiplying Impact and Probability essentially prioritises a High Impact, Low Probability event the same as a Low Impact, High Probability event. But many experts in risk management think these should not be prioritized equally. High Impact, Low Probability events should be prioritised higher. So I think this is a really interesting answer.

Q7. “How should sponsors follow up on significant issues identified through monitoring, including communication of such issues?”

One part of the answer here has left me aghast. “…some examples of corrective and preventive actions that may be needed include retraining…” I have helped investigate issues in clinical trials so many times, and run root cause analysis training again and again. I always tell people that retraining is not a corrective action. Corrective actions should be based on the root cause(s). See a previous post on this and the confusing terminology. If you think someone needs retraining, ask yourself “why?” Could it be:

- - They were trained but didn’t follow the training. Why? Could it be one or more of the Behavior Engineering Model categories was not supported e.g. they didn’t have time, they didn’t have the right tools, they weren’t provided with regular feedback to tell them how they were doing? Etc. If it’s one of these, then focus on that. Retraining will not be effective.
  - They haven’t ever received training. Why? Maybe they were absent when the rest of the staff was trained and there was no plan to make sure they caught up later. They don’t need retraining – they were never trained. They need training. And is it possible that there might be others in this situation? Who else might have missed training and needs training now? Maybe at other sites too.
  - There was something missing from the training (as looks increasingly likely as one possible root cause in the tragic case of the Boeing 737 Max). Then the training needs to be modified. And it’s not about retraining one person or one site on training they had already received. It’s about training everyone on the revised training. Of course, later on, you might want to try to understand why an important component was missing from the training in the first place.

I firmly believe retraining is never the answer. There must be something deeper going on. If your only action is retraining, then you’ve not got to the root cause. I can accept reminding as an immediate action – but it’s not based on a root cause. It is more about providing feedback and is only going to have a short-term effect. An elephant may never forget but people do.

Got questions or comments? Interested in training options? Contact me.

Beyond Human Error

One of my most frequently viewed posts is on human errors. I am intrigued by this. I’ve run training on root cause analysis a number of times and occasionally someone will question my claim that human error is not a root cause. Of course, it may be on the chain of cause-and-effect but why did the error occur? And you can be sure it’s not the first time the error has occurred – so why has it occurred on other occasions? What could be done to make the error less likely to occur? Using this line of questioning is how we can make process improvements and learn from things that go wrong rather than just blame someone for making a mistake and “re-training” them.

There is another approach to errors which I rather like. I was introduced to it by SAM Sather of Clinical Pathways. It comes from Gilbert’s Behavior Engineering Model and provides six categories that need to be in place to support the performance of an individual in a system:

Category	Example questions
Expectations & Feedback	Is there a standard for the work? Is there regular feedback?
Tools, Resources	Is there enough time to perform well? Are the right tools in place?
Incentives & Disincentives	Are incentives contingent on good performance?
Knowledge & Skills	Is there a lack of knowledge or skill for the tasks?
Capacity & Readiness	Are people the right match for the tasks?
Motives & Preferences	Is there recognition of work well done?

Let’s take an example I’ve used a number of times: getting documents into the TMF. As you consider Gilbert’s Behavior Engineering Model you might ask:

- Do those submitting documents know what the quality standard is?
- Do they have time to perform the task well? Does the system help them to get it right first time?
- Are there any incentives for performing well?
- Do they know how to submit documents accurately?
- Are they detail-oriented and likely to get it right?
- Does the team celebrate success?

I have seen systems with TMF where most of the answers to those questions is “no”. Is there any wonder that there are rejection rates of 15%, cycle times of many weeks and TMFs that are never truly “inspection ready”?

After all, “if you always do what you’ve always done, you will always get what you’ve always got”. Time to change approach? Let’s get beyond human error.

Got questions or comments? Interested in training options? Contact me.

DIGR-ACT^® is a registered trademark of Dorricott Metrics & Process Improvement Ltd.

Picture: Based on Gilbert’s Behavior Engineering Model

Searching For Unicorns

I read recently that we have reached “peak unicorn”. I wonder if that is true. I joined a breakout discussion at SCOPE in Florida last month entitled “RBM and Critical Reasoning Skills” and the discussion shifted to unicorns. The discussion was about how difficult it is to find people with the right skills and experience for central monitoring. They need to understand the data and the systems. They need to have an understanding of processes at investigator sites. And they need to have the critical reasoning skills to make sense of everything they are seeing, to dig into the data and to escalate concerns to a broader group for consideration. Perhaps this is why our discussion turned to unicorns – these are people who are perhaps impossible to find.

It does, though, strike me in our industry how much we focus on the need for experience. Experience can be very valuable, of course, but it can also lead to “old” ways of thinking without the constant refreshing of a curious mind, new situations and people. And surely we don’t have to just rely on experience? Can’t we train people as well? After all, training is more than reading SOPs and having it recorded in your training record for auditors to check. It should be more than just the “how” for your current role. It should give you some idea of the “why” too and even improve your skills. I asked the group in the breakout discussion whether they thought critical reasoning skills can be taught – or do they come only from experience? Or are they simply innate? The group seemed to think it was rather a mixture but the people who excel at this are those who are curious – who want to know more. Those who don’t accept everything at face value.

If we can help to develop people’s skills in critical reasoning, what training is available? Five Whys is often mentioned. I’ve written about some of the pitfalls of Five Whys previously. I’m excited to announce that I’ve been working with SAM Sather of Clinical Pathways to develop a training course to help people with those critical thinking skills. We see this as a gap in the industry and have developed a new, synthesized approach to help. If you’re interested in finding out more, go to www.digract.com.

Unfortunately, looking for real unicorns is a rather fruitless exercise. But by focusing on skills, perhaps we can help to train future central monitors in the new ways they need to think as they are presented with more and more data. And then we can leave the unicorns to fairy tales!

Don’t Waste a Good Mistake…Learn From It

Everyone is so busy. There’s not enough time to even think! This seems to be a challenge in many areas of business – we expect more and more from fewer people. Tom DeMarco describes this situation in his book “Slack” which I have recently re-read. And I think he’s on to something when he quotes “Lister’s Law – People under time pressure don’t think faster.” And of course, that’s right. Put people under time pressure and they will try to cut out wasted time. And they can re-prioritize so they spend more time on that task. They can work longer hours. But eventually, there is a limit and so people start to take cognitive short-cuts…”this problem is the same as one I’ve encountered before and so the solution must be the same”. Of course, that might be the right conclusion but if you don’t have the available time to interrogate it a little further then you run the risk of implementing the wrong solution and even making the problem worse.

One of the reasons I often hear as to why people don’t do root cause analysis is that they don’t have the time. People don’t want to be seen analysing a problem – much better to be taking action. But what if the action is the wrong action and is not based on the root cause? If the action is “re-training” you can be sure no-one has taken the time to really understand why the problem occurred. Having a good method you can rely on is part of the battle (I suggest DIGR^® of course). But even knowing how is no good if you simply don’t have the time. Not having the time is ultimately a management issue. If managers asked “why” questions more and encouraged their staff to take time to think, question and get to root cause rather than rushing to a short-term fix, we would have true learning.

If we are not learning from things that go wrong to try to stop it recurring then we have missed an opportunity. If the culture of an organization is for learning and improvement then management must support staff with the right encouragement to understand, and good tools. But above all they must provide the time to really understand an issue, get to root cause and implement actions to try to stop recurrence. And if your manager isn’t providing time and encouraging you in this, challenge them on it – and get them to read this blog!

As Robert Kiyosaki said “Don’t waste a good mistake…learn from it.”

DIGR^® is a registered trademark of Dorricott Metrics & Process Improvement Ltd.

Is more QC ever the right answer? Part II

In part I of this post, I described how some processes have been developed that they can end up being the worst of all worlds by adding a QC step – they take longer, cost more and give quality the same (or worse) than a one step process. So why would anyone implement a process like this? Because “two sets of eyes are better than one!”

What might a learning approach with better quality and improved efficiency look like? I would suggest this:

In this process, we have a QC role and the person performing that role takes a risk-based approach to sampling the work and works together with the Specialist to improve the process by revising definitions, training etc. The sampling might be 100% for a Specialist who has not carried out the task previously. But would then reduce down to low levels as the Specialist demonstrates competence. The Specialist is now accountable for their work – all outputs come from them. If a high level of errors is found then an escalation process is needed to contain the issue and get to root cause (see previous posts). You would also want to gather data about the typical errors seen during the QC role and plot them (Pareto charts are ideal for this) to help focus on where to develop the process further.

This may remind you of the move away from 100% Source Document Verification (SDV) at sites. The challenge with a change like this is that the process is not as simple – it requires more “thinking”. What do you do if you find a certain level of errors? This is where the reviewer (or the CRA in the case of SDV) need a different approach. It can be a challenge to implement properly. But it should actually make the job more interesting.

So, back to the original question: Is more QC ever the answer? Sometimes – But make sure you think through the consequences and look for other options first.

In my next post, I’ll talk about a problem I come across again and again. People don’t seem to have enough time to think! How can you carry out effective root cause analysis or improve processes without the time to think?

Is More QC Ever the Right Answer? Part I

In a previous post, I discussed whether retraining is ever a good answer to an issue. Short answer – NO! So what about that other common one of adding more QC?

An easy corrective action to put in place is to add more QC. Get someone else to check. In reality, this is often a band-aid because you haven’t got to the root cause and are not able to tackle it directly. So you’re relying on catching errors rather than stopping them from happening in the first place. You’re not trying for “right first time” or “quality by design”.

“Two sets of eyes are better than one!” is the common defence of multiple layers of QC. After all, if someone misses an error, someone else might find it. Sounds plausible. And it does make sense for processes that occur infrequently and have unique outputs (like a Clinical Study Report). But for processes that repeat rapidly this approach becomes highly inefficient and ineffective. Consider a process like that below:

Specialist I carries out work in the process – perhaps entering metadata in relation to a scanned document (investigator, country, document type etc). They check their work and modify it if they see errors. Then they pass it on to Specialist II who checks it and modifies it if they see any errors. Then the reviewer passes it on to the next step. Two sets of eyes. What are the problems with this approach?

It takes a long time. The two steps have to be carried out in series i.e. Specialist II can’t QC the same item at the same time as Specialist I. Everything goes through two steps and a backlog forms between the Specialists. This means it takes much longer to get to the output.
It is expensive. A whole process develops around managing the workflow with some items fast-tracked due to impending audit. It takes the time of two people (plus management) to carry out the task. More resources means more money.
The quality is not improved. This may seem odd but if we think it through. There is no feedback loop in the process for Specialist I to learn from any errors that escape to Specialist II so Specialist I continues to let those errors pass. And the reviewer will also make errors – in fact the rework they do might actually add more errors. They may not agree on what is an error. This is not a learning process. And what if the process is under stress due to lack of resources and tight timelines? With people rushing, do they check properly? Specialist I knows That Specialist II will pick up any errors so doesn’t check thoroughly. And Specialist II knows that Specialist I always checks their work so doesn’t check thoroughly. And so more errors come out than Specialist II had not been there at all. Having everything go through a second QC as part of the process takes away accountability from the primary worker (Specialist I).

So let’s recap. A process like this takes longer, costs more and gives quality the same (or worse) than a one step process. So why would anyone implement a process like this? Because “two sets of eyes are better than one!”

What might a learning approach with better quality and improved efficiency look like? I will propose an approach in my next post. As a hint, it’s risk-based!

Let’s Stop Confusing Everyone With CAPA!

I am really not a fan of the term “CAPA”. I think people’s eyes glaze over at the mention of it. It is seen as an administrative burden that the Quality Department and Regulators foist onto the people actually trying to do the real work. And I think it’s a mis-named term. CAPA stands for Corrective Action, Preventive Action. When there is a serious issue arising in a clinical trial, a CAPA is raised. This is meant to get beyond the immediate fire-fighting of the situation and to get to root cause so that corrective and/or preventive actions can be put in place. Sounds sensible. But what about when I ask you what the difference is between a corrective and a preventive action?

ISO9001:2008 defines them as:

Corrective Actions – “The organization shall take action to eliminate the causes of nonconformities in order to prevent recurrence.”

Preventive Actions – “The organization shall determine action to eliminate the causes of potential nonconformities in order to prevent their occurrence.”

Not very easy to get your head around in part because of the use of the word ‘prevent’ in both definitions. And if a Preventive Action is designed to prevent occurrence then that means the nonconformity (error) cannot have already occurred. And yet a CAPA is raised when a nonconformity (error) has occurred. So the PA part of CAPA seems wrong to me. The different definitions of Corrective and Preventive have caused no end of confusion as organisations implemented ISO9001. The good news is that in ISO9001:2015, there is a significant update in this area. When a significant issue (non-conformity) occurs you are expected to implement those immediate actions to contain the issue (termed Corrections) and also Corrective Actions to try to prevent recurrence. But the Preventive Actions are not associated with the issue. They now fit into an overall risk approach. By assessing risks in processes up-front and then continuously through their life-cycle, you are expected to develop ways to reduce the risk. These are the Preventive Actions or in risk language, the Mitigations.

Sound familiar? In clinical trials of course, we have the ICH addendum (ICH E6 R2) bringing in significant language on risk which brings it more in line with the revised ISO9001:2015 standard and is a welcome change. What is odd is that the addendum includes the following in 5.20.1:

If noncompliance that significantly affects or has the potential to significantly affect human subject protection or reliability of trial results is discovered, the sponsor should perform a root cause analysis and implement appropriate corrective and preventive actions.

This, unfortunately, mentions preventive actions next to corrective ones without any explanation of the difference and no link to the approach to risk in section 5.0. So it seems the confusion will remain in our area of work. And that confusion is compounded by our use of the CAPA terminology.

I would vote to get rid of the CAPA term all together and talk about CAR (Corrective Action Requests) and Risk. Maybe along with that, we could rehabilitate the whole approach. Done well with good root cause analysis and corrective actions, CARs are an important part of a learning organization. They should not be seen as some tedious administration that the Quality Department is requesting.

What do you think? Perhaps it’s all clear to you and you think CAPA is a great term?

In my next post I want to go back into the root cause analysis (RCA) process itself – whether DIGR^® or another method. I’ll talk more about the corrosive effect of blame on RCA and how to overcome it.

DIGR^® is a registered trademark of Dorricott MPI Ltd.

Picture: ccPixs.com

Root cause analysis can help you sleep at night

Who cares about root cause analysis (RCA)? Of course, we all do now it’s in the revised GCP, the ICH E6 (R2) Addendum*. But does it really matter? It’s easiest to think through from the perspective of an example. Consider a hypothetical situation where you are the Clinical Trial Lead on a vaccine study. Information is emerging that a number of the injections of trial vaccine have actually been administered after the expiry date of the vials. This has happened at several sites. Some example actions you might take immediately are – review medical condition of the subjects affected, review stability data to try to estimate the risk, ask CRAs to check expiry dates on all vaccine at sites on their next visit, remind all sites of the need to check the expiry date prior to administering the vaccine. These and many other actions are ones your team could quickly generate and implement and they will likely contain the issue for now. It took no root cause analysis to generate these actions. Could you sleep being confident in the knowledge that the problem won’t recur?

Without RCA, you don’t really know why the problem occurred and so you can’t fix it at the source. All you can do is put in additional checks and as these are implemented reactively, they may not be properly thought through and people may be poorly trained (or not trained at all) on the additional checks. We also know that while checks are valuable in a process they are not 100% effective when carried out by people. In this example we can be sure that the pharmacist dispensing and the nurse administering the vaccine have been trained to check the expiry date and yet we still have cases where expired vaccine has been administered. Do we really think that reminding the pharmacist and nurse is going to be enough to fix the problem forever? In a future blog, I will describe a powerful technique for RCA but for now, imagine you had managed to carry out a root cause analysis on this situation.

What you might discover in carrying out a RCA is that there is no defined process for informing sites of expired vaccine and requiring them to quarantine it. Or perhaps that the expiry date is written in American date format but being read in European format (3/8/16 being 8-Mar-2016 or 3-Aug-2016). Whatever the actual root cause, by finding what it is (or they are) you can start to consider options to try to stop recurrence. And with additional checks you could look for early signals in case these actions are not effective. By taking these actions, would you be more likely to sleep at night?

Think you know about RCA? In my next blog I will reveal why the Five Whys method we’re always told to use is not good enough for a complex situation such as this. And later, I will provide a description of a powerful technique for RCA that seems to be seldom used. If you want to hear more, please subscribe to my blog on the left of the screen. All comments welcomed!

And did you notice that I haven’t mentioned CAPA once (until now!)

* Section 5.20.1 Addendum: “If noncompliance that significantly affects or has the potential to significantly affect human subject protection or reliability of trial results is discovered, the sponsor should perform a root cause analysis and implement appropriate corrective and preventive actions.” [my emphasis]