A Personal Reflection as a Working Group Member on CLSI IVD Guidance Documents
Back in the 80s when I was still a child in my early 20s, there was a standards group called NCCLS. If I look that up on Google (as I can’t for the life of me remember that far back) it stood for the National Committee for Clinical Laboratory Standards. In 2005, the name was changed to Clinical and Laboratory Standards Institute or CLSI (not to be confused with CSI but I would bet they used guidance from CLSI).
This institution had been around since the late 60’s, and by the time I was working a real job in the 90’s, they were very much a part of how things got done. These documents were how we approached the IVD development work we did day in and day out. Over the years I’ve found that if you tell the Food and Drug Administration (FDA) that you are evaluating say, precision according to EP-5, (and you in fact do that), your life will be a lot easier. I’ve come to understand that developers all over the world are using these documents which makes my job as a Principal Consultant at Qserve a bit easier as we can start by speaking the same language and conceptually understand what we are trying to demonstrate and what is important in doing that.
Back in 2014, I was at a consulting company called Myraqa. It was there that I started working with a statistician who had been working at the FDA and more importantly was one of the authors of CLSI EP12, User Protocol for Evaluation of Qualitative Test Performance. When we first met, I graciously gave her an earful of how it made me crazy that “these guidance documents are written for stats folks and not for us biological science folks.” Fortunately for me, she said, “… tell me what you mean,” and a conversation started. She asked me,” ... so how can we fix that?” When she found out that CLSI was going to update EP12, she told me I could have a seat at the table and help them make things better. I took the opportunity.
Over the past eight years, I’ve been active on two working committees involved with updating EP12 (as noted above) and EP17 (Limit of Detection). It has been an amazing ride.
The people who do this work bring their experience and real-world science to the table along with regulators. It is incredible to see what different things are being done. It can be a relief to find out that other folks are trying to deal with the same issues you are. The exchanges back and forth can be enthusiastic and sometimes exhausting, but there are moments when breakthroughs happen; everyone is suddenly quiet and there is a brave person who asks out loud, “Are we in violent agreement?” Only to have another person indicate “yes,” and the feeling of accomplishment is wonderful (the sun comes out and the birds sing!).
I’ve also found the exchanging of ideas and issues is very helpful. When someone tells me they understand the point I’m trying to make and understand why I’m frustrated, or they have the same issue, I’m grateful to know that 1) I’m not crazy, and 2) I may have found someone that can help me make things better. The ultimate goal is not to reinvent the wheel each time you are developing a new product. The science can be hard enough, understanding the process to get things done should not be.
With so many advances in technology happening these days it can be quite challenging to find a way to consolidate information in the guidance so everyone can find it useful. To make sure that we are all working toward the same expectation of what good science is, and that your data is worth the paper it’s written upon. Some things we agree upon and some things we don’t. We keep trying and with lots of hard work, we try our best to reach consensus.
At Myraqa, we had one rule that was simply, “Do good work.” Then, as now while at Qserve, for me it’s all about helping people do good science. When my clients are wondering about the work that needs to be done, I ask them if they’ve heard of the “grandma test.” Then I go on to explain that if you feel absolutely confident you would use the results from the assay you are working on for your grandma (or pick the family member of your choice), then things are okay, if not, you are not done.
I can’t speak for everyone working on guidance documents, but I would imagine that doing good science is the common bond that motivates us. For me, this is wrapped up in trying to make sure things are explained so that the work can be done efficiently to allow us to pass the grandma test.
If you have the same passion, I’d encourage you to try it out. Did I mention that we are all volunteers? Some of us are lucky that our companies recognize how important this work is. For others, they do it on their own time. Perhaps the next time you read one of these documents, take a moment to read the author’s names. I find myself doing that these days and whisper, “thanks.”
Get in touch
Are you curious about guidance documents for your IVDs? Reach out to us and we are happy to support you.