Challenges and solutions in monitoring assay performance for vaccine development: an interview with Atul Chaudhari

[00:04] Emma Hall: Hello and welcome to our In the Zone podcast on the importance of monitoring assay performance, sponsored by PPD clinical research business of Thermo Fisher Scientific. I’m your host, Emma Hall, and today I’m joined by Atul Chaudhari, Manager of Research and Development in the Vaccine Sciences Lab at PPD Laboratory Services. Atul has years of experience as a research scientist in numerous multidisciplinary fields of life sciences. He has specialized in microbiology, nanobiotechnology, pre-clinical trials and regenerative medicine, with a major focus on developing novel antibacterial strategies and immunoassays. Thank you very much for joining us, Atul.

[00:48] Atul Chaudhari: Thank you for having me, Emma, and giving me this opportunity to share my experience on assay performance and monitoring.

[00:56] Emma Hall: So, the first question I have for you today is why is monitoring assay performance and maintenance important?

[01:04] Atul Chaudhari: Before we discuss that, I would like to just briefly mention why these immunoassays are important. As we all know, all the vaccine studies are very important before licensing any vaccine formulation, and these immunoassays provide a critical platform for the evaluation of efficacy against the pathogen. So, that is why these assays play a very critical role in any vaccine program. Therefore, it is very, very important to have an immunoassay to give us 100% confidence that it’s evaluating the immune responses for the given vaccine. And therefore, it is very important to monitor its performance and also keep the assay in a maintained condition. So, for the success of any vaccine formulation, or for any immunogenicity study, is it very important that the assay performs as expected within the criteria and also [that] it keeps performing the same until the end of the clinical study.

[02:24] Emma Hall: Amazing, thank you. So you mentioned there that one of the benefits, I guess, of monitoring assay performance is for the success of any vaccine formulation. What are some other benefits of qualification and validation when developing cell and virus neutralization assays, and do you have any specific examples?

[02:46] Atul Chaudhari: Sure. So as I said, these vaccine immunogenicity assays are basically the foundation of any vaccine program. Not necessarily them being the primary endpoint in a clinical study, but they do provide important information on the ability of a vaccine to offer productive immunity against the pathogen. So therefore, it is very important to have an assay qualified and validated. So what does qualification and validation of these assays provide? A qualified assay is used mostly for testing of clinical samples in Phase 1 and Phase 2A, whereas Phase 2B and onwards sample testing in a clinical study uses a validated assay.

So basically, these validated or qualified assays provide us with critical information on vaccines, such as defining optimal vaccine formulation or dose scheduling, etc. And also, post-licensure of these vaccines — some of the activities include if the [scientists] need to improve the formulation or expand on some new clinical indications, etc. [So] these validated assays are very useful. Also, since these vaccine studies are performed on a healthy population to test the vaccine efficacy, [they] play a really integral role in evaluating the vaccine efficacy. 

[04:23] Emma Hall: Thank you. What are the most common challenges you encounter in monitoring assay performance? 

[04:31] Atul Chaudhari: As far as assay monitoring and performance [are concerned], before I jump into those challenges, I would like to briefly describe how assay monitoring and performance are categorized. So basically, I would like to categorize it as assay monitoring and assay maintenance. So what assay monitoring does is it allows us to monitor the performance of the assay; how the assay is performing over the period of time once the clinical testing has started. How we do that is we either [perform] trending [assessment], monitoring and proficiency panel testing.

So when I say that we are doing trending/monitoring, there are quality control samples in an assay when we develop the assay, and we see how these quality control samples are performing and trending over a period of time. That gives us confidence that the assay is performing well. Usually these samples have criteria on them and they should ideally follow the ‘trend’ in the sense that they should be performing similarly in each and every assay that is being performed. So that is how you monitor the assay using trending assessment.

Another way of monitoring the assay is doing your proficiency panel tests. So when I say proficiency panels, they are nothing but a bunch of samples that you test after every month, usually when you have a validated assay. So after the assay has been validated, you do the proficiency panel testing every month for 1 year. And after that 1-year period, you do the proficiency panel testing quarterly, to show that whatever panel you have been testing this assay on is being performed the same as it was performed in a validated assay. So this way [of monitoring] also gives you confidence that your assay is performing consistently and there’s no deviation. So this is what is being done in terms of assay monitoring.

For assay maintenance, there are some critical reagents of the assay, which you need to replace because the lot is limited on the vendor side and another new lot is available. So what you end up doing is, you end up qualifying or bridging that new lot of a critical reagent with the old critical reagent that is being used in the assay. This is called maintaining the assay, meaning that whatever reagent that is called ‘critical’ for the assay performance, is being breached if a new lot is available for that critical reagent.

So, there are several challenges when you talk about assay monitoring or assay maintenance. And one such example is trending investigation. Sometimes if the control samples are not performing well and they are going out of trend, meaning that they’re deviating from their usual performance, then we have to perform investigations, [to find out] ‘why is this happening?’ On a technical level we do investigations, and [also] on a reagent level, and on a lab level, et cetera, et cetera.

And also, some of the other challenges we do find during assay maintenance [include] critical reagent bridging, where sometimes, you know, the new lot is not performing similarly [compared to] the old lot. So we need to do some development activities to evaluate how or what, for example, secondary antibody is being used as a critical reagent, and if a new lot is performing at a different dilution.

So these evaluations we have to perform as R&D, and then [we] go ahead with a formal bridging experimentation to make sure that what we are bridging as a critical reagent is actually meeting the acceptance criteria as per the validity.

[09:04] Emma Hall: Amazing, thank you. So you mentioned a couple of ways to overcome some of those challenges, for example performing investigations when your control samples are deviating. Do you have any other ways that you would overcome some of those challenges?

[09:22] Atul Chaudhari: Yeah, sure. So, the other thing that we are also doing on a regular basis is [that] when the assay is usually in late clinical testing, we [put] some efforts [into], [for example], monitoring the assay on a regular basis, like in real time. At least to begin with, once the clinical testing has begun we make sure that the trending is monitored officially or unofficially, where a research scientist or a scientist would look at the runs every time and they would look at the data and how the QC samples are performing just to make sure that we don’t see any red flags, or if we do, then we start investigating right away.

And also when we do the assay, when the assay is developed and validated, we always prepare ourselves, in terms of assay maintenance, to gauge how much of a critical reagent availability we have and [if] our project is as per the clinical study duration and [meets] the need of those critical reagents.

So we are trying to make sure that, you know, [we] minimize the challenges in monitoring the assays or maintaining the assays by keeping regular checks on the performance of the assay and also by constraint projections.

[11:02] Emma Hall: Now moving on, how does assay maintenance contribute to the advancement of vaccine development?

[11:08] Atul Chaudhari: Yeah, so this question actually circles back to the first question that I answered, or I think the second question I answered: why do you need validated or qualified assays? So [the answer to this question] is the same thing. You have validated assays and you want to make sure that the assay is performing consistently and that whatever you are testing on your immunoassay, [for example] whatever clinical sample you’re testing on your immunoassay, is actually reliable and true for the success of a vaccine program.

So [it’s] not just the validated assay [that] is important, but it’s also important to make sure that the assay that you have validated is performing consistently and there’s no deviation and you can have 100% confidence of what your clinical samples are being tested upon.

So that plays a very important role in the success of a vaccine program, as far as, you know, the assay maintenance and even monitoring of that assay performance are concerned, besides just the validation of the assay. So it’s a whole package basically to [have] a successful vaccine program.

[12:30] Emma Hall: Amazing, thank you. Finally, looking more into the long-term, what would be the best approach to maintain the assay performance long-term without affecting the quality?

[12:41] Atul Chaudhari: Yeah sure, so basically as I said, there are very key components of assay performance and assay maintenance. So as I said, when you’re dealing with assay performance and assay maintenance, you have to consider several factors such as, for example, evaluation for trending performance. You know, I said we normally evaluate the control sample performance. So you want to make sure that you have enough volumes of such samples that would [last] for a long time until your clinical testing is [completed].

Another thing you want to consider in selecting the samples, besides just volume, is [if] they span the range of the assay. So when you have, for example, four control samples, you want to make sure that all these control samples span the range of the assay, so that when you’re trending the assay, you can actually see that the assay is trending well based on these four samples because they span the entire range of the assay.

So you want to make sure, besides volume, [if] you have, you know, the subjects’ availability, if at all you need to [take blood samples]. Or at least you need to consider the pooling of multiple subjects for creating such large volume samples. Then another thing for maintenance is basically [that] you want to make sure that you have a diverse group of samples, like when you are designing your proficiency panel sample, you always make sure they span the assay range, but at the same time, they have a diversity such as a mixed population, either vaccinated or convalescent (which is naturally infected samples), if possible a mix of male or female or pediatric samples if available. These are the main parameters I would consider, and their long term availability is something that I would consider to maintain the assay performance long-term. And also having the key components of the assay in a required quantity, and also [ensuring] the people who are running these assays successfully [have] their training and their qualification maintained.

So usually when the assay is validated or qualified, the people who run the assays are qualified as well. So their qualification is due in 3 or 6 months. So we have to make sure that the regular analyst qualifications are being maintained and you don’t have to do that as a separate effort, but make sure that a qualified analyst continues to run [the assay].

Also make sure that there’s no gap in between their clinical runs that would deem them unqualified to run the assay. So that approach is something that I would consider to make sure the assay is maintained and performance is also maintained. Besides that, there’s new risk mitigation by making sure your instruments or other critical components of your assays are also maintained and monitored carefully.

So yes, these are some of the approaches that you should consider to make sure that your assay is performing consistently for a long period of time without affecting the quality.

[16:41] Emma Hall: Yeah, definitely. Thank you so much. And do you have any final thoughts for us before we finish up?

[16:49] Atul Chaudhari: Yeah. As I said, the maintenance or performance and monitoring of these critical parts of a validated assay are very, very important for the success of any vaccine program. And it’s a whole package, as I already said. You develop the assay, then develop validation, and it just doesn’t end there. After validation of the assay, I think it continues until the end of the clinical study where you also focus more on assay maintenance and performance and how your assay is consistently performing because that is a very important part of the success of  any vaccine program that is evaluating the protective immunity of a vaccine formulation against a particular pathogen.

[17:50] Emma Hall: Amazing, thank you very much for sharing your experience with us Atul.

[17:54] Atul Chaudhari: Yeah, it’s my pleasure. Thank you very much for having me here and letting me share my experience on assay performance and maintenance.

[18:02] Emma Hall: You’re very welcome. I’m sure our audience will love it as well. To our audience, you can find more resources on the importance of monitoring assay performance and assay maintenance in our infographic and short video, which make up the rest of this In the Zone feature. And of course, you can visit PPD Laboratory Services for even more information.