Webinar Q&A follow-up: singlicate ADA analysis

Written by ICON plc
General interest Immunogenicity
singlicate

Thank you to everyone who attended our webinar on Singlicate ADA analysis. Below are ICON’s responses to the questions posed during the live event. We hope this is a useful resource and thank our webinar attendees and our speakers, Paul Bolliger and Alissa Cornacchia, for their presentation.

View the full Spotlight feature here.

Q&A follow-up

Regulation:

  1. Do you expect any challenges from regulatory agencies concerning singlicate assays?

ICON’s expectation would be that there will be some questions about the approach, in particular focusing on precision. If you have development data showing very low well-to-well %CV, which leads you to validate in singlicate, inspectors may ask to see that development data. Otherwise, a validation conducted in singlicate should provide Regulatory Agencies with all the data they need. A retrospective move from duplicate to singlicate should have a validation report showing the validation assessments completed in singlicate to demonstrate no significant changes in assay performance.

  1. Is singlicate ADA analysis acceptable from a regulatory perspective?

Please refer to the publications presented regarding singlicate analysis. While immunogenicity assays are not PK assays, ICH M10 has clearly stated that singlicate analysis of ligand binding assays (LBAs) is acceptable if the method validation criteria are still met. Advances in instrumentation and reagent quality should not be overlooked and we shouldn’t always continue to do things the ‘old’ way as the bioanalytical field advances. Nevertheless, there are always concerns when changes are first implemented.

  1. How are the results of an unknown sample accepted without precision value?

We demonstrated that the frequency of unknown samples having an unacceptable %CV is often very low (<1% of results), therefore there is very little added benefit of being able to identify duplicate wells with >20% CV. In addition, some labs disregard well-to-well precision if both wells are in agreement, e.g., both wells are above the cut point. Furthermore, it is a tradition in LBAs to use two wells to confirm each other, whereas for LC–MS/MS analysis, samples are always measured in singlicate wells. Your confidence in the reported result (singlicate or mean of duplicate) is characterized during validation and must be within regulatory requirements. Finally, please realize that duplicate wells are typically only used for the final read-out step of the assay. All dilutions and acid dissociation steps are usually done in single mode, so we are only demonstrating the technical precision of analyzing a solution in duplicate wells.

  1. Have any of the sponsors you work with got as far as submitting singlicate ADA data as part of a biologics license application/marketing authorization application? If so, did you receive any regulatory feedback?

We have tested clinical samples with singlicate ADA methods but to our best knowledge, these data have not yet been submitted to regulatory agencies. So, no feedback from regulators yet.

Method validation questions:

  1. Can a new method validation be performed in singlicate?

Yes, assuming you have developed an ADA method with a high degree of precision.

  1. You mentioned validation does not need to compare duplicate vs singlicate, can you expand on that?

The only time you would need to compare singlicate and duplicate in a validation would be a retrospective analysis of duplicate data. This would be applicable in case singlicate analysis is requested for an assay that was validated and used for sample analysis in duplicate. I would not recommend validating new assays twice (data processed in duplicate vs. first-well singlicate). During method validation, you are establishing your method performance characteristics and if they meet acceptance criteria then you have successfully validated your method. Additionally, most assays are run in biological singlicate and technical duplicates, meaning we’re diluting the sample once, but adding the diluted sample to the analysis plate twice. Therefore, the duplicate measurement is primarily measuring instrument precision.

  1. If the validation was done in duplicate, is it possible to switch to single-well analysis for a clinical study if the validation data are reassessed using the first well data only?

In our opinion, yes. This approach has been used for PK and biomarker assays in the past and is supported by the ICH M10 Expert Working Group for PK validations [1]. However, an additional validation showing acceptable method performance would be required.

  1. Assuming an ADA study where validation has been completed and sample analysis is ongoing, what size dataset would you recommend as a minimum to perform these assessments to check for the possibility of singlicate moving forward? Would you carry out any further lab assessments to verify this?

The validation would contain sufficient data to demonstrate singlicate vs. duplicate performance. Any additional data collected during sample analysis can help characterize the performance further.

  1. If validation is performed in singlicate and the data shows that the inter-assay precision is >20%, what would be your approach? Will validation need to be repeated?

If inter-assay precision exceeds 20%, we recommend reviewing the data in the report and determining if the method is still fit to detect ADA. There are times when an assignable cause is identified and has necessitated repeat validation work, e.g., faulty pipette. Other times the inter-assay precision of the high positive control is unacceptable, but the low positive control and negative control are just fine. That suggests the inter-assay precision failure is not due to singlicate wells, but variability in only the responses over days/analysts/instruments. If titer precision was acceptable in this case, I see no reason not to accept the validation data. If precision cannot be improved, the assay should be run in duplicate, or the method should be further optimized.

  1. I assume that the methods you have validated to do singlicate analysis do have a fixed screening and fixed confirmatory cut point. Could you comment on this point, please?

Yes, the methods shared have fixed screening cut point factors and fixed confirmatory cut points.

Singlicate vs. duplicate questions:

  1. Do you have real clinical study sample analysis data for duplicate vs singlicate comparison? How do the ADA results change by switching to singlicate?

We have done this comparison within ICON on a limited dataset and there was no significant impact on the clinical outcomes. Whenever duplicate-well results are recalculated using only the first or only the second well per sample, results do not differ significantly. It is only for samples that are just around the assay screening cut point that results might by chance be at the other end of the cut point. However, that is then corrected in the confirmatory assay. No true ADA-positive samples are typically missed. Literature is available confirming such observations [2]. As a note, it is important to assess singlicate samples (1st well) against singlicate (1st well) processed validation data and duplicate clinical results against the mean of duplicate validation data to properly compare duplicate vs. singlicate datasets.

  1. Should assay precision be the only deciding factor when choosing between singlicate vs duplicate or would you recommend looking at other parameters too? If yes, what are they?

Precision is the single most important scientific reason to choose for singlicate vs. duplicate, but the other assay validation parameters as described in the FDA guidance on immunogenicity should obviously be met as well. However, there is no reason to believe that the number of replicates would affect the outcome of the other validation experiments.

  1. Did you evaluate the singlicate vs duplicate assay performance in assays other than ACE?

We have not yet tested other ADA method types at ICON, but this approach is not limited to ACE methods.

  1. Are transfer plates used for singlicate as they are for duplicate?

Yes, the minimum required dilution and possible acid dissociation steps are typically performed in a separate plate before transferring to the MSD plate. Please note that in duplicate analysis most steps are performed in singlicate anyways and only the final measurement is done in duplicate wells.

  1. Are the plate controls all run in singlicate as well or are these run in duplicate, i.e. the NCC is typically run as n=4 in a duplicate analysis?

When an assay is run in singlicate wells, all controls and blanks are also run in singlicate wells. However, two quality control (QC) samples per level are often used and these are analyzed as separate results. It is up to your SOPs and scientific judgment to determine how many repeats of each single-well sample are required in your assay.

  1. Have you done any retrospective analysis with methods that were run in true duplicates?

No, we have not done such a comparison at ICON.

General questions:

  1. Do you lower the cut point from the 95th to 94th percentile when using singlicate, so that no true positive samples are missed?

Cut points are not adjusted when using singlicate analysis. There have been discussions about the benefit of adjusting the percentile, but it will most likely not change clinical outcomes.

  1. Have you used assay platforms other than MSD for singlicate analysis?

Yes, as indicated during the webinar, this has also been done for an absorbance assay. There is no reason that assays other than MSD bridging assays cannot follow the same principles providing the precision of the assay is sufficient.

  1. In the 3rd study, why are the intra-assay LPC precision acceptance criteria set at <=10%?

The ‘target’ acceptance during method development was set to <10% as a rule of thumb only, as this is an indicator that the overall assay precision is very good. At this time, we are using singlicate ADA analysis on assays that demonstrate a higher level of precision. The validation acceptance criteria were not modified and still need to demonstrate intra-assay precision within 20% CV.

  1. What is the actual impact of singlicate ADA analysis on the per-sample analysis cost?

The costs of materials are reduced by at least 50%, as approximately half of the reagents, plates and pipette tips are needed. Most costs in a contract research laboratory, however, are based on labor. This will not decrease as much; it may be in the order of 10%, but that can differ between projects and assays.

  1. Would you consider introducing incurred sample reanalysis (ISR) samples to confirm results if singlicate analysis is used?

No. ISR is based on regulatory expectations for PK assays only and should not be applied to other assay types like biomarkers or immunogenicity assays. Method repeatability is shown during assay validation and continuously monitored using run acceptance QCs. ISR assessment does not add anything and is increasing overhead, so should not be done. Finally, all samples that screen positive are reanalyzed in a confirmatory assay tier anyway, adding another layer of assay precision data.

  1. Is singlicate well precision evaluated similarly for both screening and confirmatory assay formats?

Yes, screening and confirmatory precision are evaluated in the same manner.

  1. Can preclinical studies be analyzed in singlicate, or does it apply to clinical only?

This approach can be used for both clinical and non-clinical studies as long as the precision of the method allows.

  1. What is the sample acceptance criterion for samples when testing in singlicate?

This would not be any different than for duplicate analysis, except that there will be no %CV over the duplicate-well criteria. If a sample screens positive, it would be tested in the confirmatory assay. Please note that in other assay types such as LC–MS/MS analysis, samples are never analyzed in duplicate. It is a tradition in the LBA community, that may not be necessary.

  1. Are there any statistical tests used to directly compare datasets? Or are visual depictions, comparison of parameters and %CV assessments sufficient to make the conclusion?

Distribution plots of duplicate-well results are likely a good way to visualize duplicate-well assay performance. However, no statistical test is currently recommended to be used as a default; the approach should be defined by each individual laboratory.

  1. How routinely do you perform ADA analysis in singlicate?

Most Sponsors are still expecting duplicate-well ADA analysis, but now that we are moving to singlicate analysis in PK assays based on what is written in ICH M10, we see an increasing interest in singlicate immunogenicity testing. So, currently it is a small part of all ADA work, but it is being discussed in many projects and applied in several circumstances now.

  1. Singlicate analysis for ADA assays is a worthwhile approach but would most likely work best for precise methods such as MSD-ECL platforms with bridging ADA formats. Can you comment on the use of singlicate analysis for cell-based and ELISA-based ADA methods?

For ELISA-based methods, precision can be as good as for MSD bridging assays, so that should certainly be feasible for many methods. For cell-based assays, precision is indeed generally lower, so single-well analysis is less likely. However, for any method, regardless of the technology or use, precision should be evaluated and based on those data it should be decided how many replicates are needed for a reliable final result. In many contemporary LBA assay formats a single-well approach is acceptable, but not always. Duplicate or triplicate well analyses must still be done if dictated by the precision data of the assay.

  1. How do you qualify analysts for these assays with singlicates?

Analyst qualification is not a regulatory expectation. However, if you choose to perform analyst qualifications, the run layout can be similar to what you are used to for duplicate-well analysis. Acceptance criteria are typically those used for assay validation runs.

  1. How do you perform cut point analysis in singlicate, i.e., how many individuals, which balanced design?

The number of individuals should not change so typically 50 individual lots can be used for determining the assay cut points. The number of plates will obviously decrease. There is no regulatory expectation on the exact design of the assay validation, so it is up to the laboratory to determine the plate layouts.

 

References

  1. ICH M10 Expert Working Group. ICH M10: Bioanalytical Method Validation and Study Sample Analysis – Training Material. 27 January 2024. [Accessed 19/09/2024].
  2. Bertran E, Birnboeck HF, Schick E. Single-Well Analysis in Ligand-Binding Assays: Retrospective Evaluation of Validation and Study Sample Data of PK and ADA Assays using ELISA. Presented at the European Bioanalytical Forum 10th Open Symposium. Barcelona, Spain, 15–17 November 2017.

 

This feature was produced in association with: