Anti-drug antibody (ADA) assays are developed to detect and characterize an antibody response generated against bio-therapeutic products. The total antibody (TAb) assay (TAb) detects the total antibodies generated against the therapeutic proteins. The total antibody assay positive samples that are positive for total ADA are tested for the neutralizing capacity. These TAb-positive samples may be subjected to isotype characterization as necessary.
These assays are performed in three tiers:
a) screening assay,
b) confirmation assay,
c) titer assay
Assay cut points are established with three basic assumptions
The cut points are usually a statistically determined cut off value above which the samples are considered positive. During pre-study validation, the cut points are established by analyzing a set of approximately fifty drug-naïve healthy individual samples. The cut points are established with the following basic assumptions:
a) No prior exposure to the drug
b) No preexisting antibody is present against the drug (true negatives)
c) Represents the disease population
Assay cut points are set at the different false positive rates for three tiers of ADA assay:
The screening cut point is calculated with a target 5% false-positive rate. The confirmation cut-point is calculated with the target 1% false-positive rate. The titer cut point is generally set at a 1% false-positive rate. Based on the recent FDA guidance (2019) titer cut points can be the same as the screening cut-points with a target 5% false-positive rate.
Multi-step approach for establishing assay cut points
Assay cut points are established by analyzing fifty drug-naïve healthy individuals:
Ideally, the serum samples from the disease population should be used for cut point determination. The samples from the disease population may not be available during the pre-validation study. Instead, the serum samples from fifty drug-naïve healthy individual donors are selected for analysis of cut points. These drug naïve samples are assayed on screen and confirmed steps by two analysts over three days for a total of six runs.
These assays are performed in three tiers:
a) screening assay,
b) confirmation assay,
c) titer assay
Assay cut points are established with three basic assumptions
The cut points are usually a statistically determined cut off value above which the samples are considered positive. During pre-study validation, the cut points are established by analyzing a set of approximately fifty drug-naïve healthy individual samples. The cut points are established with the following basic assumptions:
a) No prior exposure to the drug
b) No preexisting antibody is present against the drug (true negatives)
c) Represents the disease population
Assay cut points are set at the different false positive rates for three tiers of ADA assay:
The screening cut point is calculated with a target 5% false-positive rate. The confirmation cut-point is calculated with the target 1% false-positive rate. The titer cut point is generally set at a 1% false-positive rate. Based on the recent FDA guidance (2019) titer cut points can be the same as the screening cut-points with a target 5% false-positive rate.
Multi-step approach for establishing assay cut points
Assay cut points are established by analyzing fifty drug-naïve healthy individuals:
Ideally, the serum samples from the disease population should be used for cut point determination. The samples from the disease population may not be available during the pre-validation study. Instead, the serum samples from fifty drug-naïve healthy individual donors are selected for analysis of cut points. These drug naïve samples are assayed on screen and confirmed steps by two analysts over three days for a total of six runs.
The repeated analysis over three days by two analysts is performed to factor in the analytical variability during the cut point calculation. The signal response obtained from these samples is further processed to determine the assay cut point. Based to Shankar et al. 2008, non-transformed, log-transformed, and negative control (NC) normalized data should be evaluated for establishing a cut point.
Recent reports show that the negative control normalized data are widely accepted for the calculation of the cut point. The NC-normalized data accounts for changes in the assay signal that may arise from day-day or plate-plate variability.
References
FDA Guidance on Immunogenicity Assay 2019
Shankar et al, 2008
Zhang et al. 2013, JIM
Jani et al. 2015, Future Science
Jaki et al. 2011, JPBA
Schaarschmidt et al. 2015, JIM
The Box Plot analysis method is used for the identification and exclusion of outliers:
Box Plot analysis is performed to assess, visualize, and exclude the statistical outliers (analytical and biological outliers). The Box Plot Analysis identifies outliers based on value > 75 percentile +1.5 interquartile range or value < 25% -1.5 interquartile range. Analytical outliers are the single sample value that is aberrantly larger or smaller than other replicates. They are the result of random effects (assay variabilities) such as plate-plate, inter-analyst, and inter-day variabilities.
Box Plot analysis is performed to assess, visualize, and exclude the statistical outliers (analytical and biological outliers). The Box Plot Analysis identifies outliers based on value > 75 percentile +1.5 interquartile range or value < 25% -1.5 interquartile range. Analytical outliers are the single sample value that is aberrantly larger or smaller than other replicates. They are the result of random effects (assay variabilities) such as plate-plate, inter-analyst, and inter-day variabilities.
The biological outliers have consistently different results a sample replicates compared to other samples, it includes reactive pre-existing antibodies and non-specific binding components in a matrix that produce positive responses in the assay. Both analytical and biological outliers are removed and the data are assessed for the normal distribution.
Gaussian Distribution Curve and Shapiro Wilks Test is used for the test of homogeneity
After the exclusion of outliers, the normal distribution of the pooled cut point data from a total of six runs is evaluated using the Shapiro-Wilk (SW) Test. The distribution is considered normal when the p-value is less than 0.05 for the SW test. This is critical decision-making in the calculation of the cut point. The parametric cut point calculation method is used for the normally distributed data set (SW test P>0.05) and the nonparametric cut-point method for the data set that have non-normal distribution.
A floating parametric cut-point is used when the variances are equal among six runs
Gaussian Distribution Curve and Shapiro Wilks Test is used for the test of homogeneity
After the exclusion of outliers, the normal distribution of the pooled cut point data from a total of six runs is evaluated using the Shapiro-Wilk (SW) Test. The distribution is considered normal when the p-value is less than 0.05 for the SW test. This is critical decision-making in the calculation of the cut point. The parametric cut point calculation method is used for the normally distributed data set (SW test P>0.05) and the nonparametric cut-point method for the data set that have non-normal distribution.
A floating parametric cut-point is used when the variances are equal among six runs
The mixed-effect ANOVA (one-way) method is applied to evaluate the means and variances among six cut point run performed across three days. In the ANOVA analysis, the means and variances are evaluated using F-test and Levene test respectively.
- For the NC-normalized data set with equal variances, the floating cut point is used. The floating cut point is a derived factor that is multiplied or added to the NC response to obtain the plate-specific cut point.
- For the non-transformed or log-transformed data with unequal means and equal variances, a floating cut point is calculated.
- For the non-transformed or log-transformed data with equal means and variances, a fixed cut point is calculated.
- For the NC-normalized data set with unequal means and variances, analyze interplate, inter-analyst, or inter-day variations. In this scenario, it is recommended to re-develop the assay or collaborate with the regulatory agency for acceptability.
Parametric Cut Point = Mean + 1.65 X Standard deviation (SD)
Robust Parametric Cut Points
A recent publication from Devanarayan et al. (2017) described an updated statistical approach for the calculation of cut points. This white paper recommends the universal use of NC-normalized data for the calculation of the screening/titer cut points. It also emphasizes the use of the robust parametric cut-point method. The robust parametric method utilizes the median and median absolute deviation (MAD) and this is less susceptible to outliers compared parametric method (Mean/SD).
A recent publication from Devanarayan et al. (2017) described an updated statistical approach for the calculation of cut points. This white paper recommends the universal use of NC-normalized data for the calculation of the screening/titer cut points. It also emphasizes the use of the robust parametric cut-point method. The robust parametric method utilizes the median and median absolute deviation (MAD) and this is less susceptible to outliers compared parametric method (Mean/SD).
Robust Parametric Cut Point = Median (NC-Normalized) + 1.65 X (1.48 X MAD)
Summary:
Summary:
This post elaborates on the guidance and approaches for calculating the cut-point for anti-drug antibody assays. The cut points are statistically derived from the non-transformed, log-transformed, and NC-normalized data.
The basic steps include:
a) Analysis of fifty drug-naïve individual serum
b) Assessment and Exclusion of Outliers
c) Test of data Homogeneity
d) ANOVA test for comparing means and variances
e) Calculate the cut point using nonparametric, parametric, and robust parametric methods
c) Test of data Homogeneity
d) ANOVA test for comparing means and variances
e) Calculate the cut point using nonparametric, parametric, and robust parametric methods
 |
| Decision Tree for Calculation of Cut Point |
References
FDA Guidance on Immunogenicity Assay 2019
Shankar et al, 2008
Zhang et al. 2013, JIM
Jani et al. 2015, Future Science
Jaki et al. 2011, JPBA
Schaarschmidt et al. 2015, JIM
Thank you for this article. With regard to normal/non-normal distribution, I believe what's stated in the text for the SW p-value is backwards. The distribution is considered normal when the p-value is ≥ 0.05, and non-normal when it's <0.05.
ReplyDeleteThank you for pointing out; the post has been updated
Delete