Bioanalytical Community

 View Only
  • 1.  Acceptance criteria for cross validation

    This message was posted by a user wishing to remain anonymous
    Posted 03-06-2023 08:27
    This message was posted by a user wishing to remain anonymous

    Would ISR acceptance criteria (i.e. >67% of samples within 20% for small molecule 30% for LBA) still be considered as an acceptable approach to evaluate cross validation data although it is not mentioned in ICH M10?

    Thanks!



  • 2.  RE: Acceptance criteria for cross validation
    Best Answer

    Community Leadership
    Posted 03-06-2023 11:41

    I have seen labs use the ISR criteria for incurred samples in cross-validation. If it's good enough to show that the results are comparable within a lab, applying the same criteria to results between labs seems appropriate as well.

    That said, I prefer to also use the Bland-Altman approach described in Rocci et al (DOI: 10.1208/aapsj0903040). As an AAPS member, you can download AAPS Journal articles at no additional cost. I like this approach because it provides a confidence interval for the mean and tolerance intervals for the difference between individual samples. It also incorporates the outliers in a more statistically robust way than the 2/3 rule which does not take into account how far outside of acceptance criteria they are.



    ------------------------------
    Joleen White Ph.D.
    AAPS 2023 Global Health Community Past Chair
    Bioanalytical 101 Course Development
    Head of Bioassay Development
    Gates Medical Research Institute
    Cambridge MA
    [email protected]

    Disclaimer: Opinions expressed are solely my own and do not express the views or opinions of my employer.
    ------------------------------



  • 3.  RE: Acceptance criteria for cross validation

    Posted 03-06-2023 14:28

    @Joleen White

    Do we have consensus on the acceptance criteria for B & A plots? According Giavarina (2015), The B&A plot method only defines the intervals of agreements, it does not say whether those limits are acceptable or not. Acceptable limits must be defined a priori, based on clinical necessity, biological considerations or other goals. Do we propose that 2/3rd samples should be within the limit of agreement?



    ------------------------------
    Ritankar Majumdar
    Senior Lead Scientist
    Covance Inc. - Chantilly, Va
    Chantilly VA
    [email protected]

    Disclaimer: Opinions expressed are solely my own and do not express the views or opinions of my employer.
    ------------------------------



  • 4.  RE: Acceptance criteria for cross validation

    This message was posted by a user wishing to remain anonymous
    Posted 03-09-2023 07:44
    This message was posted by a user wishing to remain anonymous

    I also think a clear acceptance criteria should be defined if a statistic approach is used. I am not a statistician and could be wrong, but I heard that statistic data can be interpreted differently or even 'manipulated'.




  • 5.  RE: Acceptance criteria for cross validation

    Community Leadership
    Posted 03-07-2023 08:39

    Yes agreed, as in ICH M10, there is no requirement to show that two methods meet any specific criteria during cross validation.  As Joleen mentions, using a technique such as Bland-Altman plot can be much more informative than simple ISR in understanding how two methods compare, especially in variable methods such as LBAs.  For example, perhaps highly diluted samples correlate in the method but low or MRD only samples did not correlate.   That phenomena may be missed in a "passing" ISR test, but would be readily visible in a Bland-Altman plot and allow a better understanding of how two methods / labs compare in in the cross validation.     



    ------------------------------
    Robert Dodge Ph.D.
    Lead Biologics Products Bioanalysis
    Novartis
    Princeton NJ
    [email protected]

    Disclaimer: Opinions expressed are solely my own and do not express the views or opinions of my employer.
    ------------------------------