EP15-A3-verification-of-precision-and-estimation-o

CLSIEP15-A3:verificationofprecisionandestimationofbiasWearepleasedtohaveaguestessayexplainingthelatestinMethodVerification,specificallythenewestversionoftheCLSIguidelineEP15onMethodVerification.Forlabsseekingaquickchecktoinsuretheirmethodsaremeetingmanufacturerspecifications,EP15maybetherightchoiceWhat'sNewinCLSIEP15-A3:UserVerificationofPrecisionandEstimationofBias;ApprovedGuideline-ThirdEditionR.NeillCarey,Ph.D.July2015IntroductionCLSIEP15wasreleasedasanA3documentinSeptember2014.Thisisitsfourthiteration,andalthoughitretainsmuchofitsoriginalapproach,thereweresomesignificantchangesintheA3version.Themostsignificantchangeisthecreationofarelativelysimpleexperimentthatgivesreliableestimatesofameasurementprocedure'simprecisionanditsbias.TheessentialstoaccomplishthiswerepresentinEP15throughallofitspreviousversions,buttheyarerefinedandcombinedinEP15-A3tomakeasingleexperiment.Here'sabriefdescriptionoftheprotocol.SpecificationofAcceptablePerformanceBeforedoinganythingelse,theusershouldspecifytotalallowableerror,andderivefromittheallowablestandarddeviation(or%CV),andtheallowablebias.Theusershouldascertainthattheimprecisionofthecandidatemeasurementproceduremeetsthecriterionforallowableimprecisionbeforebeginningtheevaluation.Ifthemeasurementprocedure'simprecisionreportedinpublications,suchasthemanufacturer'sstatedimprecision,doesnotmeetthecriterion,theprecisionverificationproceduredescribedinEP15-A3isnotappropriate.Beingalimitedexperiment,itislessrigorousthantheexperimentdescribedinCLSIEP5-A3,whichwouldbemoreappropriate.VerificationofPrecisionEP15firstdescribesaprecisionverificationexperiment.Iftheuserisevaluatingaprocedureforwhichtherearemanufacturer'sprecisionclaims,orpublishedprecisionresults,thatweredevelopedusingCLSIEP5,theusercanverifythepublishedprecisioninanexperimentlastingasfewasfivedays.Patientsamples,referencematerials,proficiencytestingsamples,orcontrolmaterialsmaybeusedasthetestsamples,providedthereissufficientsamplematerialfortestingeachsamplefivetimesperrunforfivetosevenruns.Precisionshouldbetestedwithtwoormoresamplematerialsatdifferentmedicaldecisionpointconcentrations.Theexperimentproducesatleast25replicatescollectedoveratleast5daysforeachsamplematerial.Therepeatability(previouslytermedwithin-run)andthewithin-laboratory(previouslytermedtotal)standarddeviationsarecalculatedbyananalysisofvariancetechnique(ANOVA)thatproperlyaccountsforthewithin-runandbetween-runcontributionstotheoverallimprecisionofthemeasurementprocedure.TheuserneedsaccesstosoftwaretodotheANOVAcalculations,buttheyareavailableinExcel,Minitab,Analyze-it,andothersoftwarepackagesthatdostatisticalcalculations.Therepeatabilityandwithin-laboratorystandarddeviationsarethencomparedtotheclaimedorpublishedstandarddeviations.Ifthecalculatedstandarddeviationsarelessthanthepublishedvalues,theuserhasverifiedtheclaim.Sometimesthecalculatedstandarddeviationsmayexceedthepublishedvalues,andyetthetruestandarddeviationsarelessthanthepublishedvalues.Forexample,ifthetruestandarddeviationswereactuallyexactlyequaltotheirclaimedcounterparts,thecalculatedstandarddeviationswouldexceedtheirpublishedcounterpartsfiftypercentofthetimeinverificationexperiments.Toallowforthispossibility,theusercalculatesaverificationlimitbasedonthepublishedstandarddeviationandthesizeoftheuser'sexperiment.Ifthecalculatedstandarddeviationislessthantheverificationlimit,itisnotstatisticallysignificantlylargerthanthepublishedstandarddeviation,andtheuserhasverifiedthepublishedprecision.Ifthecalculatedprecisionexceedstheverificationlimit,thecalculatedstandarddeviationisstatisticallysignificantlylargerthanthepublishedstandarddeviation,andtheuserhasfailedtoverifythepublishedimprecision.Thedocumentincludestablestosimplifythecalculationoftheverificationlimit.EstimationofBiasBecausetheprecisionexperimenthassomanyreplicatemeasurements,collectedoverseveraldays,resultsfromtheprecisionexperimentmaybeusedtomakeareliableestimateofthebiasofthemeasurementprocedurerelativetotheassigned(target)valuesofthesamplematerialsusedintheexperiment.Theonlyrequirementisthattheassignedvaluemustbeavailable.Thechoiceofmaterialdependsonthepurposeoftheuserinestimatingthebias.Twoormoreappropriatematerialsshouldbetestedintheprecisionexperiment.Iftheuserisinterestedinestimatingbiasrelativetothepeergroupforproficiencytesting,andwantstoestimatehowthemeasurementprocedurewillperformwellonproficiencytesting,proficiencytestingmaterialswithpeergroupvaluesforthemeasurementprocedurebeingevaluatedareappropriate.Forbiasrelativetothequalitycontrolpeergroup,qualitycontrolmaterialswithpeergroupvaluesforthemeasurementprocedureareappropriate.Assayedqualitycontrolmaterialsarenotappropriate,unlesspeergroupinformationisavailable.Typically,thereisnowaytoestimatetheuncertaintyoftheassayedvalues,whichisneededtodetermineifthecalculatedbiasisstatisticallysignificant.Internationallyrecognizedhighorderreferencematerials,suchasamaterialfromtheU.S.NationalInstituteofStandardsandTechnology,orfromtheJointCommitteeforTraceabilityinLaboratoryMedicine,orfroms