Spline Estimators for the Functional Linear Model

SplineEstimatorsfortheFunctionalLinearModel:Consistency,ApplicationandSplusImplementationHerveCardoty,FredericFerratyzandPascalSardazyUniteBiometrieetIntelligenceArticielleINRA,Toulouse,BP2731326,Castanet-TolosanCedexzLaboratoiredeStatistiqueetProbabilites,UMRCNRSC5583,UniversitePaulSabatier,31062Toulousecedex,FranceAbstractThefunctionallinearmodelisaregressionmodelinwhichtheexplanatoryvariableisacontinuoustimeprocessobservedinaclosedintervalofR:Hence,the\vectorofparameterstobeestimatedbelongstotheinnitedimensionalspaceofR-valuedoperatorsdenedonaspaceoffunctions.Weproposeheretwoestimatorsofthefunctionalparameterofsuchamodelbymeansofsplinefunctions.Theseestimatorstakeintoaccountthedimensionalityproblemandweprovetheirconsistency.Therstonereliesonatruncatedfunctionalprincipalcomponentsanalysisandthesecondisbasedonpenalizedregressionsplines.Theseestimatorsarecomparedbymeansofsimulationsandappliedtoexplainwinterwheatyieldwithrespecttoclimaticvariations.Keywords:functionallinearmodel,functionaldataanalysis,splines,principalcomponentsregression,regularization,ridgeregression,Hilbertspacevaluedrandomvariables,convergence.1IntroductionRegressionAnalysisisoftenconcernedwithfunctionaldata.Forinstance,inseveralapplications,thepredictorscanbethoughtasdiscretizationsoffunctions.Practi-ciansgetusedtoemployingstatisticalproceduresthatcandrivesuchsituationsor,fromaslightlydierentpointofview,thatallowtodealwithmanypredictorswithahighdegreeofcollinearityamongitandfewobservations.FrankandFriedman(1993)analyzechemometricsregressiontoolsthatareintendedforthissecondsitu-ationandthatarepartialleastsquares(PLS)andprincipalcomponentsregression(PCR).Theauthorsputtheseprocedureswithridgeregression(RR)inageneralstatisticalsetting:thegoalistoconstrainthecoecientvectorinalinearregressiontimetemperatures050100150200250300−2000100200(a)timeprecipitations050100150200250300−1000100200(b)Figure1:(a)Dailycumulativecenteredtemperatures(t=0is1Octoberandt=309isthebeginningofAugustofthefollowingyear).(b)Dailycumulativecenteredprecipitations.modeltobeinsomesubspace,insuchawaythattheprojectedpredictorvariableshavelargersamplevariance.ABayesianmotivationforthisisthenprovided.Ontheotherhand,theseproceduresdonottakeintoaccountthefunctionalnatureofthedatafortherstsituationdescribedabove.AspointedoutbyHastieandMallows(1993)inthediscussionoftheaforementionedpaper,itmighthavesomegainindeveloppingfunctionalmodelsforthissettingthat,besidesapplicationsinchemometrics,includeseveralothertopicsuchasimageanalysis.Moregenerally,apartofstatisticalliteraturehasbeenrecentlyconcernedwithfunctionaldata.ThemonographfromRamsayandSilverman(1997)givesgoodinsightsintoavarietyofmodelsdealingwithdatatakenascurves.Thepurposeofthispaperistoconsiderthefunctionallinearmodelinwhichthepredictorisacontinuoustimerandomprocessandtheresponseisscalar.Thisstudyismotivatedbyanagronomicalexampleinwhichonewantstoknowwhatarethemostimportanteectsofclimaticvariationsonwinterwheatyields.Wehaveasampleofn=198yieldsandtheexplanatoryvariablesaredailycumulativeprecipitationsandtemperatures(seegure1).EstimationofthelinearlinkbetweenthepredictorandtheresponseisachievedbymeansofasmoothversionofPCRi.e.smoothprincipalcomponentsregression(SPCR).Alternatively,thefunctionalcoecientisestimatedusingaB-splinesex-2pansionthatminimizesapenalizedleastsquarescriterionthatcanbeseenasasmoothversionofRR.LetusnotethatotherbasisfunctionssuchaswaveletsorFourierdecompositioncouldbeusedequivalentlyforestimatingthiscoecientfunc-tion.Ineachcase,theestimatorneedstoberegularized(Leurgansetal.1993,Ram-sayandSilverman,1997)inordertobeconsistent.Thisregularizationisachievedbytruncatingthebasisexpansionforthesmoothprincipalcomponentsregressionandbyaddingaregularityconstraintforthepenalizedsplineestimator.InSection2,bothmodelandestimatorsaredened.TheratesofconvergenceoftheestimatesarederivedinSection3.InSection4,comparisonsbetweendierentproceduresaswellascomputationalaspectsarediscussedbymeansofaMonteCarlostudy.Then,themethodologyofthispaperisappliedtothewinterwheatyielddata.Theendofthepaperisdevotedtotheproofs.Finally,SplusprogramsforcarryingouttheestimationaregivenintheAppendix.2TheFunctionalLinearModelInageneralregressionproblem,wehavearesponsevariableYtobepredictedbyasetofvariablesX1;:::;Xd.Inthefunctionalcontextdescribedintheintroduction,X1;:::;XdarediscretizationsofasamecurveXatpointst1;:::;td,thatisXj=X(tj);j=1;:::;dwheretjisinsomecompactsetsupposedtobe[0;1]withoutlossofgenerality.Onewaytomodelizethisregressionproblemistoassumethat(X;Y)isapairofrandomvariablesdenedonthesamespace,withXvaluedintheseparableHilbertspaceH,ofsquareintegrablefunctionsdenedon[0;1],andwithYvaluedinR.Theso-calledFunctionalLinearModel-withscalarresponse-isthendenedasIE[YjX=x]=(x);x2H;(1)whereisacontinuouslinearoperatordenedonHandvaluedinR.WewilldenotebyH0thespaceofsuchoperatorsandby:;:,respectivelyk:k,theusualinnerproduct,respectivelythenorm,inH.ItisobviousthatthemodelabovecanberewrittenasIE[YjX=x]=;x=Z10(t)x(t)dt;(2)w