Asymptotic properties of backfitting estimators

AsymptoticPropertiesofBackttingEstimatorsJeanD.OpsomerDepartmentofStatisticsIowaStateUniversityJuly20,1998AbstractWhenadditivemodelswithmorethantwocovariatesarettedwiththebackttingalgorithmproposedbyBujaetal.[2],thelackofexplicitexpres-sionsfortheestimatorsmakesstudyoftheirtheoreticalpropertiescumbersome.Recursionprovidesaconvenientwaytoextendexistingtheoreticalresultsforbivariateadditivemodelstomodelsofarbitrarydimension.Inthecaseoflocalpolynomialregressionsmoothers,recursiveasymptoticbiasandvarianceexpres-sionsforthebackttingestimatorsarederived.Theestimatorsareshowntoachievethesamerateofconvergenceasthoseofunivariatelocalpolynomialregression.Inthecaseofindependencebetweenthecovariates,non-recursivebiasandvarianceexpressions,aswellastheasymptoticallyoptimalvaluesforthebandwidthparameters,areprovided.1IntroductionTheadditivemodel,originallysuggestedbyFriedmanandStuetzle[4],assumesthattheconditionalexpectationfunctionofthedependentvariablecanbewrittenasthesumofsmoothtermsinthecovariates:E(YjX=(x1;:::;xD))=m1(x1)+:::+mD(xD):(1)Stone[17]showedinthecontextofregressionsplinesthattheadditivemodelhastheverydesirablepropertyofreducingafullD-dimensionalnonparametricregressionproblemtoonethatcanbettedwiththesameasymptoticeciencyasaunivari-ateproblem.Bujaetal.[2]proposedabackttingalgorithmasapracticalmethodemail:[e-mail]1forttingadditivemodelsusinganytypeofsmoothersandexploreditsproperties.Theadditivemodelhasnowbecomeawidelyusedmultivariatesmoothingtechnique,inlargepartduetotheextensivediscussioninHastieandTibshirani[7]andtheavailabilityofttingroutinesinGAMandS-Plus,describedinChambersandHastie[3].Muchrecentresearchonbackttingestimatorshasdealtwiththeconvergenceofthealgorithm.Bujaetal.[2]studythebivariateadditivemodelindetail,andshowthatboththeconvergenceofthealgorithmanduniquenessofitssolutiondependonthebehavioroftheproductofthetwosmoothermatrices.ForD-dimensionalmodels,theyprovidesucientconditionsforconvergenceofthealgorithm.Theseconditionsarequiterestrictive,sincetheyapplyonlytononparametricregressionmethodsthatproducesymmetric,shrinkingsmoothermatrices,aclassoflinearsmoothersfurtherdenedinBujaetal.[2]thatincludesprojectionsmootherssuchasparametrictermsandregressionsplines,aswellassmoothingsplines.Theyintroducetheconceptofconcurvitytodescribenonlineardependenciesbetweenthecovariatesthatleadtodegenerate(non-unique)solutionstothebackttingalgorithm.HardleandHall[6]andAnsleyandKohn[1]furtherexploretheconvergenceofthealgorithminthecontextofprojectionandsplinesmoothers,respectively.OpsomerandRuppert[14]derivesucientconditionsfortheexistenceanduniquenessoftheestimatorsforthebivariateadditivemodelforlocalpolynomialregression,awidelyusednon-projectionsmoother.Despitethison-goingresearch,thereisstillnoanswer,applicabletogenerallinearsmoothers,tothequestionsofconvergenceofthebackttingalgorithmanduniquenessoftheestimatorsforadditivemodelsofdimensiongreaterthan2.ByderivingexplicitexpressionsfortheestimatorsofD-dimensionaladditivemodels,thisarticlewillshowthatexistenceoftheestimatorsdependsonaspecictypeof\interactionbetweenthesmoothermatrices,andhencecanbeviewedasageneralizationoftheresultsofBujaetal.[2]inthebivariatecase.Alsoofinterestarethestatisticalpropertiesofbackttingestimators.Thistopicisbecomingmoreimportantasothermethodsforttingadditivemodelsareappearingintheliterature,suchasthemarginalintegrationmethodproposedbyLintonandNielsen[11]forthebivariateadditivemodelandgeneralizedtothemulti-dimensionalsettingbyHengartner[8],andthebacktting-projectionalgorithmofLintonetal.[10].Thefactthatthebackttingestimatorsaredenedasthesolutionofaniterative2algorithmhasmadethestudyoftheirstatisticalpropertiesmoredicult.Whenthesmoothersareprojectionsontowell-denedsubspaces,theestimatorscanbedenedwithouttheuseofbacktting,asexplainedinHastieandTibshirani[7].Severalauthorshaveusedthisapproachtoderivepropertiesofbackttingestimatorsforadditivemodels.See,forinstance,Stone[17]forregressionsplinesandWahba[18],Guetal.[5]forsmoothingsplines.OpsomerandRuppert[14]giveasymptoticbiasandvarianceexpressionsthatapplydirectlytothebackttingestimatorsdescribedbyHastieandTibshirani[7],inthecontextofthebivariateadditivemodelttedbylocalpolynomialregression,awidelyusednon-projectionsmoother.Inthecurrentarticle,theirresultswillbegeneralizedbyderivingtheasymptoticbiasandvarianceofthebackttingestimatorsoftheD-dimensionaladditivemodel.ThestudyofthebivariatemodelinOpsomerandRuppert[14]providesmostofthemethodological\toolsneededforthecurrentarticleandwillbefrequentlyreferredto.Itshouldbenotedthatsincetheresultsonthestatisticalpropertiesofthebackttingestimatorsinbotharticlesapplytoanon-projectionsmoothingmethod,theyareonlyvalidwhenthetrueunderlyingmeanfunctionisassumedtobeoftheform(1).Theoutlineofthearticleisasfollows.Section2derivesexplicitexpressionsforthebackttingestimatorsofthecomponentfunctionsofD-dimensionaladditivemodelsforgenerallinearsmoothers.InSection3,thebackttingestimatorsarefurtherexploredinthecaseoflocalpolynom