A consistent test for nonlinear out of sample pred

AConsistentTestforNonlinearOutofSamplePredictiveAccuracy¤ValentinaCorradi1andNormanR.Swanson21UniversityofExeter2TexasA&MUniversityApril2001AbstractInthispaper,wedrawonboththeconsistentspeci¯cationtestingandthepredictiveabilitytestingliteraturesandproposeanintegratedconditionalmomenttypepredictiveaccuracytestthatissimilarinspirittothatdevelopedinBierens(1982,1990)andBierensandPloberger(1997).Thetestisconsistentagainstgenericnonlinearalternatives,andisdesignedforcomparingnestedmodels.Oneimportantfeatureofourapproachisthatthesamelossfunctionisusedforin-sampleestimationandout-of-sampleprediction.Inthisway,weruleoutthepossibilitythatthenullmodelcanoutperformthenestinggenericalternativemodel.ItturnsoutthatthelimitingdistributionoftheICMtypeteststatisticthatweproposeisafunctionalofaGaussianprocesswithacovariancekernelthatre°ectsboththetimeseriesstructureofthedataaswellasthecontributionofparameterestimationerror.Asaconsequence,criticalvaluesaredatadependentandcannotbedirectlytabulated.OneapproachinthiscaseistoobtaincriticalvalueupperboundsusingtheapproachofBierensandPloberger(1997).Anotherapproach,whichwefollowinthispaper,istoestablishthevalidityofaconditionalp-valuemethodforconstructingcriticalvaluesthatissimilarinspirittothatproposedbyHansen(1996)andInoue(2001),althoughweadditionallyaccountforparameterestimationerror.InaseriesofMonteCarloexperiments,the¯nitesamplepropertiesofthreevariantsofthepredictiveaccuracytestareexamined.Our¯ndingssuggestthatthevariousvariantsofthetesthavegood¯nitesamplepropertieswhenquadraticlossisspeci¯ed,evenforsamplesassmallas400observations.However,non-quadraticlossfunctionssuchaslinexlosshavethefeaturethatlargersamplesizes(ofsay1000observations)arerequiredinordertoensureadequate¯ntitesampleperformance.JELclassi¯cation:C22,C51.Keywords:Conditionalp-value,nonlinearcausality,outofsamplepredictiveaccuracy,parameterestimationerror.¤ValentinaCorradi,DepartmentofEconomics,UniversityofExeter,StreathamCourt,ExeterEX44PU,U.K.,[e-mail],DepartmentofEconomics,TexasA&MUniver-sity,CollegeStation,TX77843,USA,[e-mail]editor,twoanonymousreferees,JamesDavidson,CliveW.J.Granger,BruceHansen,DavidHendry,YongmiaoHong,andseminarparticipantsattheCardi®ConferenceonLongMemoryandNonlinearTimeSeries,theUniver-sityofBirmingham,the2001MeetingsoftheRoyalStatisticalSociety,andthe2001WinterMeetingsofthe1EconometricSocietyforusefulcommentsandsuggestions.PartsofthispaperwerewrittenwhilethesecondauthorwasvisitingtheUniversityofCalifornia,SanDiego,andheisgratefultotheeconometricsgroupthereforprovidingastimulatingresearchenvironment.SwansonadditionallythanksthePrivateEnterpriseResearchCenteratTexasA&MUniversityforresearchsupport.1IntroductionInrecentyears,muchattentionhasbeengivenintheeconometricsliteraturetotheissueofpre-dictiveability.OneofthemostimportantrecentcontributionsistheseminalpaperofDieboldandMariano(DM:1995),inwhichaquitegeneraltestofequalpredictiveaccuracybetweentwocompetingmodelsisproposed.Sincethen,e®ortshavebeenmadetogeneralizeDMtypetestsinorderto:accountforparameterestimationerror(West(1996)andWestandMcCracken(1998));allowfornondi®erentiablelossfunctionstogetherwithparameterestimationerror(McCracken(2000));extendtheDMframeworktothecaseofintegratedandcointegratedvariables(ClementsandHendry(1999a,b,2001)andCorradi,SwansonandOlivetti(2001));andaddresstheissueofjointcomparisonofmorethantwocompetingmodels(Sullivan,TimmermannandWhite(1999)andWhite(2000)).Otherpapersapproachtheissueofpredictiveaccuracytestingviatheuseofencompassingandrelatedtests(seee.g.Chao,CorradiandSwanson(2000),ClarkandMc-Cracken(2000),Harvey,LeybourneandNewbold(1997)andMcCracken(1999)).1Oneofthecommonfeaturesofmanyofthepaperscitedaboveisthatnonnestedforecastingmodelsarecom-pared.However,appliedeconometriciansareofteninterestedincomparingthepredictiveaccuracyofnestedcompetingmodels.Themostobviouscontextinwhichnestedmodelsshouldbecom-parediswhenpredictiveabilityisequatedwithout-of-sampleGrangercausality,forexample.Inparticular,itisoftenofinteresttoassesswhetherhistoricaldatafromonevariableareusefulwhenconstructingaforecastingmodelforanothervariable,henceouruseofterminologysuchas\out-of-sampleGrangercausality.2Anotherfeatureoftheabovepapersisthattheycompareagivenandknownsetofmodels.Moreprecisely,theyeithercomparetwodi®erentmodelsortheycompareagivenbenchmark(orreference)modelwithmultiplealternativemodels.Needlesstosay,theremayexistsomeothermodelwhich,althoughnotincludedinthe¯nitesetofcompetingmodels,yieldssuperiorforecasts.Thisisafeatureofpredictiveability(oraccuracy)testswhichhasbeenad-1EventhoughtheDMpapersignalledrenewedinterestinthearea,itshouldbestressedthatrelatedtestshadbeenproposedinthepast(seee.g.GrangerandNewbold(1986)).2Granger(1980)summarizeshispersonalviewpointontestingforcausality,andoutlineswhatheconsiderstobeausefuloperationalversionofhisoriginalde¯nitionofcausality(Granger(1969)).Thisoperationalversionisbasedonacomparisonoftheone-stepaheadpredictiveabilityofco