搜索
AnIntroductiontoBootstrapMethodsusingArcIainPardoeDepartmentofAppliedStatisticsSchoolofStatisticsUniversityofMinnesota,St.Paul,MN55108TechnicalReportNumber631WorkSupportedbytheNationalScienceFoundation,GrantDUE96-52887February24,2000AbstractThisreportpresents(1)thebasicideasofbootstrappingwhenappliedtomul-tiplelinearregression,asdescribedin[2,3],and(2)howtoimplementtheseideasusingArc,thecomputerpackagethataccompanies[1].1IntroductionThisreportprovides(1)anintroductiontobootstrapmethodsinlinearregressionanal-ysesasdiscussedin[2,3],and(2)computercodeforusewiththeprogramArc,de-scribedin[1],thatimplementstheseanalyses.Theremainderofthissectionoutlinesthegeneralideasbehindlinearregression.InSection2,Isummarizethebasicboot-strapapproachtostatisticalinference,andpresenttwowaysofapplyingittolinearregression.Next,IdescribesomeoftheissuesinvolvedwithestimatingandtestingmeanfunctioncoefficientsinSections3and4respectively.Finally,Ipresentsomeex-amplesinSection5,andoutlinesomeoftheotherareasinregressionwherebootstrapmethodscouldbeusedtogoodeffectinSection6.ThemostrecentversionofArccanbeobtainedontheInternetfromthelinkfileficproblemsdiscussed.Asthesemayhaveinterestinvarioussituations,manyreaderswillfindtheadditionsuseful.Theadditionscanalsoprovideastarting1pointforimplementationofthebootstrapinothersituations,butforthisthereadermustbeabletoreadandwritecomputerprogramsinthelanguagelisp.Thereareseveralwaysofgettingstarted.Tierneyin[4]providesaveryreadableintroduction.Severalon-linereferencescanbeobtainedfrom://“SimulationsusingArc”maybehelpful,andisavailablefrom[1],regressionconcernsaresponse and predictors, .Thegeneralgoalinregressionistostudyhowtheconditionaldistributionof changesasthevalueof changes,oftenconcentratingonthemeanfunction, .Inmanyregressionproblems,theresponse iswritten !#where iscalledthestatisticalerrorandtheweights#%$’&areknown,positivenum-bers.Anotherfeatureoftheconditionaldistributionof thatisoftenstudiedinregressionisthevariancefunction(*)+, * ’(*) + - . /0#.Let1bea24365vectoroftermsderivedfrom .Typically,1willconsistofaconstant1foranintercept,and 7298 5, additionalfunctionsof ,likepolynomialsorothertransformations.Thelinearregressionmodelhasmeanfunction : * ; 1 * =? @=A CB D E E E =GF,H IB F0H J ’K 1(1)whereKL M N O= I =F0H isa2*3P5vectorofmeanfunctioncoefficients,andvariancefunction(*)+ Q R S.0#(2)Theseassumedformsofthemeanandvariancefunctionsimplythat O 0 T U&and(*) +, O . V WRS.Thisreflectsanalternativewayofspecifyingthegeneralformofthelinearregressionmodel—thelinearmeanfunction(1)togetherwiththeassumptionthatthedistributionoftheerrorsisindependentof .Forafullparametricanalysis,thedistributionof ,oralternativelyof ,mustbespecified.Fornormallydistributederrors,theleastsquarestheoryofregressionestimationandinferenceprovidesstraightforward,exactmethodsforanalysis.Butfornon-normalerrors,thesemethodshavethepotentialtobeinaccurateormisleading.Resamplingmethodssuchasthebootstrapprovideanalternativemethodology,withthepotentialtobothXreinforceconclusionsarrivedatusingnormaltheory,and2Ytoprovideestimationandinferencetechniquesinsituationswherenormaltheorydoesnotseemtobejustified.Theexamplesinthisreportfocusmainlyonthefirstofthesegoals,althoughSec-tion6mentionssomeareasthatcouldinvolvemoreinthewayofthesecondofthesegoals.2Twoalternativeparadigmsforusingbootstrapmeth-odsinlinearregressionThebootstrapisadata-basedsimulationmethodforstatisticalinference.Thebasicideaisasfollows.Iwishtomakeaninferenceabouta(population)quantity,sayZ,forwhichIhaveadata-basedestimate,[Z.Ithenwanttogetsomeideaofthedistributionofmyestimate,withouthavingtomakeassumptionsaboutmydata(forexample,thatitcomesfromamultivariatenormaldistribution).Onewaytodothisistoresamplewithreplacementfrommydatatogetabootstrapsample(ofthesamesizeasmyoriginalsample,andmadeupofcasesfrommyoriginalsample,someappearingonce,sometwice,andsoon,andsomenotappearingatall).Ithencreatealargenumber,\,ofsuchbootstrapsamples,andcalculate[Zforeachsample.(Fornotation,Idenotebootstrapestimateswithastar,andhence[Zforabootstrapsampleisdenoted[ZG].)These\[ZG]’scontaininformationthatcanbeusedtomakeinferencesfromthedata;essentially,[ZG]isto[Zas[ZistoZ.Someofthetypesofinferencepos