搜索
2.2FiltrationsLet(Ω;F)beameasureablespace.Afiltrationindiscretetimeisasequenceof¾–algebrasfFtgsuchthatFt½FandFt½Ft+1forallt=0;1;:::.Incontinuoustime,thesecondconditionisreplacedbyFs½Ftforalls·t.3MarkovprocessesTheideaofaMarkovprocessistocapturetheideaofashort-memorystochasticprocess:onceitscurrentstateisknown,pasthistoryisirrelevantfromthepointofviewofpredictingitsfuture.Definition.Let(Ω;F)beameasurablespaceandlet(P;F)be,respectively,aprobabilitymeasureonandafiltrationofthisspace.LetXbeastochasticprocessindiscretetimeon(Ω;F).ThenXiscalleda(P;F)-Markovprocessif1.XisF¡adapted,and112.Foreacht2Z+andeachBorelsetB½B(R)P(Xt+12BjFt)=P(Xt+12Bj¾(Xt)):(9)Sometimeswhentheprobabilitymeasureandfiltrationareunderstood,wewilltalksimplyofaMarkovprocess.Remark.OftenthefiltrationFistakentobethatgeneratedbytheprocessXitself.Proposition.Let(Ω;F;P;F)beafilteredprobabilityspaceandletXbea(P;F)-Markovprocess.Lett;k2Z+:Letf:R!RbeaBorelfunctionsuchthatf(Xt+k)isintegrable.Then;E[f(Xt+k)jFt]=E[f(Xt+k)j¾(Xt)](10)andhencethereisaBorelfunctiong:Z+£Z+£R!Rsuchthat,foreacht;E[f(Xt+k)jFt]=g(t;k;Xt):(11)Proof.Toshowitfork=1;usethatf(Xt+1)isa¾(Xt+1)¡measurablerandomvariableandhenceisthelimitofamonotoneincreasingsequenceof¾(Xt+1)¡simplerandomvariables.ButsuchrandomvariablesarelinearcombinationsofindicatorfunctionsofsetsX¡1t+1(B)withBaBorelset.Thiscompletestheprooffork=1:Toproveitforarbitrarypositivek,useinduction.Toproveitfork+1assumingittruefork,usethelawofiteratedexpectations.Thevectorcaseisasimpleextensionofthescalarcase.However,itisimportantthatthedefinitionofavectorMarkovprocessisnotthateachcomponentisMarkov.12Instead,werequirethatalltherelevant(forthefutureofX)bitsofinformationinFtareinthe¾¡algebrageneratedbyallthestochasticvariablesinXt;i.e.¾(Xt)isdefinedasthesingle¾¡algebra¾(fX1;t;X2;t;:::;Xn;tg).Thismeansthat,foreachBorelfunctionf:Rn!Rm,E[f(Xt+k)jFt]=E[f(Xt+k)j¾(Xt)](12)andhencethatthereisaBorelfunctiong:Z+£Rn!Rmsuchthat,foreacht,E[f(Xt+1)jFt]=g(t;Xt):(13)(Thecasek=1issoimportantthatwestressitherebyignoringgreatervaluesofk.)3.0.1ProbabilitytransitionfunctionsandtimehomogeneityDefinition.Let(Ω;F;P;F)beafilteredprobabilityspaceandletXbea(P;F)-Markovprocess.Then,foreacht=0;1;2:::itsprobabilitytransitionfunctionQt:R£B(R)![0;1]isdefinedviaQt(x;B)=P(Xt2BjXt¡1=x):(14)NotethatanyMarkovprocesshasasequenceofprobabilitytransitionfunctions.Notealsothatforeachfixedtandx,Qt+1(x;¢)isaprobabilitymeasureonB(R):Meanwhile,ifwefixB,Qt+1(Xt(¢);B)isarandomvariable.Indeed,itisthecon-ditionalprobabilityofXt+12BgivenXt;i.e.Qt+1(Xt;B)=EhIX¡1t+1(B)¯¯¯¾(Xt)i.Moreover,theconditionalexpectationofany¾(Xt+1)-measurablerandomvariable(givenXt)isanintegralwithrespecttothemeasureQt+1inthefollowingsense.13Proposition.Let(Ω;F;P;F)beafilteredprobabilityspaceandletXbea(P;F)-Markovprocess.LethQtibeitsprobabilitytransitionfunctionsandletZ2L1(Ω;¾(Xt+1);P).Then,foreacht=0;1;:::E[ZjXt]=ZRf(y)Qt+1(Xt;dy)(15)or,putdifferently,wehaveforeacht=0;1;:::andeachx;E[ZjXt=x]=ZRf(y)Qt+1(x;dy):(16)Proof.WewillshowitfirstforanindicatorvariableZ=IX¡1t+1(A)whereA2B(R):Thenf(y)=IA(y):WenowneedtoshowthattherandomvariableZRf(y)Qt+1(Xt;dy)qualifiesastheconditionalexpectationE[ZjXt]:Clearlyitis¾(Xt)¡measurable.Butdoesitintegratetotherightthing?Well,letG2¾(Xt)andrecallthat,bydefinition,Qt+1(Xt;A)=E³IX¡1t+1(A)j¾(Xt)´:HenceZGZRf(y)Qt+1(Xt;dy)P(d!)=ZGZRIA(y)Qt+1(Xt;dy)P(d!)==ZGQt+1(Xt;A)P(d!)=ZGE³IX¡1t+1(A)j¾(Xt)´P(d!)=ZGIX¡1t+1(A)P(d!):(17)Meanwhile,sinceZ=IX¡1t+1(A)weobviouslyhaveZGZP(d!)=ZGIX¡1t+1(A)P(d!).(18)14ToshowthetheoremforanarbitraryZ2L1(Ω;¾(Xt+1);P);usetheMonotoneConvergenceTheorem.WenowusetheprobabilitytransitionfunctiontodefineatimehomogeneousMarkovprocess.Definition.Let(Ω;F;P;F)beafilteredprobabilityspaceandletXbea(P;F)-Markovprocess.LethQti1t=1beitsprobabilitytransitionfunctions.IfthereisaQsuchthatQt=Qforallt=1;2;:::thenXiscalledatimehomogeneousMarkovprocess.Proposition.Let(Ω;F;P;F)beafilteredprobabilityspaceandletXbeatimehomogeneous(P;F)-Markovprocess.Foranynonnegativeintegersk;t;letYt+k2L1(Ω;¾(Xt+k);P).Thenforeachk=0;1;:::thereisaBorelfunctiongk:R!Rsuchthat,foreacht=0;1;:::E[Yt+kjFt]=gk(Xt):(19)Inparticular,thereisaBorelfunctionhsuchthat,foreacht=0;1;:::E[Yt+1jFt]=h(Xt):(20)3.1Finite–stateMarkovchainsindiscretetimeThisisperhapsthesimplestclassofMarkovprocesses.Let(Ω;F;P)beaprobabilityspaceandletX=fx1;x2;:::;xngbeafiniteset.X:Z+!Xbeastochasticprocess.Denoteby¹tthevectorofprobabilitiesthatXt=xiandsupposethereis15asequenceofmatricesΓtsuchthat¹t+1=Γt¹t:(21)ThenXissaidtobeafinite–stateMarkovchainindiscretetime.CallΓttheprobabilitytransitionmatrix.Theinterpretationof(21)isthefollowing.P(Xt+1=xijXt=xj)=Γt(i;j):IfΓt=Γwecalltheprocessandtime–homogenenousorstationary.IfΓissufficientlywell–behaved,thenXhasauniquestationarydistribution.Definition.LetXbeastationaryfinite–stateMarkovchainandletTbethetimeofthefirstvisittostatejaftert=0.Thenstatejiscalledrecurrent(opposite:transient)ifP(fT1gjX0=xj)=1:Definition.Thejiscalledperiodicwithperiod±1if±isthelargestintegerforwhichP(fT=n±forsomen¸1gjX0=j)=1:Ifthereisnosuch±1,thenjiscalled