Continuous-Time Gauss-Markov Processes with Fixed

JournalofMathematicalSystems,Estimation,andControlc1994Birkhauser-BostonVol.4,No.4,1994,pp.1{24Continuous-TimeGauss-MarkovProcesseswithFixedReciprocalDynamicsAlessandroBeghiyAbstractContinuingtheworkstartedin[11],inthispaperweexaminetheconstructionofGauss-Markovprocesseswithxedreciprocaldynamics.WeshowhowtoconstructGauss-Markovprocesses,de-nedonaniteinterval,havingxedinitialandend-pointdensitiesandbelongingtoagivenreciprocalclass.Theproblemofchangingtheend-pointdensityofaMarkovprocess,whileremaininginthesamereciprocalclass,isalsoconsidered.Astochasticinterpretationoftheresultsintermsofanoptimalcontrolproblemisgiven.Keywords:Gaussianprocesses,reciprocalprocesses,stochasticoptimalcontrolAMSSubjectClassications:60G151IntroductionInthispaperwedescribeaprocedureforconstructingcontinuous-timeGauss-Markovprocesseswithxedreciprocaldynamics.Ananalogousstudyforthediscrete-timecasehasbeenpresentedin[11].AIRnvaluedstochasticprocessx(t)denedfort2I=[0;T]isreciprocalifforany[t1;t2]Itheprocessintheinteriorof[t1;t2]isconditionallyindependentoftheprocessinI[t1;t2]givenx(t1)andx(t2).Thetime-reversibilityoftheMarkovpropertyimpliesthatMarkovprocessesarenecessarilyre-ciprocal,whiletheconverseisfalse[7].AmotivationforinvestigatingtherelationsbetweenMarkovandreciprocalprocessescanbefoundintheworkofSchrodinger[17].Inhisattemptofgivingaprobabilisticinter-pretationtohisresultsinquantummechanics,SchrodingerhadtodealReceivedJanuary11,1993;receivedinnalformJune20,1993;summaryappearedinVolume4,Number4.yTheresearchdescribedinthispaperwaspartiallyperformedwhilevisitingtheInstituteofTheoreticalDynamcis,UniversityofCalifornia,DavisCA95616.1A.BEGHIwithMarkovprocesseslivingonaniteinterval,withpositivetransitiondensityq(s;x;t;y)andwiththepropertyofhavingpreassignedboththeinitialprobabilitydensityp0(x)andthenalone,pT(x).Clearly,fromtheknowledgeofthetransitiondensityandoftheinitialonewecanpredictthatatthetimet=Tthenalprobabilitydensityisp(T;x)=Zq(;;T;x)p0()d:(1.1)AninterestingquestionariseswhenpT(x)isdierentfromp(T;x):whichistheMarkovprocessx(t)thathasp0(x)andpT(x)asinitialandnaldensities,andisinsomewaytheclosesttox(t)?Inuencedbythisproblem,Bernsteinin1932introducedtheclassofreciprocalprocesses[1].Inhisfundamentalcontributiontothetheoryofreciprocalprocesses,Jamison[7,8,9]showedthat,similarlytowhathappensforMarkovprocesses,thenitejointdensitiesofareciprocalprocessx(t)canbedeterminedbyonlytwofunctions,namelythejointdensity(x0;0;xT;T)oftheprocessattwopointsandthereciprocal(orthree-point)transitiondensityq(r;x;s;y;t;w),whereZAZBq(r;x;s;y;t;w)dxdw=Prob(x(s)=yjx(r)2A;x(t)2B):(1.2)Twoprocesseshavingthesamethree-pointtransitiondensityaresaidlo-callyequivalent[8],[3],andthentheassignmentofthefunctionq(r;x;s;y;t;w)denesanequivalenceclassofprocesses.JamisonalsoshowedthatthereciprocaltransitiondensityofaMarkovprocesscanbeobtainedfromitsMarkovtransitiondensityviathefollowingfactorization:q(s;x;t;y;u;w)=p(s;x;t;y)p(t;y;u;w)p(s;x;u;w);0stuT:(1.3)Thefactorization(1.3)isnotunique.Infact,therearemanyMarkoviantransitiondensitiespthatgivethesamereciprocaltransitiondensityq,i.e.therearemanyMarkovprocesseshavingthesamereciprocaldescription.InthispaperwestudyhowtheseMarkovprocessesarerelatedtoeachother.WewilllimitourdiscussiontotheGaussiancontext,whereaninterestingcharacterizationofreciprocalprocessesisavailable.Infact,ithasbeenshownbyKrener,LevyandFrezza[12],[10]thatalltheGaussianreciprocalprocessesinagivenclasscanbeobtainedvaryingtheboundaryconditionsofaself-adjoint,second-orderstochasticboundary-valueproblem(SBVP).ThemaingoalofthispaperistoshowhowtoconstructaMarkovprocessbelongingtoagivenreciprocalclasswithprescribedinitialandnaldensities.WepresentamethodthatrequiresthesolutionofanalgebraicRiccatiequation,whichisobtainedbystudyingthepropertiesofthesetofboundaryconditionsfortheSBVPcharacterizingthereciprocalclass2MARKOVPROCESSESWITHFIXEDRECIPROCALDYNAMICSthatyieldMarkovprocesses.WeconsideralsotheproblemofchangingthenaldensityofaMarkovprocesswhileremaininginthesamereciprocalclass.Recently,dierentauthors[19,5,2]haveproposedaformulationofSchrodinger’sproblemintermsofastochasticoptimalcontrolproblem.Morespecically,theprocessx(t)whichhasnalprobabilitydensitypT(x)isconsideredtobeobtainedbyapplyingtox(t)aminimumenergycontrol.Weshowthatourresultsareinagreementwiththesepreviousones.Thepaperisorganizedasfollows.InSection2wereviewthecharac-terizationofGaussianreciprocalprocessesproposedbyKrener,FrezzaandLevy[10],andinSection3weintroduceanalternativesetofboundarycon-ditionsfortheSBVPconsideredin[10].TheprocedureforconstructingaMarkovprocesswithgivenend-pointmarginaldensitiesandreciprocaldy-namicsispresentedinSection4.InSection5weaddresstherelatedprob-lemofchangingtheend-pointdensityofaMarkovprocesswhileremaininginthesamereciprocalclass.Finally,weconcludegivinganinterpretationoftheresultsintermsofastochasticoptimalcontrolproblem.2ModelsofGaussianReciprocalProcessesKrener,FrezzaandLevyshowedin[10]that,undersuitableassumptionswhichwewilldisc