统计计算考试题目

统计计算题目4.2.EpidemiologistsareinterestedinstudyingthesexualbehaviorofindividualsatriskforHIVinfection.Suppose1500gaymenweresurveyedandeachwasaskedhowmanyriskysexualencountershehadintheprevious30days.Letindenotethenumberofrespondentsreportingiencounters,fori=1,...,16.Table4.2summarizestheresponses.ThesedataarepoorlyfittedbyaPoissonmodel.Itismorerealistictoassumethattherespondentscomprisethreegroups.First,thereisagroupofpeoplewho,forwhateverreason,reportzeroriskyencountersevenifthisisnottrue.Supposearespondenthasprobabilityofbelongingtothisgroup.Withprobability,arespondentbelongstoasecondgrouprepresentingtypicalbehavior.Suchpeoplerespondtruthfully,andtheirnumbersofriskyencountersareassumedtofollowaPoisson()distribution.Finally,withprobability1,arespondentbelongstoahigh-riskgroup.Suchpeoplerespondtruthfully,andtheirnumbersofriskyencountersareassumedtofollowaPoisson()distribution.Theparametersinthemodelare,,and.AtthetthiterationofEM,weuse),,,()()()()()(ttttttodenotethecurrentparametervalues.Thelikelihoodoftheobserveddataisgivenby160160!)(),,|(iniiinnL,whereexp)1(exp1)(0iiiifori=1,...,16.Theobserveddataare160,,nn.Thecompletedatamaybeconstruedtobe16,0,0,,,,ttznnn,and16,0,,,ppnn,whereikn,,denotesthenumberofrespondentsingroupkreportingiriskyencountersandtzk,andpcorrespondtothezero,typical,andpromiscuousgroups,respectively.Thus,0,0,0,0ptznnnnandipitinnn,,fori=1,...,16.Let.1500160iinNDefine,)()(00z,)(exp)(iiit)(exp)1()(iiipfori=0,...,16.Thesecorrespondtoprobabilitiesthatrespondentswithiriskyencountersbelongtothevariousgroups.a.ShowthattheEMalgorithmprovidesthefollowingupdates:,)()(001Nzntt,)(160)(1itiitNtn,)()(160)(160)()1(itiiitiittntin.)()(160)(160)()1(itiiitiitpnpinb.Estimatetheparametersofthemodel,usingtheobserveddata.c.Estimatethestandarderrorsandpairwisecorrelationsofyourparameterestimates,usinganyavailablemethod.解：（1）),|(),|()()2|()2(0)1|()1(1)()()|()|(),|()()()()()(ililillikikikkinkinknkinkinknkinkinknktinkzxfxfxpzxpzpzxpzpxpzpzxpzxzpzxzE则有;)()(00z;)()(iiiet;)()1()(iiiep其中nkz表示不同组，X表示危险性行为。即得证；（2）下证EM算法更新推导过程：计算E步的Q函数：160)(160)(160)(160)()(00131)()(131)()()!(ln)()!(ln)()1ln()(ln)(ln)()}|(ln()){ln(,|(],|)}|(ln(){ln([],|)|,([ln),(initiinitiitiiitiitikNikktnkztikNikknkztztiiiepietpntnznxpxzExxpzExzxpEQ（(3)计算M步的Q函数求极值过程：（i）由于要使Q函数达到最大，同时参数必须满足131kk，运用拉格朗日乘法可得NntinkzkxzEN1)(),(1从而有Nzntt)()(00)1(；160)()1()(itiitNtn；(ii）由于要使Q函数达到最大，即对),,,|,(lnzxp求偏导。0)()(),(160160)()()(iitiitiittntinQ从而有160160)()(itiiitiitntin；0)()(),(160160)()()(iitiitiitpnpinQ从而有160160)()(itiiitiipnpin；即得证。算法：（1）首先将混合正态模型的参数初始化为),,,()0()0()0()0()0(；（2）E步：通过混合正态分布进行随机模拟得到n个样本)...,,,(x21nxxx，计算完全数据对数似然)|,(logzxp关于数据z的期望值，对数似然函数的期望160)(160)(160)(160)()(00)()()!(ln)()!(ln)()1ln()(ln)(ln)(],|)|,([ln),(initiinitiitiiitiittztiiiepietpntnznxzxpEQ（3）M步：最优化期望值),()(tQ，即通过迭代找到),()(tQ的最大值)1(t；即,)()(001Nzntt,)(160)(1itiitNtn,)()(160)(160)()1(itiiitiittntin.)()(160)(160)()1(itiiitiitpnpin题目7.2SimulatingfromthemixturedistributioninEquation(7.6)isstraightforward[seepart(a)ofProblem7.1].However,usingtheMetropolis–Hastingsalgorithmtosimulaterealizationsfromthisdistributionisusefulforexploringtheroleoftheproposaldistribution.a.ImplementaMetropolis–HastingsalgorithmtosimulatefromEquation(7.6)with7.0,using)2^01.0),((txNastheproposaldistribution.Foreachofthreestartingvalues,x(0)=0,7,and15,runthechainfor10,000iterations.Plotthesamplepathoftheoutputfromeachchain.Ifonlyoneofthesamplepathswasavailable,whatwouldyouconcludeaboutthechain?Foreachofthesimulations,createahistogramoftherealizationswiththetruedensitysuperimposedonthehistogram.Basedonyouroutputfromallthreechains,whatcanyousayaboutthebehaviorofthechain?b.Nowchangetheproposaldistributiontoimprovetheconvergencepropertiesofthechain.Usingthenewproposaldistribution,repeatpart(a).算法：1.从两个正态总体里分别以0.7和0.3的概率产生100个随机模拟样本)...,,,(y21nyyy2.选取一个建议分布)1,0()|(.)(Ugt，从建议分布)|(）（tg中抽取一个候选值。3.计算Metropolis-Hastings比率)|()()|()(),()(*)(*)(**)(ttttgfgfR（通常实际中用贝叶斯推断得到的一个比率。2221)(2121)(2221*2121*)(**)()(exp)1()(exp)(exp)1()(exp)|()|(),(ititiittyyyyyLyLR4.以等于R的概率接受*，如果接受,则*)1(t,如果没有接受,则)()1(tt。5.增加t,重复上述过程，直到)(t收敛。题目7.5Aclinicaltrialwasconductedtodeterminewhetherahormonetreatmentbenefitswomenwhoweretreatedpreviouslyforbreastcancer.Eachsubjectenteredtheclinicaltrialwhenshehadarecurrence.Shewasthentreatedbyirradiationandassignedtoeitherahormonetherapygrouporacontrolgroup.Theobservationofinterestisthetimeuntilasecondrecurrence,whichmaybeassumedtofollowanexponentialdistributionwithparameter(hormonetherapygroup)or(controlgroup).Manyofthewomendidnothaveasecondrecurrencebeforetheclinicaltrialwasconcluded,sothattheirrecurrencet