A Bayesian factor analysis model with generalized

ABayesianFactorAnalysisModelWithGeneralizedPriorInformationDanielB.RoweDivisionofHumanitiesandSocialSciences,CaliforniaInstituteofTechnology,USAAbstractIntheBayesianapproachtofactoranalysis,availablepriorknowl-edgeregardingthemodelparametersisquantiedintheformofpriordistributionsandincorporatedintotheinferences.Theincor-porationofpriorknowledgehastheaddedconsequenceofeliminat-ingtheambiguityofrotationfoundinthetraditionalfactoranalysismodel.PreviousBayesianfactoranalysiswork(Press&Shigemasu1989,&Press1998,Rowe2000a,andRowe2000b),hasconsideredmainlynaturalconjugatepriordistributionsforthemodelparam-eters.AsismentionedinPress(1982),Rothenburg(1963)pointedoutthatthewithanaturalconjugatepriordistribution,theel-ementsinthecovariancematricesareconstrainedandthusmaynotberichenoughtopermitfreedomofassessment.Inthispa-per,generlizednaturalconjugatedistributionsareusedtoquantifyandincorporateavailablepriorinformationwhichpermitcompletefreedomofassessment.1IntroductionAfactoranalysisisperformedtoexplaintherelationshipamongasetofobservedvariablesintermsofasmallernumberofunobservedvariablesorlatentfactorswhichunderlietheobservations.Thissmallernumberofvariablescanbeusedtondameaningfulstructureintheobservedvariables.Thisstructurewillaidintheinterpretationandexplanationoftheprocessthathasgeneratedtheobservations.IntheBayesianapproachtofactoranlysis(Press&Shigemasu1989,Rowe&Press1998,Rowe2000a,andRowe2000bhenceforthPS89,RP98,R00a,andR00b)theclassicalnormalsamplingmodelisassumed,butthedisturbancecovariancematrixisspeciedtobeafullpositivedenitematrix.Oneofthepriorassumptions(PS89)isthatthepriorexpectedvalueofthedisturbancecovariancematrixisdiagonalinordertorepresenttraditionalviewsofthefactormodelcontaining\commonand\specicfactors.AsnotedinPress(1982),Rothenburg(1963)pointedoutthatthenaturalconjugatepriordistributionhascovariancematrixelementsthatareconstrainedandthusmaynotberichenoughtoassessthepriorparameters.Inthispaper,generalizednaturalconjugatepriordistributionsarespeciedfortheunknownmatriceswhichpermitcompletefreedomofassessment.Bayesianstatisticalmethodsnotonlyincorporateavailablepriorinfor-mationeitherfromsubstantiveexpertsorpreviousdata,butallowtheknowledgeregardingtheparametervaluestoaccumulateassubsequentdataisacquired.Inthenon-Bayesianfactoranalysismodel,thefactor2loadingmatrixisdeterminateuptoanorthogonalrotation.Typicallyaf-teranon-Bayesianfactoranalysis,anorthogonalrotationisperformedonthefactorloadingmatrixaccordingtooneofmanysubjectivecriteria.ThisisnotthecaseinBayesianfactoranalysis.Therotationisautomaticallyfound.ThemodelparametersareestimatedbybothGibbssampling(Geman&Geman,1984andGelfand&Smith,1990)anditeratedconditionalmodes(Lindley&Smith,1972andO’Hagen,1994)whichndposteriormarginalmeanandposteriorjointmodal(maximumaposteriori)estimatesrespec-tively.Theplanofthepaperistoreviewthemodelandtoadoptpriordistri-butionsinSection2.PresenttheconditionalposteriordistributionsalongwiththeGibbssamplingandICMalgorithmsinSection3.InSection4anexampleisdetailed,andestimatesfromboththeGibbssamplingandtheICMestimationmethodsarepresented.2Model2.1LikelihoodFunctionTheBayesianfactoranalysismodelis:(xjj;;fj)=+fj+j;mp;(p1)(p1)(pm)(m1)(p1)(2.1)forj=1;:::;n,wherexjisthejthobservation,istheoverallpopulationmean,isamatrixofconstantscalledthefactorloadingmatrix;fjisthefactorscorevectorforsubjectj;andthej’sareassumedtobemutually3uncorrelatedandnormallydistributedN(0;)variables.Inthetraditionalmodel,istakentobeadiagonalmatrixsothatcom-monandspecicfactorscanreadilydistinguished.TheBayesianmodels,taketobeageneralsymmetric,positivedenitecovariancematrixwiththepropertyofbeingaprioridiagonalontheaverage,i.e.,E()=adiagonalmatrix.Itisassumedthat,,thefi’s,andareunobservableandthatthedistributionofeachxjcanbewrittenasp(xjj;;fj;)=(2)p2jj12e12(xjfj)01(xjfj):(2.2)Ifproportionalityisdenotedby\/andtheKronekerproductbythen,thelikelihoodfor(;;F;)isp(Xj;;F;)/jjn2e12tr1(Xen0F0)0(Xen0F0)(2.3)wherethep-variateobservationvectorsonnsubjectsare,X0=(x1;:::;xn),thefactorscoresareF0=(f1;:::;fn),andtheerrorsofobservationareE0=(1;:::;n).Thenotationp()willgenericallydenoteadistributionwhichisdistinguishedbyitsargument.Theproportionalityconstantin(2.3)dependsonlyon(p;n)andnoton(;;F;).2.2PriorsGeneralizednaturalconjugatepriordistributionsarespeciedfortheunknownparameterswhichpermitcompletefreedomofassessment.Thejointpriordistributionis:4p(;;F;)/p()p()p()p(F);(2.4)wherep()/jj12e12(0)01(0)(2.5)p()/jj12e12(0)01(0);(2.6)p()/jj2e12tr1B;2p;(2.7)p(F)/e12trF0F(2.8)with;;B;0andBadiagonalmatrix.Ageneralizednaturalconju-gatenormaldistributionisspeciedforthepopulationmeanwhere0andaremeanandcovariancehyperparameterstobeassessed.Thevector=vec(0)isspeciedtohavethegeneralizednaturalconjugatenormaldistributionwithmeanandcovariancehyperparameters0=vec(00)and.ThematrixfollowsanInvertedWishartdistribution,withhyper-parameters(;B)whicharetobeassessed.It