搜索
AtutorialonPrincipalComponentsAnalysisLindsayISmithFebruary26,2002Chapter1IntroductionThistutorialisdesignedtogivethereaderanunderstandingofPrincipalComponentsAnalysis(PCA).PCAisausefulstatisticaltechniquethathasfoundapplicationinfieldssuchasfacerecognitionandimagecompression,andisacommontechniqueforfindingpatternsindataofhighdimension.BeforegettingtoadescriptionofPCA,thistutorialfirstintroducesmathematicalconceptsthatwillbeusedinPCA.Itcoversstandarddeviation,covariance,eigenvec-torsandeigenvalues.ThisbackgroundknowledgeismeanttomakethePCAsectionverystraightforward,butcanbeskippediftheconceptsarealreadyfamiliar.Thereareexamplesallthewaythroughthistutorialthataremeanttoillustratetheconceptsbeingdiscussed.Iffurtherinformationisrequired,themathematicstextbook“ElementaryLinearAlgebra5e”byHowardAnton,PublisherJohnWiley&SonsInc,ISBN0-471-85223-6isagoodsourceofinformationregardingthemathematicalback-ground.1Chapter2BackgroundMathematicsThissectionwillattempttogivesomeelementarybackgroundmathematicalskillsthatwillberequiredtounderstandtheprocessofPrincipalComponentsAnalysis.Thetopicsarecoveredindependentlyofeachother,andexamplesgiven.Itislessimportanttoremembertheexactmechanicsofamathematicaltechniquethanitistounderstandthereasonwhysuchatechniquemaybeused,andwhattheresultoftheoperationtellsusaboutourdata.NotallofthesetechniquesareusedinPCA,buttheonesthatarenotexplicitlyrequireddoprovidethegroundingonwhichthemostimportanttechniquesarebased.IhaveincludedasectiononStatisticswhichlooksatdistributionmeasurements,or,howthedataisspreadout.TheothersectionisonMatrixAlgebraandlooksateigenvectorsandeigenvalues,importantpropertiesofmatricesthatarefundamentaltoPCA.2.1StatisticsTheentiresubjectofstatisticsisbasedaroundtheideathatyouhavethisbigsetofdata,andyouwanttoanalysethatsetintermsoftherelationshipsbetweentheindividualpointsinthatdataset.Iamgoingtolookatafewofthemeasuresyoucandoonasetofdata,andwhattheytellyouaboutthedataitself.2.1.1StandardDeviationTounderstandstandarddeviation,weneedadataset.Statisticiansareusuallycon-cernedwithtakingasampleofapopulation.Touseelectionpollsasanexample,thepopulationisallthepeopleinthecountry,whereasasampleisasubsetofthepop-ulationthatthestatisticiansmeasure.Thegreatthingaboutstatisticsisthatbyonlymeasuring(inthiscasebydoingaphonesurveyorsimilar)asampleofthepopulation,youcanworkoutwhatismostlikelytobethemeasurementifyouusedtheentirepop-ulation.Inthisstatisticssection,Iamgoingtoassumethatourdatasetsaresamples2ofsomebiggerpopulation.Thereisareferencelaterinthissectionpointingtomoreinformationaboutsamplesandpopulations.Here’sanexampleset: Icouldsimplyusethesymbol torefertothisentiresetofnumbers.IfIwanttorefertoanindividualnumberinthisdataset,Iwillusesubscriptsonthesymbol toindicateaspecificnumber.Eg. referstothe3rdnumberin ,namelythenumber4.Notethat isthefirstnumberinthesequence,not likeyoumayseeinsometextbooks.Also,thesymbol willbeusedtorefertothenumberofelementsintheset Thereareanumberofthingsthatwecancalculateaboutadataset.Forexample,wecancalculatethemeanofthesample.Iassumethatthereaderunderstandswhatthemeanofasampleis,andwillonlygivetheformula: !$# Noticethesymbol (said“Xbar”)toindicatethemeanoftheset .Allthisformulasaysis“Addupallthenumbersandthendividebyhowmanythereare”.Unfortunately,themeandoesn’ttellusalotaboutthedataexceptforasortofmiddlepoint.Forexample,thesetwodatasetshaveexactlythesamemean(10),butareobviouslyquitedifferent: &%’ & % ( *) + , Sowhatisdifferentaboutthesetwosets?Itisthespreadofthedatathatisdifferent.TheStandardDeviation(SD)ofadatasetisameasureofhowspreadoutthedatais.Howdowecalculateit?TheEnglishdefinitionoftheSDis:“Theaveragedistancefromthemeanofthedatasettoapoint”.Thewaytocalculateitistocomputethesquaresofthedistancefromeachdatapointtothemeanoftheset,addthemallup,divideby .- ,andtakethepositivesquareroot.Asaformula:/ !$# 10 - 32540 - 2Where/istheusualsymbolforstandarddeviationofasample.Ihearyouasking“Whyareyouusing0 6- 2andnot ?”.Well,theanswerisabitcomplicated,butingeneral,ifyourdatasetisasampledataset,ie.youhavetakenasubsetofthereal-world(likesurveying500peopleabouttheelection)thenyoumustuse0 7- 2becauseitturnsoutthatthisgivesyouananswerthatisclosertothestandarddeviationthatwouldresultifyouhadusedtheentirepopulation,thanifyou’dused .If,however,youarenotcalculatingthestandarddeviationforasample,butforanentirepopulation,thenyoushoulddivideby insteadof0 8- 2.Forfurtherreadingonthistopic,thewebpage: 0 - 320 - 3240-101008-2412242010100Total208Dividedby(n-1)69.333SquareRoot8.3266Set2: 0 - 320 - 3248-249-1111111224Total10Dividedby(n-1)3.333SquareRoot1.8257Table2.1:Calculationofstandarddeviationdifferencebetweeneachofthedenominators.Italsodiscussesthedifferencebetweensamplesandpopulations.So,forourtwodatasetsabove,thecalculationsofstandarddeviationa