Face Recognition Using Eigenfaces

cFaceRecognitionUsingEigenfacesMatthewA.TurkandAlexP.PentlandVisionandModelingGroup,TheMediaLaboratoryMassachusettsInstituteofTechnologyAbstractWepresentanapproachtothedetectionandidentificationofhumanfacesanddescribeawork-ing,near-real-timefacerecognitionsystemwhichtracksasubject’sheadandthenrecognizestheper-sonbycomparingcharacteristicsofthefacetothoseofknownindividuals.Ourapproachtreatsfacerecognitionasatwo-dimensionalrecognitionprob-lem,takingadvantageofthefactthatfacesarearenormallyuprightandthusmaybedescribedbyasmallsetof2-Dcharacteristicviews.Faceimagesareprojectedontoafeaturespace(“facespace”)thatbestencodesthevariationamongknownfaceimages.Thefacespaceisdefinedbythe“eigen-faces”,whicharetheeigenvectorsofthesetoffaces;theydonotnecessarilycorrespondtoisolatedfea-turessuchaseyes,ears,andnoses.Theframeworkprovidestheabilitytolearntorecognizenewfacesinanunsupervisedmanner.1IntroductionDevelopingacomputationalmodeloffacerecogni-tionisquitedifficult,becausefacesarecomplex,multidimensional,andmeaningfulvisualstimuli.Theyareanaturalclassofobjects,andstandinstarkcontrasttosinewavegratings,the“blocksworld”,andotherartificialstimuliusedinhumanandcomputervisionresearch[l].Thusunlikemostearlyvisualfunctions,forwhichwemayconstructdetailedmodelsofretinalorstriateactivity,facerecognitionisaveryhighleveltaskforwhichcom-putationalapproachescancurrentlyonlysuggestbroadconstraintsonthecorrespondingneuralac-tivity.Wethereforefocusedourresearchtowardsdevel-opingasortofearly,preattentivepatternrecogni-tioncapabilitythatdoesnotdependuponhavingfullthree-dimensionalmodelsordetailedgeometry.Ouraimwastodevelopacomputationalmodeloffacerecognitionwhichisfast,reasonablysimple,andaccurateinconstrainedenvironmentssuchasanofficeorahousehold.Althoughfacerecognitionisahighlevelvisualproblem,thereisquiteabitofstructureimposedonthetask.Wetakeadvantageofsomeofthisstruc-turebyproposingaschemeforrecognitionwhichisbasedonaninformationtheoryapproach,seekingtoencodethemostrelevantinformationinagroupoffaceswhichwillbestdistinguishthemfromoneCH2983-5/91/0000/0586/$01.OO(01991IEEEanother.Theapproachtransformsfaceimagesintoasmallsetofcharacteristicfeatureimages,calledeigenfaces”,whicharetheprincipalcomponentsoftheinitialtrainingsetoffaceimages.Recognitionisperformedbyprojectinganewimageintothesnb-spacespannedbytheeigenfaces(“facespace”)andthenclassifyingthefacebycomparingitspositioninfacespacewiththepositionsofknownindividuals.Automaticallylearningandlaterrecognizingnewfacesispracticalwithinthisframework.Recogni-tionunderreasonablyvaryingconditionsisachievedbytrainingonalimitednumberofcharacteristicviews(e.g.,a“straighton”view,a45’view,andaprofileview).Theapproachhasadvantagesoverotherfacerecognitionschemesinitsspeedandsim-plicity,learningcapacity,andrelativeinsensitivitytosmallorgradualchangesinthefaceimage.1.1BackgroundandrelatedworkMuchoftheworkincomputerrecognitionoffaceshasfocusedondetectingindividualfeaturessuchastheeyes,nose,mouth,andheadoutline,anddefin-ingafacemodelbytheposition,size,andrelation-shipsamongthesefeatures.BeginningwithBled-soe’s[2]andKanade’s[3]earlysystems,anumberofautomatedorsemi-automatedfacerecognitionstrategieshavemodeledandclassifiedfacesbasedonnormalizeddistancesandratiosamongfeaturepoints.RecentlythisgeneralapproachhasbeencontinuedandimprovedbytherecentworkofYuilleetal.[4].Suchapproacheshaveprovendifficulttoextendtomultipleviews,andhaveoftenbeenquitefrag-ile.Researchinhumanstrategiesoffacerecogni-tion,moreover,hasshownthatindividualfeaturesandtheirimmediaterelationshipscompriseaninsuf-ficientrepresentationtoaccountfortheperformanceofadulthumanfaceidentification[5].Nonetheless,thisapproachtofacerecognitionremainsthemostpopularoneinthecomputervisionliterature.Connectionistapproachestofaceidentificationsepktocapturetheconfigurational,orgestalt-likenatureofthetask.FlemingandCottrell[6],build-ingonearlierworkbyKohonenandLahtio[7],usenonlinearunitstotrainanetworkviabackpropa-gationtoclassifyfaceimages.Stonham’sWISARDsystem[8]hasbeenappliedwithsomesuccesstobi-naryfaceimages,recognizingbothidentityandex-pression.Mostconnectionistsystemsdealingwithfacestrratthrinputimageasageneral2-Dpattern,“’586andcanmakenoexplicituseoftheconfigurationalpropertiesofaface.Onlyverysimplesystemshavebeenexploredtodate,anditisunclearhowtheywillscaletolargerproblems.RecentworkbyBurtetal.usesa“smartsensing”approachbasedonmultiresolutiontemplatematch-ing[9].Thiscoarse-to-finestrategyusesaspecial-purposecomputerbuilttocalculatemultiresolutionpyramidimagesquickly,andhasbeendemonstratedidentifyingpeopleinnear-real-time.Thefacemod-elsarebuiltbyhandfromfaceimages.2EigenfacesforRecognitionMuchofthepreviousworkonautomatedfacerecog-nitionhasignoredtheissueofjustwhataspectsofthefacestimulusareimportantforidentification,assumingthatpredefinedmeasurementswererele-vantandsufficient.Thissuggestedtousthataninformationtheoryapproachofcodinganddecod-ingfaceimagesmaygiveinsightintotheinformationcontentoffaceimages,emphasizingthesignificantlocalandglobal“features”.Su