Face-Recognition-A-Literature-Survey

FaceRecognition:ALiteratureSurveyW.ZHAOSarnoffCorporationR.CHELLAPPAUniversityofMarylandP.J.PHILLIPSNationalInstituteofStandardsandTechnologyANDA.ROSENFELDUniversityofMarylandAsoneofthemostsuccessfulapplicationsofimageanalysisandunderstanding,facerecognitionhasrecentlyreceivedsignificantattention,especiallyduringthepastseveralyears.Atleasttworeasonsaccountforthistrend:thefirstisthewiderangeofcommercialandlawenforcementapplications,andthesecondistheavailabilityoffeasibletechnologiesafter30yearsofresearch.Eventhoughcurrentmachinerecognitionsystemshavereachedacertainlevelofmaturity,theirsuccessislimitedbytheconditionsimposedbymanyrealapplications.Forexample,recognitionoffaceimagesacquiredinanoutdoorenvironmentwithchangesinilluminationand/orposeremainsalargelyunsolvedproblem.Inotherwords,currentsystemsarestillfarawayfromthecapabilityofthehumanperceptionsystem.Thispaperprovidesanup-to-datecriticalsurveyofstill-andvideo-basedfacerecognitionresearch.Therearetwounderlyingmotivationsforustowritethissurveypaper:thefirstistoprovideanup-to-datereviewoftheexistingliterature,andthesecondistooffersomeinsightsintothestudiesofmachinerecognitionoffaces.Toprovideacomprehensivesurvey,wenotonlycategorizeexistingrecognitiontechniquesbutalsopresentdetaileddescriptionsofrepresentativemethodswithineachcategory.Inaddition,relevanttopicssuchaspsychophysicalstudies,systemevaluation,andissuesofilluminationandposevariationarecovered.CategoriesandSubjectDescriptors:I.5.4[PatternRecognition]:ApplicationsGeneralTerms:AlgorithmsAdditionalKeyWordsandPhrases:Facerecognition,personidentificationAnearlierversionofthispaperappearedas“FaceRecognition:ALiteratureSurvey,”TechnicalReportCAR-TR-948,CenterforAutomationResearch,UniversityofMaryland,CollegePark,MD,2000.Authors’addresses:W.Zhao,VisionTechnologiesLab,SarnoffCorporation,Princeton,NJ08543-5300;email:[e-mail];R.ChellappaandA.Rosenfeld,CenterforAutomationResearch,UniversityofMaryland,CollegePark,MD20742-3275;email:{rama,ar}@cfar.umd.edu;P.J.Phillips,NationalInstituteofStandardsandTechnology,Gaithersburg,MD20899;email:[e-mail]/hardcopyofpartorallofthisworkforpersonalorclassroomuseisgrantedwith-outfeeprovidedthatthecopiesarenotmadeordistributedforprofitorcommercialadvantage,thecopyrightnotice,thetitleofthepublication,anditsdateappear,andnoticeisgiventhatcopyingisbypermissionofACM,Inc.Tocopyotherwise,torepublish,topostonservers,ortoredistributetolistsrequirespriorspecificpermissionand/orafee.c
2003ACM0360-0300/03/1200-0399$5.00ACMComputingSurveys,Vol.35,No.4,December2003,pp.399–458.400Zhaoetal.1.INTRODUCTIONAsoneofthemostsuccessfulapplicationsofimageanalysisandunderstanding,facerecognitionhasrecentlyreceivedsignifi-cantattention,especiallyduringthepastfewyears.Thisisevidencedbytheemer-genceoffacerecognitionconferencessuchastheInternationalConferenceonAudio-andVideo-BasedAuthentication(AVBPA)since1997andtheInternationalCon-ferenceonAutomaticFaceandGestureRecognition(AFGR)since1995,system-aticempiricalevaluationsoffacerecog-nitiontechniques(FRT),includingtheFERET[Phillipsetal.1998b,2000;Rizvietal.1998],FRVT2000[Blackburnetal.2001],FRVT2002[Phillipsetal.2003],andXM2VTS[Messeretal.1999]pro-tocols,andmanycommerciallyavailablesystems(TableII).Thereareatleasttworeasonsforthistrend;thefirstisthewiderangeofcommercialandlawenforcementapplicationsandthesecondistheavail-abilityoffeasibletechnologiesafter30yearsofresearch.Inaddition,theprob-lemofmachinerecognitionofhumanfacescontinuestoattractresearchersfromdis-ciplinessuchasimageprocessing,patternrecognition,neuralnetworks,computervision,computergraphics,andpsychology.Thestrongneedforuser-friendlysys-temsthatcansecureourassetsandpro-tectourprivacywithoutlosingouriden-tityinaseaofnumbersisobvious.Atpresent,oneneedsaPINtogetcashfromanATM,apasswordforacomputer,adozenotherstoaccesstheinternet,andsoon.Althoughveryreliablemethodsofbiometricpersonalidentificationexist,forTableI.TypicalApplicationsofFaceRecognitionAreasSpecificapplicationsVideogame,virtualreality,trainingprogramsEntertainmentHuman-robot-interaction,human-computer-interactionDrivers’licenses,entitlementprogramsSmartcardsImmigration,nationalID,passports,voterregistrationWelfarefraudTVParentalcontrol,personaldevicelogon,desktoplogonInformationsecurityApplicationsecurity,databasesecurity,fileencryptionIntranetsecurity,internetaccess,medicalrecordsSecuretradingterminalsLawenforcementAdvancedvideosurveillance,CCTVcontrolandsurveillancePortalcontrol,posteventanalysisShoplifting,suspecttrackingandinvestigationexample,fingerprintanalysisandretinaloririsscans,thesemethodsrelyonthecooperationoftheparticipants,whereasapersonalidentificationsystembasedonanalysisoffrontalorprofileimagesofthefaceisofteneffectivewithoutthepartici-pant’scooperationorknowledge.Someoftheadvantages/disadvantagesofdifferentbiometricsaredescribedinPhillipsetal.[1998].TableIlistssomeoftheapplica-tionsoffacerecognition.Commercialandlawenforcementap-plicationsofFRTrangefromstatic,controlled-formatphotographstoun