Monocular Precrash Vehicle Detection Features and

IEEETRANSACTIONSONIMAGEPROCESSING,VOL.15,NO.7,JULY20062019MonocularPrecrashVehicleDetection:FeaturesandClassifiersZehangSun,GeorgeBebis,andRonaldMillerAbstract—Robustandreliablevehicledetectionfromimagesac-quiredbyamovingvehicle(i.e.,on-roadvehicledetection)isanim-portantproblemwithapplicationstodriverassistancesystemsandautonomous,self-guidedvehicles.Thefocusofthisworkisontheissuesoffeatureextractionandclassificationforrear-viewvehicledetection.Specifically,bytreatingtheproblemofvehicledetectionasatwo-classclassificationproblem,wehaveinvestigatedseveraldifferentfeatureextractionmethodssuchasprincipalcomponentanalysis,wavelets,andGaborfilters.Toevaluatetheextractedfea-tures,wehaveexperimentedwithtwopopularclassifiers,neuralnetworksandsupportvectormachines(SVMs).Basedonoureval-uationresults,wehavedevelopedanon-boardreal-timemonoc-ularvehicledetectionsystemthatiscapableofacquiringgrey-scaleimages,usingFord’sproprietarylow-lightcamera,achievinganaveragedetectionrateof10Hz.Ourvehicledetectionalgorithmconsistsoftwomainsteps:amultiscaledrivenhypothesisgener-ationstepandanappearance-basedhypothesisverificationstep.Duringthehypothesisgenerationstep,imagelocationswherevehi-clesmightbepresentareextracted.Thisstepusesmultiscaletech-niquesnotonlytospeedupdetection,butalsotoimprovesystemrobustness.Theappearance-basedhypothesisverificationstepver-ifiesthehypothesesusingGaborfeaturesandSVMs.ThesystemhasbeentestedinFord’sconceptvehicleunderdifferenttrafficconditions(e.g.,structuredhighway,complexurbanstreets,andvaryingweatherconditions),illustratinggoodperformance.IndexTerms—Gaborfilters,neuralnetworks(NNs),principalcomponentanalysis(PCA),supportvectormachines(SVMs),ve-hicledetection,wavelets.I.INTRODUCTIONEVERYminute,onaverage,atleastonepersondiesinave-hiclecrash.Autoaccidentsalsoinjureatleasttenmillionpeopleeachyear,andtwoorthreemillionofthemseriously.Thehospitalbill,damagedproperty,andothercostswilladdupto1%–3%oftheworld’sgrossdomesticproduct[1].EachyearintheUnitedStates,motorvehiclecrashesaccountforabout40,000deaths,morethanthreemillioninjuries,andover$130billioninfinanciallosses[2].Thelossistoostartlingtobeig-nored.Withtheaimofreducinginjuryandaccidentseverity,ManuscriptreceivedNovember29,2004;revisedOctober14,2005.ThisworkwassupportedinpartbytheFordMotorCompanyunderGrant2001332R;inpartbytheUniversityofNevada,Reno,underanAppliedResearchInitia-tive(ARI)Grant;andinpartbytheNationalScienceFoundationunderCRCDGrant0088086.TheassociateeditorcoordinatingthereviewofthismanuscriptandapprovingitforpublicationwasDr.ZoltanKato.Z.SunwaswiththeComputerVisionLaboratory,UniversityofNevada,Reno,NV89557USA.HeisnowwitheTreppidTechnologies,LLC,Reno,NV89521USA(e-mail:[e-mail]).G.BebisiswiththeComputerVisionLaboratory,DepartmentofComputerScienceandEngineering,UniversityofNevada,Reno,NV89557USA(e-mail:[e-mail]).R.MilleriswiththeVehicleDesignR&ADepartment,FordMotorCompany,Dearborn,MI48126-2798USA(e-mail:[e-mail]).DigitalObjectIdentifier10.1109/TIP.2006.877062Fig.1.Varietyofvehicleappearancesposesabigchallengeforvehicledetection.precrashsensingisbecominganareaofactiveresearchamongautomotivemanufacturers,suppliersanduniversities.Severalnationalandinternationalprojectshavebeenlaunchedoverthepastseveralyearstoinvestigatenewtechnologiesforimprovingsafetyandaccidentprevention.Vehicleaccidentstatisticsdisclosethatthemainthreatsadriverisfacingarefromothervehicles.Consequently,anon-boardautomotivedriverassistancesystemaimingtoalertadriveraboutdrivingenvironments,andpossiblecollisionwithothervehicleshasattractedalotofattention.Inthesesystems,robustandreliablevehicledetectionisthefirststep—asuccessfulvehicledetectionalgorithmwillpavethewayforvehiclerecognition,vehicletracking,andcollisionavoidance.Thefocusofthispaperisontheproblemofopticalsensorbasedvehicledetection.Acomprehensivereviewonon-roadvehicledetectionsystemscanbefoundin[3],whilemoregeneraloverviewsofintelligentdriverassistancesystemscanbefoundin[4],[5],[2].Vehicledetectionbasedonopticalsensorsisverychal-lengingduetohugewithin-classvariabilities.Forexample,vehiclesmayvaryinshape[Fig.1(a)],size,andcolor.Also,vehicleappearancedependsonitspose[Fig.1(b)]andisaffectedbynearbyobjects.Complexoutdoorenvironments,e.g.,illuminationconditions[Fig.1(c)],clutteredbackground,andunpredictableinteractionsbetweentrafficparticipants[Fig.1(d)]aredifficulttocontrol.Usingon-boardmovingcamerasmakessomewellestablishedtechniques,suchasback-groundsubtraction,unsuitable.Moreover,on-boardvehicle1057-7149/$20.00©2006IEEE2020IEEETRANSACTIONSONIMAGEPROCESSING,VOL.15,NO.7,JULY2006Fig.2.Illustrationofthetwo-stepvehicledetectionstrategy.detectionsystemshavehighcomputationalrequirements.Theyneedtobeabletoprocessacquiredimagesatreal-timeorclosetoreal-timetosavemoretimefordriverreaction.A.Two-StepSchemeAlthoughitisachallengingtasktobeaccomplished,manyopticalsensorbasedvehicledetectionalgorithmsandsystemshavebeenproposedandimplemented.Themajorityofthemfollowtwobasicsteps:1)hypothesisgeneration(HG)wherethelocationsofp