自动驾驶中的计算机视觉

ComputerVisionforAutonomousVehicles1Content1.ObjectDetection2.SemanticSegmentation3.Reconstruction4.Motion&PoseEstimation5.Tracking6.SceneUnderstanding21.ObjectDetection1.2DObjectDetection2.3DObjectDetectionfrom2DImages3.3DObjectDetectionfrom3DPoint4.PersonDetection3Reliabledetectionofobjectsisacrucialrequirementtorealizeautonomousdriving.Theobjectdetectionsinclude:1.ObjectDetectionApaperlistofobjectdetectionusingdeeplearningfrom2014tonow(2019)41.ObjectDetection5DisscusionObjectdetectionworksalreadyquitewellincaseofhighresolutionwithlittleocclusions.Remainingmajorproblemsacrosstasksaredetectionofsmallobjectsandhighlyoccludedobjects.Furthermore,alargeamountofdistantobjectsneedstobedetectedinsomecaseswhichisstillachallengingtaskformodernmethodssincetheamountofinformationprovidedbytheseobjectsisverylow.2.SemanticSegmentationThegoalofsemanticsegmentationistoassigneachpixelintheimagealabelfromapredefinedsetofcategories.ThetaskisillustratedinFigure12withallpixelofacertaincategorycolorizedinasspecificcolorinasceneoftheCityscapesdatasetbyCordtsetal.(2016)recordedinZurich.62.SemanticSegmentation2.1SemanticInstanceSegmentationThegoalofsemanticinstancesegmentationissimultaneousdetection,segmentationandclassificationofeveryindividualobjectinanimage.7Theinstancesegmentationtaskismuchmoredifficultthanthesemanticsegmentationtask.Eachinstanceneedtobecarefullyannotatedseparatelywhereasinsemanticsegmentationgroupsofonesemanticclasscanbeannotatedtogetherwhentheyoccurnexttoeachother.Inaddition,thenumberofinstancevariesgreatlybetweendifferentimages.2.SemanticSegmentation2.2SemanticSegmentationwithMultipleFramesAsautonomoussystemsaretypicallyequippedwithvideocameras,temporalcorrelationbetweenadjacentframescanbeexploitedtoimprovesegmentationaccuracy,efficiencyandrobustness.82.SemanticSegmentation2.3SemanticSegmentationof3DData2Dimageslackimportantinformationsuchasthe3Dshapeandscaleofobjectswhicharestrongcuesforobjectclasssegmentationandfacilitatethedetectionandseparationofindividualobjectinstances.92.SemanticSegmentation2.4SemanticSegmentationofStreetSideViewsOneimportantapplicationofsemanticsegmentationforautonomousvehiclesistosegmentstreet-sideimages(i.e.,buildingfacades)intoitscomponents(wall,door,window,vegetation,balcony,store,mailboxetc.).Suchsemanticsegmentationsareusefulforaccurate3Dreconstruction,memory-efficient3Dmapping,robustlocalizationaswellaspathplanning.102.SemanticSegmentation2.5SemanticSegmentationofAerialImagesTheaimofaerialimageparsingistheautomatedextractionofurbanobjectsfromdataacquiredbyairbornesensors.Aerialimageparsescanbeusedtoautomaticallybuildroadmaps(eveninremoteareas)andkeepthemup-to-date.Furthermore,informationfromaerialimagescanbeusedforlocalization.112.SemanticSegmentation2.6RoadSegmentation12Segmentationofroadscenesisacrucialproblemincomputervisionforapplicationssuchasautonomousdrivingandpedestriandetection.Forinstance,inordertonavigate,anautonomousvehicleneedstodeterminethedrivablefreespaceaheadanddetermineitsownpositionontheroadwithrespecttothelanemarkings.However,theproblemischallengingduetothepresenceofavarietyofdifferentlyshapedobjectssuchascarsandpeople,differentroadtypesandvaryingilluminationandweatherconditions.3.Reconstruction134.Motion&PoseEstimation142DMotionEstimation–OpticalFlow3DMotionEstimation–SceneFlowEgo-MotionEstimation5.TrackingIntracking,thegoalistoestimatethestateofoneormultipleobjectsovertimegivenmeasurementsofasensor.Challenges:1.Objectsarepartiallyorfullyoccludedbyotherobjectsorthemselves.2.Theresemblanceofdifferentobjects.3.Theinteractionofobjectsincaseofpedestriansfurtherincreasestheamountofocclusionsandmakesitdifficulttotrackeachindividualobject.4.Difficultlightingconditionsandreflectionsinmirrorsor.windows156.SceneUnderstandingOneofthebasicrequirementsofautonomousdrivingistofullyunderstanditssurroundingareasuchasacomplextrafficscene.Thecomplextaskofoutdoorsceneunderstandinginvolvesseveralsub-taskssuchasdepthestimation,scenecategorization,objectdetectionandtracking,eventcategorization,andmore.16ChallengeHowtoimprovegeneralizationabilityofthemodel.（如何提高模型的泛化能力）Howdoyoumakeuseofsmall-scaleandsuper-large-scaledata（如何利用小规模和超大规模数据）Comprehensivesceneunderstanding（全面场景理解）Automaticnetworkdesign（自动化网格设计）1718