机器视觉技术与应用

数字视频技术---机器视觉技术与应用数字视频技术视频与图像信息无处不在PersonalphotoalbumsSurveillanceandsecurityMovies,news,sportsMedicalandscientificimages计算机视觉与相关学科ComputerVisionImageProcessingMachineLearningArtificialIntelligenceRoboticsPsychologyNeuroscienceComputerGraphicsApplicationsofcomputervisionDriverassistance(collisionwarning,lanedeparturewarning,rearobjectdetection)FactoryinspectionMonitoringforsafety(Poseidon)Readinglicenseplates,checks,ZIPcodesSurveillanceAutonomousdriving,robotnavigationApplicationsofcomputervisionAssistivetechnologiesEntertainment(SonyEyeToy)MoviespecialeffectsDigitalcameras(facedetectionforsettingfocus,exposure)Visualsearch(MSRLincoln)1531561481521491471391461421501461441371251201191361461511641721751831881962002052082142142192171591511501481401381391291191048682899710711511813012813212814416016817918820020821322021221414914615314714714613299737887961051201381511451571631711651611461261571841902012152122142141451501541489691736886148126936772789610711712713113412715416616718319420019514314017515115315114712085677584839492817878918311712614417820020120320817512715918519619520614614413912379667483796964625850465454666080861081411911842001871231441751981991351301158764779079788581635557565370626168595884105168194196183131151185197128116927182941031018310188667090804239538873768211687971441881951901661712031351208483108127135115100927949857459000506952791571411008413618720620418920014410391115139147127918780724461842500050181456914216416711393130193199208203WorldbehindthepictureVisionasasourceofsemanticinformationslidecredit:Fei-Fei,Fergus&TorralbaObjectcategorizationskybuildingflagwallbannerbuscarsbusfacestreetlampslidecredit:Fei-Fei,Fergus&TorralbaSceneandcontextcategorization•outdoor•city•traffic•…slidecredit:Fei-Fei,Fergus&TorralbaQualitativespatialinformationslantedrigidmovingobjecthorizontalverticalslidecredit:Fei-Fei,Fergus&Torralbarigidmovingobjectnon-rigidmovingobjectChallenges:viewpointvariationChallenges:illuminationChallenges:scaleChallenges:deformationChallenges:occlusionChallenges:backgroundclutterChallenges:objectintra-classvariationI.EarlyvisionCamerasandsensorsLightandcolorLinearfilteringEdgedetection*=Featureextraction:cornerandblobdetectionBasicimageformationandprocessingslidecredit:Fei-Fei,Fergus&TorralbaII.“Mid-levelvision”Fitting:LeastsquaresHoughtransformRANSACAlignment•FittingandgroupingIII.Multi-viewgeometryProjectivestructurefrommotion:Herebedragons!StereoAffinestructurefrommotionTomasi&Kanade(1993)EpipolargeometryIV.RecognitionPatchdescriptionandmatchingClusteringandvisualvocabulariesBag-of-featuresmodelsClassificationV.AdvancedTopicsTimepermitting…SegmentationArticulatedmodelsFacedetectionMotionandtracking1.Extractfeatures2.Learn“visualvocabulary”3.Quantizefeaturesusingvisualvocabulary4.Representimagesbyfrequenciesof“visualwords”解读视觉世界的过程NormalizepatchDetectpatches[MikojaczykandSchmid’02][Mata,Chum,Urban&Pajdla,’02][Sivic&Zisserman,’03]ComputeSIFTdescriptor[Lowe’99]Slidecredit:JosefSivicFeatureextractionLearningthevisualvocabularyClustering…Imagerepresentation…..frequencycodewordsEdgedetection•Goal:Identifysuddenchanges(discontinuities)inanimageIntuitively,mostsemanticandshapeinformationfromtheimagecanbeencodedintheedgesMorecompactthanpixels•Ideal:artist’slinedrawing(butartistisalsousingobject-levelknowledge)Featureextraction:CornersandblobsImagealignmentApplication:ViewInterpolation~comanici/MSPAMI/msPamiResults.htmlMeanshiftsegmentation•Anadvancedandversatiletechniqueforclustering-basedsegmentationD.ComaniciuandP.Meer,MeanShift:ARobustApproachtowardFeatureSpaceAnalysis,PAMI2002.•ThemeanshiftalgorithmseeksamodeorlocalmaximumofdensityofagivendistributionChooseasearchwindow(widthandlocation)ComputethemeanofthedatainthesearchwindowCenterthesearchwindowatthenewmeanlocationRepeatuntilconvergenceMeanshiftalgorithmRegionofinterestCenterofmassMeanShiftvectorMeanshiftSlidebyY.Ukrainitz&B.SarelRegionofinterestCenterofmassMeanShiftvectorMeanshiftSlidebyY.Ukrainitz&B.SarelRegionofinterestCenterofmassMeanShiftvectorMeanshiftSlidebyY.Ukrainitz&B.SarelRegionofinterestCenterofmassMeanShiftvectorMeanshiftSlidebyY.Ukrainitz&B.SarelRegionofinterestCenterofmassMeanShiftvectorMeanshiftSlidebyY.Ukrainitz&B.SarelRegionofinterestCenterofmassMeanShiftvectorMeanshiftSlidebyY.Ukrainitz&B.Sarel~comanici/MSPAMI/msPamiResults.htmlMeanshiftsegmentationresults混合高斯背景重建算法Applicationsofsegmentationtovideo•BackgroundsubtractionAstaticcameraisobservingasceneGoal:separatethestaticbackgroundfromthemovingforegroundUsesofmotion•Estimating3Dstructure•Segmentingobjectsbasedonmotioncues•Learningdynamicalmodels•Recognizingeventsandactivities•Improvingvideoquality(motionstabilization)Motionfield•Themotionfieldistheprojectionofthe3DscenemotionintotheimageOpticalflow•Definition:opticalflowistheapparentmotionofbrightnesspatternsintheimage•Ideally,opticalflowwouldbethesameasthemotionfield•Havetobecareful:apparentmotioncanbecausedbylightingchangeswithoutanyactualmotionThinkofau