ism(implicit-shape-mode)

1ComputerVision:[e-mail]n-ComputerInteractionResearchGroup,UniversitätKarlsruhe(TH)cv:hciDr.EdgarSeemann2LocalFeaturesComputerVisionforHuman-ComputerInteractionResearchGroup,UniversitätKarlsruhe(TH)cv:hciDr.EdgarSeemann3SofarPartsweredefinedmanuallyPartsrepresentedthesemanticstructurei.e.face,legetc.Questions:Dothesepartsdecomposethevariabilityinanoptimalway?MustthepartshaveasemanticmeaningShouldweusesmaller/largerparts?Canwefindpartsautomatically?ComputerVisionforHuman-ComputerInteractionResearchGroup,UniversitätKarlsruhe(TH)cv:hciDr.EdgarSeemann4RequirementsforpartdecompositionRepeatablei.e.weshouldbeabletofindthepartdespitearticulationorimagetransformations(e.g.rotation,perspective,lighting)DistinctivePartshouldnotbeconfoundedwithotherpartsTheregionsshouldcontainan“interesting”structureCompactTypicallynolengthyorstrangelyshapedpartsEfficientItshouldbecomputationallyinexpensivetodetectorrepresentpartCoverpartsneedtosufficientlycovertheobjectComputerVisionforHuman-ComputerInteractionResearchGroup,UniversitätKarlsruhe(TH)cv:hciDr.EdgarSeemann5ApproachNpixelsNpixelsSimilaritymeasureAfe.g.colorBfe.g.colorA1A2A3TffdBA),(1.Findasetofdistinctivekey-points3.Extractandnormalizetheregioncontent2.Definearegionaroundeachkeypoint4.Computealocaldescriptorfromthenormalizedregion5.MatchlocaldescriptorsComputerVisionforHuman-ComputerInteractionResearchGroup,UniversitätKarlsruhe(TH)cv:hciDr.EdgarSeemann6HessianDetector[Beaudet78]HessiandeterminantIxxIyyIxy2))(det(xyyyxxIIIIHessian2)^(.xyyyxxIIIInMatlab:yyxyxyxxIIIIIHessian)(ComputerVisionforHuman-ComputerInteractionResearchGroup,UniversitätKarlsruhe(TH)cv:hciDr.EdgarSeemann7AutomaticScaleSelectionFunctionresponsesforincreasingscale(scalesignature))),((1xIfmii)),((1xIfmiiComputerVisionforHuman-ComputerInteractionResearchGroup,UniversitätKarlsruhe(TH)cv:hciDr.EdgarSeemann8Results:Laplacian-of-GaussianComputerVisionforHuman-ComputerInteractionResearchGroup,UniversitätKarlsruhe(TH)cv:hciDr.EdgarSeemann9ImplicitShapeModelComputerVisionforHuman-ComputerInteractionResearchGroup,UniversitätKarlsruhe(TH)cv:hciDr.EdgarSeemann1010K.Grauman,B.LeibeImplicitShapeModel(ISM)Basicideas1.AutomaticallylearnalargenumberoflocalpartsthatoccurontheobjectAlsoreferredtoasvisualvocabularyorappearancecodebook2.Learnastar-topologystructuralmodelFeaturesareconsideredindependentgivenobj.centerx1x3x4x6x5x2ComputerVisionforHuman-ComputerInteractionResearchGroup,UniversitätKarlsruhe(TH)cv:hciDr.EdgarSeemann11VisualVocabulary/AppearanceCodebookComputerVisionforHuman-ComputerInteractionResearchGroup,UniversitätKarlsruhe(TH)cv:hciDr.EdgarSeemann12VisualVocabularyDetectkeypointsonalltrainingexamplesExtractfeaturedescriptionsaroundkeypointsResult:AlargesetoflocalimagedescriptorsoccurringonpeopleComputerVisionforHuman-ComputerInteractionResearchGroup,UniversitätKarlsruhe(TH)cv:hciDr.EdgarSeemann13VisualVocabularyGroupvisuallysimilarlocaldescriptorsi.e.partsthatarereoccurringParts,thatoccuronlyoncearediscarded(theycouldresultfromnoiseorunusualstructures)ComputerVisionforHuman-ComputerInteractionResearchGroup,UniversitätKarlsruhe(TH)cv:hciDr.EdgarSeemann14SideNote:GroupingAlgorithmsPartitionalClusteringK-MeansGaussianMixtureClustering(EM)HierarchicalorAgglomerativeClusteringSingle-LinkGroupAverageWard’smethod(minimumvariance)ComputerVisionforHuman-ComputerInteractionResearchGroup,UniversitätKarlsruhe(TH)cv:hciDr.EdgarSeemann16ComplexityStandardApproach:Timecomplexity:O(n2logn)ComputedistancematrixConsecutivelymergethetwomostsimilarclustersSpacecomplexity:O(n2)RNNAlgorithm[deRham’80,Benzecri’82]Timecomplexity:O(n2)Spacecomplexity:O(n)Requirement:“reducibilityproperty”[Bruynooghe’77]ComputerVisionforHuman-ComputerInteractionResearchGroup,UniversitätKarlsruhe(TH)cv:hciDr.EdgarSeemann17SpaceComplexityNote,thatspacecomplexityisquiteimportantforclusteringlargedatasetsExample:100000datapointsStandarddistancematrixcontains:105*105/2=1010/2entries-~20GBifoneentryhas32bit-DoesyourPChaveenoughRAM?ComputerVisionforHuman-ComputerInteractionResearchGroup,UniversitätKarlsruhe(TH)cv:hciDr.EdgarSeemann18AgglomerativeclusteringproducesahierarchyDifficultquestion:wheretostop?Ideally,clustersshouldbevisuallycompact.ButDistancevaluedependsonfeaturedimensionality.Appropriateratio#features/#clustersdependsondatasetandinterestpointdetector.Needstobeselectedforeachdetector/descriptorcombination!ClusteringHierarchyComputerVisionforHuman-ComputerInteractionResearchGroup,UniversitätKarlsruhe(TH)cv:hciDr.EdgarSeemann19VisualVocabularyVocabularysize~10000clustersProbabilisticvotesdecide,whetherpartisimportantornotComputerVisionforHuman-ComputerInteractionResearchGroup,UniversitätKarlsruhe(TH)cv:hciDr.EdgarSeemann20LearningSpatialStructure:“Star”-ModelComputerVisionforHuman-ComputerInteractionResearchGroup,UniversitätKarlsruhe(TH)cv:hciDr.EdgarSeemann211.LearnappearancecodebookExtractlocal