基于图像的大规模三维重建

TowardsLargeScale3DReconstructionfr[e-mail]3DReconstructionfromImagesOverviewIntroductionLargescale3DreconstructionpipelineLargescaleStructure-from-MotionLargescaleMultipleViewStereoConclusionTowardsLargeScale3DReconstructionfromImages3DReconstructionReconstruct3DmodelsofscenesandobjectsbymultipleimagesautomaticallyInputOutputTowardsLargeScale3DReconstructionfromImagesWhy3DReconstruction?Store:completespaceinformationBrowse:completeandrealfeelingofsceneswithmorefreedomforusersReproduce:automaticallymodelingtoolfor3Dmodelerandsourcefor3DprinterApplication:DigitalmuseumsandcitiesVirtualrealityMovieindustryMappingandnavigationTowardsLargeScale3DReconstructionfromImagesAltizureAcustomcloudplatformfor3DreconstructionUsercanuploadtheircapturedimagestoobtain3DreconstructionresultsautomaticallyProvidesolutionsforlargescale3DreconstructionTowardsLargeScale3DReconstructionfromImagesLargeScale3DReconstructionSourceInternetimagesHighresolutionimageryandautomaticimagingdevicesScaleNumberofimages:102to106Resolution:10Mto100MpixelsLargescalechallengesAmbiguityScalabilityRedundancyMixedqualityTowardsLargeScale3DReconstructionfromImagesMergeLargeScale3DReconstructionPipelineLargescaleimagesFeatureDetectionImageRetrieval…DBFeatureMatchingRelativePosesCameraRegistrationCameraPosesandSparsePointsGlobalBundleAdjustmentIMUGPSGCPMulti-sensorFusionCameraSelectionandClusteringDenseReconstructionDensePointsDenseReconstructionDenseReconstructionMergeSurfaceSurfaceRecon-structionStructure-from-Motion(SfM)MultipleViewStereo(MVS)TowardsLargeScale3DReconstructionfromImagesStructure-from-MotionLargescaleimagesFeatureDetectionTowardsLargeScale3DReconstructionfromImagesFeatureDetectionTraditionalhandcraftfeatureSIFT,HarrisConner,etcLearningbasedfeatureLIFTLackofgeneralizationNotfor3DreconstructiontaskTowardsLargeScale3DReconstructionfromImagesGeoDescNetwork:L2-NetLearningfromSfMresultsandcoarsemeshtriangulatedfromsparsepointsTrainingsamplefrompatchessurvivedfromSfMandnon-survivedbutwithhighsimilaritiesPublication:Z.Luo,T.Shen,L.Zhou,S.Zhu,R.Zhang(Correspondingauthor),Y.Yao,T.Fang,andL.Quan,“GeoDesc:LearningLocalDescriptorsbyIntegratingGeometryConstraints,”inECCV2018TowardsLargeScale3DReconstructionfromImagesGeoDescIntroducegeometryinformation𝑆𝑝𝑎𝑡𝑐ℎ=𝑔∠𝐶𝑖𝑃𝐶𝑗,𝜎𝑖𝑔∠𝐶𝑖𝑃𝑃𝑛−∠𝐶𝑗𝑃𝑃𝑛,𝜎2Publication:Z.Luo,T.Shen,L.Zhou,S.Zhu,R.Zhang(Correspondingauthor),Y.Yao,T.Fang,andL.Quan,“GeoDesc:LearningLocalDescriptorsbyIntegratingGeometryConstraints,”inECCV2018TowardsLargeScale3DReconstructionfromImagesGeoDescTowardsLargeScale3DReconstructionfromImagesStructure-from-MotionLargescaleimagesFeatureDetectionImageRetrieval…..DatabaseTowardsLargeScale3DReconstructionfromImagesImageRetrievalforSfMFeaturebasedvocabularytreetoretrieverelativeimagesLearningbasedretrievalGoal:overlappingregioninsteadofsemanticsimilarityTrainingdata:hundredsofreconstructedmodelsinAltizureIntroducetrueoverlappinginformationtolossPublication:T.Shen,L.Zixin,L.Zhou,R.Zhang,S.Zhu,T.Fang,andL.Quan,“MatchableImageRetrievalbyLearningfromSurfaceReconstruction,”inACCV2018TowardsLargeScale3DReconstructionfromImagesStructure-from-MotionLargescaleimagesFeatureDetectionImageRetrieval…..DatabaseFeatureMatchingTowardsLargeScale3DReconstructionfromImagesGraph-basedMatchingGoalReducecandidatematchpairsbeforegeometryvalidationRobusttoambiguityPublication:T.Shen,S.Zhu,T.Fang,R.Zhang,andL.Quan,“Graph-basedconsistentmatchingforstructure-from-motion,”inECCV2016TowardsLargeScale3DReconstructionfromImagesGraph-basedMatchingMethodCameragraphinitializedbyaminimumspanningtreebasedretrievalrankingsGraphExpansionbyStrongTripletsCommunity-BasedGraphReinforcementPublication:T.Shen,S.Zhu,T.Fang,R.Zhang,andL.Quan,“Graph-basedconsistentmatchingforstructure-from-motion,”inECCV2016TowardsLargeScale3DReconstructionfromImagesStructure-from-MotionLargescaleimagesFeatureDetectionImageRetrieval…..DatabaseFeatureMatchingRelativePosesCameraRegistrationTowardsLargeScale3DReconstructionfromImagesCameraRegistrationRegisterallcamerasintoaglobalsystemIncrementalmethodsSelectnextviewbythenumberofpoints(Snavelyetal.,2006)Skeletalset(Snavelyetal.,2008b):minimumcamerasetrepresentingthewholesceneDistributedmatching®istration(Agarwaletal.,2011)AdvantageEasytofilteroutliersDisadvantageLocalinformationresultingindriftSlowTowardsLargeScale3DReconstructionfromImagesCameraRegistrationGlobalmethods(Motionaveraging)Minimize∑𝐷(𝑹𝑖𝒕𝑖,𝑹𝑗𝒕𝑗,𝑹𝑖𝑗𝒕𝑖𝑗)RotationAveraging(Avisheketal.,2013)TranslationAveraging(Jiangetal.,2013;Cui&Tan,2015)AdvantageGlobalinformationtocloseloopFastDisadvantageSensitivetooutliersTowardsLargeScale3DReconstructionfromImagesLargeScaleDistributedCameraRegistrationCombineincrementalandglobalregistrationDividecameragraphIncrementalregistrationinsubgraphsMotionaveragingtomergePublication:S.Zhu,T.Shen,L.Zhou,R.Zhang,T.Fang,andL.Quan,“Accurate