ABSTRACT Texture Mapping Range Images

TextureMappingRangeImagesAndrasFerenczComputerScienceDivisionUniversityofCaliforniaatBerkeleyMay15,2001ABSTRACTWeareconcernedwithcapturingarealscenewithbothcamerasandlaserrangescan-ners,andcombiningthemtoproduceconvincingrendersings.Inthispaper,wefocusonthetexturemappingcomponentofthispipeline.Ourautomaticalgorithmsregistertexturefromradianceimageswiththegeometryandsynthesizecompacthigh-qualitytexturemaps.Ourimageregistrationtechniqueperformsaveryefficientsearchtoautomaticallyfindthecameraposesforarbitrarypositionandorientationrelativetothegeometry.Thuswecantakephotographsfromanylocationwithoutprecalibrationbetweenthescannerandthecamera.Allthephotographsarecommbinedtoformonetexturemapforthewholescene,whichisrenderedontopofourreconstructedgeometricmesh.Thesealgorithmshavebeenappliedtolarge-scalerealdata.Additionally,wedemonstrateourabilitytoeditthiscapturedscenebymoving,inserting,anddeletingobjects.IndexTerms:SceneEditing,Object-LevelRepresentation,RangeImageSegmentation,ImageRegistration,Texture-Mapping,Image-BasedModeling,Image-BasedRendering,AugmentedReality1IntroductionCapturingrealenvironmentstofaithfullyrecreatethemonacomputerscreenhasbe-comeanimportantresearcharea.Mostoftheworkinthisfield,image-basedmodelingandrendering[27,6,32,47,28,18,11,41,40,42,45,51],hasfocusedonstaticen-vironmentsthatcanbeviewedfromnovelviewpointsaswellasundernovellightingAndrasFerencziswithComputerScienceDivision,UniversityofCalifornia,Berkeley,CA94720.E-mail:[e-mail]oneincollaborationwithYizhouYuandProfessorJitendraMalik.1conditions.However,challengesremaininmakingmodificationstogeometricprop-erties,suchastherelativeposition,orientationandsizeofobjects,andphotometricproperties,suchascolororspecularity.Forexample,wewouldliketoanimatetheob-jectsintheenvironment;ormoveastatuetoadifferentplaceinavirtualizedmuseum.Toallowsuchediting,anobjectshouldbemadeupofacollectionofsurfaceswhichinturnhavegeometricpropertiessuchassizeandshapeaswellasphotometricpropertiessuchascolorandtexture.Editingoperationsshouldbeperformedatobjectlevel,whichrequiresustogiveeachobjectgeometricandphotometricrepresentationsthatareindependentoftherestofthescene.Sinceasceneisusuallyacquiredasawhole,thiskindofobject-levelinformationisnotdirectlyavailablefromthecapturedgeometryorfromphotographs.Thereareseveralbasicproblemsrelatedtothisissue.Here,wefocusontheprob-lemofattachingconvincingtexturepropertiestotheobjects.Weuseadigitalcameratocapturesuchinformationandthenrecoverthecameraposes.Inthisway,wecansetupcorrespondencesbetweenpixelsinthephotographsand3Dpointsinthescene.Usingcalibrationtargets,wedevelopedanautomatictechniqueforrecoveringcameraposeforarbitrarypositionandorientationrelativetothegeometry.Weefficientlysearchforthecorrectmatchesbetweenthedetectedcalibrationtargetsinthesetwotypesofim-agesandthensolvealeast-squaresproblemtorecovertheparametersofcamerapose.Thistechniquehasmuchbetteraveragetimecomplexitythanpreviousalgorithms[26]inthesamecategory.Withcorrectlyregisteredimages,wecombinealltheavailablepicturestosynthesizespaceefficienttexturemapscanbeusedforhardwaretexture-mapping.Weextenendourbasicalgorithmtosmoothlybalancethecolorpropertiesofaregionfromseveralphotographsandtocompressthethetexturemapssuchthattheyaremoreefficentforrendering.1.1OverviewTheinputtoourpipelineisasetofrangeimagesandphotographs.TherangeimagesareregisteredtogetherfirsttogiveaunifiedpointcloudbyusingCyraTechnologies’software[9]toautomaticallylocatecalibrationtargetsineachscanandinteractivelysettingupcorrespondingtargetsamongdifferentscans.Therotationandtranslationbetweentwolaserscanscanberecoveredfromthreepairsofcorrespondences,butthemorethebetter.Thecalibrationtargetsaredesignedtobestronglyretroreflectiveatthewavelengthofthelaserbeaminordertobeidentifiedautomatically.Thesegmentationalgorithmisthenrunonthepointcloud,breakingitintogroups.Withsomeuserinteraction,wecanassemblethesegroupsintoobjects.Wethenbuildameshforeachobjectandrunmeshsimplificationtoreduceitscomplexity.Atthesametime,thecameraposesofthephotographsarerecoveredautomaticallyrelativetotheunifiedpointcloud.Toattachdetailedtextureinformationtotheobjects,wecomposetexturemapsusingdatafrommultiplephotographsforalltheobjects.Intheend,wecanrealisticallyre-renderthesceneusingtheextractedgeometricandphotometricpropertiesandmanipulatetheobjectsaswewish.2PointMeshesImagesTextureImagesRangePointCloudGroupsMeshesMapsObjectsRadianceImagesSegmentationPoseEstimationReconstructionRegistrationCalibratedSimplifiedFigure1:PipelineThisfigureshowsthemultiplestagesinourdataprocessingproce-dure.2PreviousWorkTheworkwepresentinthispaperhasbeenmadepossiblebypreviousworkingeom-etryacquisition,meshreconstructionandsimplification,image-basedrenderingandtexture-mapping,2Dimageregistrationandsegmentation,andrangeimageregistra-tionandsegmentation.Recentworkinlaserrangescanninghasmadeitpossibletorecoveraccurategeometryofreal-worldscenes.[3]introducedtheiterativeclosestpoint(ICP)algorithmtoregistermultiplerangeimages.[37]addressed