An-interactive-approach-for-functional-prototype-r

ComputationalVisualMediaDOI10.1007/s41095-016-0032-xVol.2,No.1,March2016,87–96ResearchArticleAninteractiveapproachforfunctionalprototyperecoveryfromasingleRGBDimageYuliangRong1,YouyiZheng2,TianjiaShao1(),YinYang3,andKunZhou1cTheAuthor(s)2016.ThisarticleispublishedwithopenaccessatSpringerlink.comAbstractInferringthefunctionalityofanobjectfromasingleRGBDimageisdifficultfortworeasons:lackofsemanticinformationabouttheobject,andmissingdataduetoocclusion.Inthispaper,wepresentaninteractiveframeworktorecovera3DfunctionalprototypefromasingleRGBDimage.Insteadofpreciselyreconstructingtheobjectgeometryfortheprototype,wemainlyfocusonrecoveringtheobject’sfunctionalityalongwithitsgeometry.Oursystemallowsuserstoscribbleontheimagetocreateinitialroughproxiesfortheparts.Afteruserannotationofhigh-levelrelationsbetweenparts,oursystemautomaticallyjointlyoptimizesdetailedjointparameters(axisandposition)andpartgeometryparameters(size,orientation,andposition).Suchprototyperecoveryenablesabetterunderstandingoftheunderlyingimagegeometryandallowsforfurtherphysicallyplausiblemanipulation.Wedemonstrateourframeworkonvariousindoorobjectswithsimpleorhybridfunctions.Keywordsfunctionality;cuboidproxy;prototype;partrelations;shapeanalysis1IntroductionThatformeverfollowsfunction.Thisisthelaw.LouisSullivan1StateKeyLabofCAD&CG,ZhejiangUniversity,Hangzhou310058,China.E-mail:Y.Rong,[e-mail];T.Shao,[e-mail]();K.Zhou,[e-mail].2ShanghaiTechUniversity,Shanghai200031,China.E-mail:[e-mail].3TheUniversityofNewMexico,Albuquerque,NM87131,USA.E-mail:[e-mail]:2015-12-01;accepted:2015-12-09WiththepopularizationofcommercialRGBDcamerassuchasMicrosoft’sKinect,peoplecaneasilyacquire3DgeometryinformationwithanRGBimage.However,duetoocclusionandnoise,recoveringmeaningful3DcontentsfromsingleRGBDimagesremainsoneofthemostchallengingproblemsincomputervisionandcomputergraphics.Overthepastyears,muchresearchefforthasbeendevotedtorecoveringhigh-quality3DinformationfromRGBDimages[1,2].Mostoftheseapproaches,startingeitherfromasingleimageormultipleimages,arededicatedtofaithfullyrecoveringthe3Dgeometryofimageobjects,whiletheirsemanticrelations,underlyingphysicalsettings,orevenfunctionalityareoverlooked.Morerecently,researchhasexploreduseofhigh-levelstructuralinformationtofacilitate3Dreconstruction[3–5].Forexample,Shaoetal.[3]leveragephysicalstabilitytosuggestpossibleinteractionsbetweenimageobjectsandobtainaphysicallyplausiblereconstructionofobjectsinRGBDimages.Suchhigh-levelsemanticinformationplaysanimportantroleinconstrainingtheunderlyinggeometricstructure.Functionalityisthecenterofobjectdesignandunderstanding.Objectsinman-madeenvironmentsareoftendesignedforsingleormultipleintendedfunctionalities(seeFig.1).ThatformeverfollowsFig.1Objectsinman-madeenvironmentsareoftendesignedforsingleormultipleintendedfunctionalities.8788Y.Rong,Y.Zheng,T.Shao,etal.functionisthelawofphysicalmanufacturing[6].Inthispaper,wedevelopaninteractivesystemtorecoverfunctionalprototypesfromasingleRGBDimage.Ourgoalistoallowanoviceusertobeabletoquicklyliftimageobjectsinto3Dusing3Dprototypes,usingjustafewhigh-levelannotationsofjointtypesandgeometric/functionalrelations;theusercanmeanwhileexplicitlyexploreandmanipulateanobject’sfunction.Wefocusonprototypeswithsimpleproxies(e.g.,cuboids)representingtheirpartsasameanstoalleviatethedifficultiesofprecise3Dreconstructionwhichisaharderproblem.Bytakingphysicalfunctionalityintoconsideration,wemaygainamuchmorefaithfulinterpretationoftheunderlyingobjects.Thefunctionalpropertiescanthenbeusedforapplicationslikein-contextdesignandmanipulation.Itisachallengingproblemtoinferobjectfunctionjustfromuserannotatedjointtypesandgeometric/functionalrelations.Oursystemshouldautomaticallyoptimizethedetailedjointparameters(axisandposition)inordertomakethepartsmovecorrectly,whereasthistaskistypicallydoneinCADsoftwarebytheuserrepeatedlyadjustingparameters.Furthermore,initialproxiesfromuser-segmenteddepthsareoftenroughwithincorrectorientationandposition,andmaybeincompletebecauseofocclusion.HenceinitialproxiesoftenfailtosatisfythefunctionalrelationssuchasAcoversB.Thereforeoursystemshouldalsooptimizetheproxyparameters(size,orientation,andposition),inordertomakepartssatisfyfunctionalrelations.OurmethodstartswithasingleRGBDimage.Theusersegmentstheimageobjectintopartsbyscribblingontheimageusingsimplestrokesorpolygons.Theneachsegmentedpartisassembledusinga3Dproxy.Weusesimplecuboidsinthispaper[7].Giventheinitialproxies,oursystemthenallowstheusertoannotatethejointtypesandfunctional/geometricrelationsbetweenparts.Inakeystage,ouralgorithmsimultaneouslyoptimizesthedetailedjointparameters(axisandposition)andtheproxyparameters(size,orientation,andposition).Finally,afunctionalprototypeisproducedwithmovingpartssatisfyingtheuserannotatedrelations.Wehavetestedoursystemonavarietyofman-madehybridfunctionalobjectstakenfromvarioussources.Ourresultsshowthatevenusingonlyafewuserannotations,theproposedalgorithmiscapableoffaithfullyi