Visual Odometry and Mapping for Autonomous Flight

VisualOdometryandMappingforAutonomousFlightUsinganRGB-DCameraAlbertS.Huang,AbrahamBachrach,PeterHenry,MichaelKrainin,DanielMaturana,DieterFox,NicholasRoyAbstractRGB-Dcamerasprovidebothacolorimageandper-pixeldepthesti-mates.Therichnessoftheirdataandtherecentdevelopmentoflow-costsensorshavecombinedtopresentanattractiveopportunityformobileroboticsresearch.Inthispaper,wedescribeasystemforvisualodometryandmappingusinganRGB-Dcamera,anditsapplicationtoautonomousflight.Byleveragingresultsfromrecentstate-of-the-artalgorithmsandhardware,oursystemenables3Dflightinclutteredenvironmentsusingonlyonboardsensordata.Allcomputationandsensingrequiredforlocalpositioncontrolareperformedonboardthevehicle,reducingthedepen-denceonunreliablewirelesslinks.Weevaluatetheeffectivenessofoursystemforstabilizingandcontrollingaquadrotormicroairvehicle,demonstrateitsuseforconstructingdetailed3Dmapsofanindoorenvironment,anddiscussitslimitations.1IntroductionStableandprecisecontrolofanautonomousmicroairvehicle(MAV)demandsfastandaccurateestimatesofthevehicle’sposeandvelocity.Inclutteredenvironmentssuchasurbancanyons,underaforestcanopy,andindoorareas,knowledgeofthe3Denvironmentsurroundingthevehicleisadditionallyrequiredtoplancollision-freetrajectories.Navigationsystemsbasedonwirelesslytransmittedinformation,suchasGlobalPositioningSystem(GPS)technologies,arenottypicallyusefulinAlbertS.Huang,AbrahamBachrach,andNicholasRoyMassachusettsInstituteofTechnology,ComputerScienceandArtificialIntelligenceLaboratory,CambridgeMA02139,e-mail:albert,abachrac,[e-mail],MichaelKrainin,andDieterFoxUniversityofWashington,DepartmentofComputerScience&Engineering,Seattle,WAe-mail:peter,mkrainin,[e-mail]entofComputerScience,PontificiaUniversidadCat´olicadeChile,Santiago,Chilee-mail:[e-mail].1Ourquadrotormicroairvehicle(MAV).TheRGB-Dcameraismountedatthebaseofthevehicle,tiltedslightlydown.thesescenariosduetolimitedrange,precision,andreception.Thus,theMAVmustestimateitsstateandplancollision-freetrajectoriesusingonlyitsonboardsensors.RGB-DcamerascaptureRGBcolorimagesaugmentedwithdepthdataateachpixel.Avarietyoftechniquescanbeusedforproducingthedepthestimates,suchastime-of-flightimaging,structuredlightstereo,densepassivestereo,laserrangescanning,etc.Whilemanyofthesetechnologieshavebeenavailabletoresearchersforyears,therecentapplicationofstructuredlightRGB-Dcamerastohomeenter-tainmentandgaming[32]hasresultedinthewideavailabilityoflow-costRGB-Dsensorswell-suitedforroboticsapplications.Inparticular,theMicrosoftKinectsen-sor,developedbyPrimeSense,providesa640480RGB-Dimageat30Hz.Whenstrippeddowntoitsessentialcomponents,theKinectweighs115g–lightenoughtobecarriedbyasmallMAV.Previously,wehavedevelopedalgorithmsforMAVflightinclutteredenviron-mentsusingLIDAR[3]andstereocameras[1].LIDARsensorscurrentlyavailableinformfactorsappropriateforuseonaMAVareveryhighprecision,butonlyproviderangemeasurementsalongaplanearoundthesensor.Sincetheycanonlydetectobjectsthatintersectthesensingplane,theyaremostusefulinenvironmentscharacterizedbyverticalstructures,andlesssoinmorecomplexscenes.StructuredlightRGB-Dcamerasarebaseduponstereotechniques,andthussharemanypropertieswithstereocameras.Theprimarydifferenceslieintherangeandspatialdensityofdepthdata.SinceRGB-Dcamerasilluminateascenewithanstructuredlightpattern,theycanestimatedepthinareaswithpoorvisualtexture,butarerange-limitedbytheirprojectors.Thispaperpresentsourapproachtoprovidinganautonomousmicroairvehiclewithfastandreliablestateestimatesanda3Dmapofitsenvironmentbyusinganon-boardRGB-Dcameraandinertialmeasurementunit(IMU).Together,theseal-lowtheMAVtosafelyoperateincluttered,GPS-deniedindoorenvironments.Thecontrolofamicroairvehiclerequiresaccurateestimationofnotonlythepositionofthevehiclebutalsothevelocity–estimatesthatouralgorithmsareabletoprovide.Estimatingavehicle’s3DmotionfromsensordatatypicallyconsistsofestimatingitsrelativemotionateachtimestepbyaligningsuccessivesensormeasurementssuchaslaserscansorRGB-Dframes,aprocessmostoftenknownas“visualodom-etry”whencomparingcameraorRGB-Dimages.TheprimarycontributionofthisVisualOdometryandMappingforAutonomousFlightUsinganRGB-DCamera3paperistoprovideasystematicexperimentalanalysisofhowthebestpracticesinvisualodometryusinganRGB-Dcameraenablethecontrolofamicroairvehicle.Givenknowledgeoftherelativemotionofthevehiclefromsensorframetosen-sorframe,the3Dtrajectoryofthevehicleintheenvironmentcanbeestimatedbyintegratingtherelativemotionestimatesovertime.Givenknowledgeofthevehiclepositionintheenvironment,thelocationsofobstaclesineachsensorframecanalsobeusedtoconstructaglobalmap.However,whileoftenusefulforlocalpositioncontrolandstability,visualodometrymethodssufferfromlong-termdriftandarenotsuitableforbuildinglarge-scalemaps.Tosolvethisproblem,wealsodemon-stratehowourpreviousworkonRGB-DMapping[14]canbeincorporatedtodetectloopclosures,correctforaccumulateddriftandmaintainarepresentationofconsis-tentposeestimatesoverthehistoryofpreviousframes.Wedes