P. Multi-camera people tracking with a probabilist

1Multi-CameraPeopleTrackingwithaProbabilisticOccupancyMapFranc¸oisFleuretJ´erˆomeBerclazRichardLengagnePascalFua´EcolePolytechniqueF´ed´eraledeLausanneLausanne,Switzerland{francois.fleuret,jerome.berclaz,pascal.fua}@[e-mail]npartbytheSwissFederalOfficeforEducationandScienceandinpartbytheIndoSwissJointResearchProgramme(ISJRP).March27,2007DRAFTAbstractGiventwotofoursynchronizedvideostreamstakenateyelevelandfromdifferentangles,weshowthatwecaneffectivelycombineagenerativemodelwithdynamicprogrammingtoaccuratelyfollowuptosixindividualsacrossthousandsofframesinspiteofsignificantocclusionsandlightingchanges.Inaddition,wealsoderivemetricallyaccuratetrajectoriesforeachoneofthem.Ourcontributionistwofold.First,wedemonstratethatourgenerativemodelcaneffectivelyhandleocclusionsineachtimeframeindependently,evenwhentheonlydataavailablecomesfromtheoutputofasimplebackgroundsubtractionalgorithmandwhenthenumberofindividualsisunknownapriori.Second,weshowthatmulti-persontrackingcanbereliablyachievedbyprocessingindividualtrajectoriesseparatelyoverlongsequences,providedthatareasonableheuristicisusedtoranktheseindividualsandavoidconfusingthemwithoneanother.Fig.1IMAGESFROMTWOINDOORANDTWOOUTDOORMULTI-CAMERAVIDEOSEQUENCESWEUSEFOROUREXPERIMENTS.ATEACHTIMESTEP,WEDRAWABOXAROUNDPEOPLEWEDETECTANDASSIGNTOTHEMANIDNUMBERTHATFOLLOWSTHEMTHROUGHOUTTHESEQUENCE.3I.INTRODUCTIONInthispaper,weaddresstheproblemofkeepingtrackofpeoplewhooccludeeachotherusingasmallnumberofsynchronizedvideossuchasthosedepictedbyFig.1,whichweretakenatheadlevelandfromverydifferentangles.Thisisimportantbecausethiskindofsetupisverycommonforapplicationssuchasvideo-surveillanceinpublicplaces.Tothisend,wehavedevelopedamathematicalframeworkthatallowsustocombinearobustapproachtoestimatingtheprobabilitiesofoccupancyofthegroundplaneatindividualtimestepswithdynamicprogrammingtotrackpeopleovertime.Thisresultsinafullyautomatedsystemthatcantrackupto6peopleinaroomforseveralminutesusingonlyfourcameras,withoutproducinganyfalsepositivesorfalsenegativesinspiteofsevereocclusionsandlightingvariations.AsshowninFig.2,oursystemalsoprovideslocationestimatesthatareaccuratetowithinafewtensofcentimetersandthereisnomeasurableperformancedecreaseifasmanyas20%oftheimagesarelost,andonlyasmalloneif30%are.Thisinvolvesthetwofollowingalgorithmicsteps:1)Weestimatetheprobabilitiesofoccupancyofthegroundplanegiventhebinaryimagesobtainedfromtheinputimagesviabackgroundsubtraction[FLF05].Atthisstage,thealgorithmonlytakesintoaccountimagesacquiredatthesametime.Itsbasicingredientisagenerativemodelthatrepresentshumansassimplerectanglesthatitusestocreatesyntheticidealimageswewouldobserveifpeoplewereatgivenlocations.Underthismodeloftheimagegiventhetrueoccupancy,weapproximatetheprobabilitiesofoccupancyateverylocationasthemarginalsofaproductlawminimizingtheKullback-Leiblerdivergencefromthe“true”conditionalposteriordistribution.Thisallowsustoevaluatetheprobabilitiesofoccupancyateverylocationasthefixedpointofalargesystemofequations.2)WethencombinetheseprobabilitieswithacolorandamotionmodelanduseaViterbialgorithmtoaccuratelyfollowindividualsacrossthousandsofframes[BFF06].Toavoidthecombinatorialexplosionthatwouldresultfromexplicitlydealingwiththejointposteriordistributionofthelocationsofindividualsineachframeoverafinediscretization,weuseagreedyapproach:Weprocesstrajectoriesindividuallyoversequencesthatarelongenoughsothatusingareasonableheuristictochoosetheorderinwhichtheyareprocessedissufficienttoavoidconfusingpeoplewitheachother.March27,2007DRAFT4Incontrasttomoststate-of-the-artalgorithmsthatrecursivelyupdateestimatesfromframetoframeandmaythereforefailcatastrophicallyifdifficultconditionspersistoverseveralconsecu-tiveframes,ouralgorithmcanhandlesuchsituations,sinceitcomputesglobaloptimaofscoressummedovermanyframes.ThisiswhatgivesittherobustnessthatFig.2demonstrates.Inshort,wecombineamathematicallywell-foundedgenerativemodelthatworksineachframeindividuallywithasimpleapproachtoglobaloptimization.Thisyieldsexcellentperformanceusingbasiccolorandmotionmodelsthatcouldbefurtherimproved.Ourcontributionisthereforetwofold.First,wedemonstratethatagenerativemodelcaneffectivelyhandleocclusionsateachtimeframeindependentlyevenwhentheinputdataisofverypoorquality,andthereforeeasytoobtain.Second,weshowthatmulti-persontrackingcanbereliablyachievedbyprocessingindividualtrajectoriesseparatelyoverlongsequences.00.20.40.60.81020406080100P(errordistance)Error(cm)20%imagesdeleted30%imagesdeleted40%imagesdeletedNoimagesdeletedFig.2CUMULATIVEDISTRIBUTIONSOFTHEPOSITIONESTIMATEERRORONA3800-FRAMESEQUENCE.SEE§VI-D.1,PAGE25FORDETAILS.Intheremainderofthepaper,wefirstbrieflyreviewrelatedworks.WethenformulateourproblemasestimatingthemostprobablestateofahiddenMarkovprocessandproposeamodelofthevisiblesignalbasedonanestimateofanoccupancymapineverytimeframe.Finally,wepresentourresultsonseverallongsequences.March27,2007DRAFT5II.RELATEDWORKState-of-the-artmethodscanbedividedintomonocularandmulti-viewapproachesthatwebrieflyreviewinthi