Figure1:Successfulrecognitionofaseriesofgestures4thYearProjectReportHandGestureRecognitionUsingComputerVisionRayLocktonBalliolCollegeOxfordUniversitySupervisor:Dr.A.W.FitzgibbonDepartmentofEngineeringScienceRaymondLockton,BalliolCollege21Contents1CONTENTS................................................................................................................22INTRODUCTION......................................................................................................32.1REPORTOVERVIEW..............................................................................................42.2PROJECTSUMMARY..............................................................................................42.3EXISTINGSYSTEMS..............................................................................................53DETECTION..............................................................................................................63.1CHOICEOFSENSORS.............................................................................................63.2HARDWARESETUP................................................................................................73.3CHOICEOFVISUALDATAFORMAT........................................................................83.4COLOURCALIBRATION.........................................................................................93.5METHODOFCOLOURDETECTION........................................................................153.6CONCLUSION......................................................................................................154REFINEMENT.........................................................................................................174.1ANALYSISOFDISTORTION..................................................................................174.2REMOVALOFSKINPIXELSDETECTEDASWRISTBANDPIXELS.............................194.3REMOVALOFSKINPIXELSDETECTEDFROMFOREARM........................................194.4CONCLUSION......................................................................................................235RECOGNITION.......................................................................................................245.1CHOICEOFRECOGNITIONSTRATEGY..................................................................245.2SELECTIONOFTESTGESTURESET.......................................................................245.3ANALYSISOFRECOGNITIONPROBLEM................................................................255.4RECOGNITIONMETHOD1:AREAMETRIC............................................................255.5RECOGNITIONMETHOD2:RADIALLENGTHSIGNATURE......................................265.6RECOGNITIONMETHOD3:TEMPLATEMATCHINGINTHECANONICALFRAME......345.7REFINEMENTOFTHECANONICALFRAME............................................................405.8REFINEMENTOFTHETRAININGDATA.................................................................415.9METHODOFDIFFERENTIATION(INCANONICALFRAME)......................................435.10REFINEMENTOFTEMPLATESCOREMETHOD(NOQUANTIZATION).......................485.11CONCLUSION......................................................................................................516APPLICATION:GESTUREDRIVENINTERFACE............................................526.1SETUP.................................................................................................................526.2DEMONSTRATION...............................................................................................537CONCLUSION.........................................................................................................567.1PROJECTGOALS.................................................................................................567.2FURTHERWORK.................................................................................................568REFERENCES.........................................................................................................589APPENDIX...............................................................................................................599.1APPENDIXA-GLOSSARY....................................................................................599.2APPENDIXB-ENTIREGESTURESET...................................................................609.3APPENDIXC-ALGORITHMS................................................................................61RaymondLockton,BalliolCollege32IntroductionThisprojectwilldesignandbuildaman-machineinterfaceusingavideocameratointerprettheAmericanone-handedsignlanguagealphabetandnumbergestures(plusothersforadditionalkeyboardandmousecontrol).Thekeyboardandmousearecurrentlythemaininterfacesbetweenmanandcomputer.Inotherareaswhere3Dinformationisrequired,suchascomputergames,roboticsanddesign,othermechanicaldevicessuchasroller-balls,joysticksanddata-glovesareused.Humanscommunicatemainlybyvisionandsound,therefore,aman-machineinterfacewouldbemoreintuitiveifitmadegreateruseofvisiona