Comparison of Edge Detectors A Methodology and Ini

ComparisonofEdgeDetectors:AMethodologyandInitialStudy1MikeHeath,SudeepSarkar,ThomasSanockiy,andKevinBowyerheath,sarkar,sanocki,[e-mail]&EngineeringyDepartmentofPsychologyUniversityofSouthFloridaUniversityofSouthFloridaTampa,Florida33620Tampa,Florida33620Abstract.Becauseofthedicultyofobtaininggroundtruthforrealimages,thetraditionaltechniqueforcomparinglow-levelvisionalgorithmsistopresentimageresults,sidebyside,andtoletthereadersubjectivelyjudgethequality.Thisisnotascienticallysatisfactorystrategy.However,humanratingexperimentscanbedoneinamorerigorousmanner,toprovideusefulquantita-tiveconclusions.Wepresentaparadigmbasedonexperimentalpsychologyandstatistics,inwhichhumansratetheoutputoflowlevelvisionalgorithms.Wedemonstratetheproposedexperimentalstrategybycomparingfourwellknownedgedetectors:Canny,Nalwa-Binford,Sarkar-Boyer,andSobel.Weanswerthefollowingquestions:Isthereastatisticallysignicantdierenceinedgedetectoroutputsasperceivedbyhumans?Dotheedgedetectionresultsofanoperatorvarysignicantlywiththechoiceofitsparameters?Foreachdetector,isitpossibletochooseasinglesetofoptimalparametersforalltheimageswithoutsignicantlyaectingtheedgeoutputquality?Doesanedgedetectorproduceedgesofthesamequalityforallimages,ordoestheedgequalityvarywiththeimage?Keywords:EdgeDetection,Low-LevelProcessing,Segmentation,PerformanceEvaluation1ThisworkissupportedbyAirForceOceofScienticResearchgrantF49620-92-J-0223andNationalScienceFoundationgrantsCDA-9424214,DBS-9213246,andIRI-9501932.1ListofTables1EdgedetectionalgorithmsinPAMI(Jan93-June95),SMC(Jan93-Aug95),R&A(Apr94-June95),CVGIP(Jan90-July95),IJCV(Jan90,Dec94),PR(Jan93-July95).Thenumberofimagesiscountedfromtheimagespresentedinthepaper.Groundtruthiscountedasobjectivespecicationofcorrectedgepixels.Thelastcolumnliststheedgealgorithmsconsideredinthecomparison.:32CombinationsofParameterSettingsfortheEdgeDetectors:::::::::::103ICC(3,k)valuesforthejudges’ratingsoftheedgeimages.Thesecondrowliststhe95%condenceintervals.:::::::::::::::::::::::::::::134AnalysisofVarianceforRatingsofEdgeImages.Thesecondcolumnliststhedegreesoffreedom(DF).Thethirdcolumnliststhetotalsumofsquares(SS).Thefourthandthefthcolumnslistthe!2andF-valuesrespectively.Andthesixthcolumnliststhesignicancelevels.::::::::::::::::::::::145ANOVAresultsforedgedetectorratings.Thecolumnslistthedegreesoffreedom(DF),sumofsquares(SS),computedFvalue,andthesignicancelevel.::::186Averageratingoftheedgedetectorratingsforxedandadaptedparameter.::187Averageratingforindividualimagesforxedandadaptedparameters.:::::198ANOVAresultsforedgedetectorratingsforxedandadaptedparameters,con-sideredseparately.:::::::::::::::::::::::::::::::::::209ANOVAresultsforpairwiseconsiderationsofedgedetectorratingsforxedpa-rameterschoices.SourceDistheedgedetector,sourceIistheimage,andsourceDIistheinteractionbetweentheedgedetectorandtheimage.:::::::::2110ANOVAresultsforpairwiseconsiderationsofedgedetectorratingsforadaptedparameterschoices.SourceDistheedgedetector,sourceIistheimage,andsourceDIistheinteractionbetweentheedgedetectorandtheimage.:::::232ListofFigures1Theeightimagesusedintheexperiments.:::::::::::::::::::::82ThehighestratingsfortheCannyandSobeledgeswereontheCoeeMakerimage.253ThehighestratingforSarkar-BoyeredgeswasontheTrashcanimage.:::::264ThehighestratingfortheNalwa-BinfordedgeswasontheConeimage.:::::273ContentsI.Introduction1II.RelatedWork2III.Methods5A.Edgedetectors::::::::::::::::::::::::::::::::::::5B.Images:::::::::::::::::::::::::::::::::::::::::6C.ANOVA::::::::::::::::::::::::::::::::::::::::7IV.Experiment#1:EdgeDetectorParameterSettings9A.Edgedetectorparametersettings::::::::::::::::::::::::::9B.Images:::::::::::::::::::::::::::::::::::::::::11C.Judgesfortheratingtask::::::::::::::::::::::::::::::11D.RatingTask::::::::::::::::::::::::::::::::::::::11E.Results:::::::::::::::::::::::::::::::::::::::::121.Aretheratingsofthejudgesconsistent?::::::::::::::::::122.Dotheratingsofanedgedetectorvarywiththeimage?:::::::::143.Doestheratingofanedgedetectorratingvarywithitsparameter?:::154.Isthereinteractionbetweenthechosenedgedetectorparametervalueandtheimage?::::::::::::::::::::::::::::::::15F.Summary:::::::::::::::::::::::::::::::::::::::16V.Experiment#2:ComparisonRatingofEdgeDetectors16A.Judgesfortheratingtask::::::::::::::::::::::::::::::17B.RatingTask::::::::::::::::::::::::::::::::::::::17C.Results:::::::::::::::::::::::::::::::::::::::::171.Isthereastatisticallysignicantdierencebetweenratingsforxedver-susadaptedparameters?:::::::::::::::::::::::::::1842.Isthereinteractionbetweenxedvs.adaptiveparametervaluesandtheedgedetectors?::::::::::::::::::::::::::::::::193.Forxedparameters,aretheredierencesinratingsbetweenedgedetectors?204.Forxedparameters,isthereinteractionbetweenthedetectorandtheimage?:::::::::::::::::::::::::::::::::::::225.Foradaptedparameters,istheredierenceinratingsbetweenedgedetec-tors?::::::::::::::::::::::::::::::::::::::226.Foradaptedparameters,arethereinteractionsbetweenedgedetectorsand