Bilateral-filtering-for-gray-and-color-images

BilateralFilteringforGrayandColorImagesC.TomasiR.ManduchiComputerScienceDepartmentInteractiveMediaGroupStanfordUniversityAppleComputer,Inc.Stanford,CA94305Cupertino,[e-mail]@apple.comAbstractProceedingsofthe1998IEEEInternationalConferenceonComputerVision,Bombay,IndiaBilateralfilteringsmoothsimageswhilepreservingedges,bymeansofanonlinearcombinationofnearbyimagevalues.Themethodisnoniterative,local,andsim-ple.Itcombinesgraylevelsorcolorsbasedonboththeirgeometricclosenessandtheirphotometricsimilarity,andprefersnearvaluestodistantvaluesinbothdomainandrange.Incontrastwithfiltersthatoperateonthethreebandsofacolorimageseparately,abilateralfiltercanen-forcetheperceptualmetricunderlyingtheCIE-Labcolorspace,andsmoothcolorsandpreserveedgesinawaythatistunedtohumanperception.Also,incontrastwithstandardfiltering,bilateralfilteringproducesnophantomcolorsalongedgesincolorimages,andreducesphantomcolorswheretheyappearintheoriginalimage.1IntroductionFilteringisperhapsthemostfundamentaloperationofimageprocessingandcomputervision.Inthebroadestsenseoftheterm“filtering,”thevalueofthefilteredimageatagivenlocationisafunctionofthevaluesofthein-putimageinasmallneighborhoodofthesamelocation.Inparticular,Gaussianlow-passfilteringcomputesaweightedaverageofpixelvaluesintheneighborhood,inwhich,theweightsdecreasewithdistancefromtheneighborhoodcen-ter.Althoughformalandquantitativeexplanationsofthisweightfall-offcanbegiven[11],theintuitionisthatimagestypicallyvaryslowlyoverspace,sonearpixelsarelikelytohavesimilarvalues,anditisthereforeappropriatetoaveragethemtogether.Thenoisevaluesthatcorruptthesenearbypixelsaremutuallylesscorrelatedthanthesignalvalues,sonoiseisaveragedawaywhilesignalispreserved.Theassumptionofslowspatialvariationsfailsatedges,whichareconsequentlyblurredbylow-passfiltering.Manyeffortshavebeendevotedtoreducingthisundesiredeffect[1,2,3,4,5,6,7,8,9,10,12,13,14,15,17].HowcanSupportedbyNSFgrantIRI-9506064andDoDgrantsDAAH04-94-G-0284andDAAH04-96-1-0007,andbyagiftfromtheCharlesLeePowellfoundation.wepreventaveragingacrossedges,whilestillaveragingwithinsmoothregions?Anisotropicdiffusion[12,14]isapopularanswer:localimagevariationismeasuredateverypoint,andpixelvaluesareaveragedfromneighborhoodswhosesizeandshapedependonlocalvariation.Diffusionmethodsaverageoverextendedregionsbysolvingpartialdifferentialequations,andarethereforeinherentlyiterative.Iterationmayraiseissuesofstabilityand,dependingonthecomputationalarchitecture,efficiency.Otherapproachesarereviewedinsection6.Inthispaper,weproposeanoniterativeschemeforedgepreservingsmoothingthatisnoniterativeandsimple.Al-thoughweclaimsnocorrelationwithneurophysiologicalobservations,wepointoutthatourschemecouldbeimple-mentedbyasinglelayerofneuron-likedevicesthatperformtheiroperationonceperimage.Furthermore,ourschemeallowsexplicitenforcementofanydesirednotionofphotometricdistance.Thisisparticularlyimportantforfilteringcolorimages.Ifthethreebandsofcolorimagesarefilteredseparatelyfromoneanother,colorsarecorruptedclosetoimageedges.Infact,differentbandshavedifferentlevelsofcontrast,andtheyaresmootheddifferently.Separatesmoothingperturbsthebalanceofcolors,andunexpectedcolorcombinationsappear.Bilateralfilters,ontheotherhand,canoperateonthethreebandsatonce,andcanbetoldexplicitly,sotospeak,whichcolorsaresimilarandwhicharenot.Onlyperceptuallysimilarcolorsarethenaveragedtogether,andtheartifactsmentionedabovedisappear.Theideaunderlyingbilateralfilteringistodointherangeofanimagewhattraditionalfiltersdoinitsdomain.Twopixelscanbeclosetooneanother,thatis,occupynearbyspatiallocation,ortheycanbesimilartoonean-other,thatis,havenearbyvalues,possiblyinaperceptuallymeaningfulfashion.Closenessreferstovicinityinthedo-main,similaritytovicinityintherange.Traditionalfilter-ingisdomainfiltering,andenforcesclosenessbyweighingpixelvalueswithcoefficientsthatfalloffwithdistance.Similarly,wedefinerangefiltering,whichaveragesimagevalueswithweightsthatdecaywithdissimilarity.Rangefiltersarenonlinearbecausetheirweightsdependonimageintensityorcolor.Computationally,theyarenomorecom-plexthanstandardnonseparablefilters.Mostimportantly,theypreserveedges,asweshowinsection4.Spatiallocalityisstillanessentialnotion.Infact,weshowthatrangefilteringbyitselfmerelydistortsanimage’scolormap.Wethencombinerangeanddomainfiltering,andshowthatthecombinationismuchmoreinteresting.Wedenotetehcombinedfilteringasbilateralfiltering.Sincebilateralfiltersassumeanexplicitnotionofdis-tanceinthedomainandintherangeoftheimagefunction,theycanbeappliedtoanyfunctionforwhichthesetwodistancescanbedefined.Inparticular,bilateralfilterscanbeappliedtocolorimagesjustaseasilyastheyareappliedtoblack-and-whiteones.TheCIE-Labcolorspace[16]endowsthespaceofcolorswithaperceptuallymeaningfulmeasureofcolorsimilarity,inwhichshortEuclideandis-tancescorrelatestronglywithhumancolordiscriminationperformance[16].Thus,ifweusethismetricinourbilat-eralfilter,imagesaresmoothedandedgesarepreservedinawaythatistunedtohumanperformance.Onlyperceptuallysimilarcolorsareaveragedtogether,andonlyperc