搜索
600IEEETRANSACTIONSONIMAGEPROCESSING,VOL.13,NO.4,APRIL2004ImageQualityAssessment:FromErrorVisibilitytoStructuralSimilarityZhouWang,Member,IEEE,AlanConradBovik,Fellow,IEEE,HamidRahimSheikh,StudentMember,IEEE,andEeroP.Simoncelli,SeniorMember,IEEEAbstract—Objectivemethodsforassessingperceptualimagequalitytraditionallyattemptedtoquantifythevisibilityoferrors(differences)betweenadistortedimageandareferenceimageusingavarietyofknownpropertiesofthehumanvisualsystem.Undertheassumptionthathumanvisualperceptionishighlyadaptedforextractingstructuralinformationfromascene,weintroduceanalternativecomplementaryframeworkforqualityassessmentbasedonthedegradationofstructuralinformation.Asaspecificexampleofthisconcept,wedevelopaStructuralSimilarityIndexanddemonstrateitspromisethroughasetofintuitiveexamples,aswellascomparisontobothsubjectiveratingsandstate-of-the-artobjectivemethodsonadatabaseofimagescompressedwithJPEGandJPEG2000.1IndexTerms—Errorsensitivity,humanvisualsystem(HVS),imagecoding,imagequalityassessment,JPEG,JPEG2000,perceptualquality,structuralinformation,structuralsimilarity(SSIM).I.INTRODUCTIONDIGITALimagesaresubjecttoawidevarietyofdistortionsduringacquisition,processing,compression,storage,transmissionandreproduction,anyofwhichmayresultinadegradationofvisualquality.Forapplicationsinwhichimagesareultimatelytobeviewedbyhumanbeings,theonly“correct”methodofquantifyingvisualimagequalityisthroughsubjec-tiveevaluation.Inpractice,however,subjectiveevaluationisusuallytooinconvenient,time-consumingandexpensive.Thegoalofresearchinobjectiveimagequalityassessmentistodevelopquantitativemeasuresthatcanautomaticallypredictperceivedimagequality.Anobjectiveimagequalitymetriccanplayavarietyofrolesinimageprocessingapplications.First,itcanbeusedtody-namicallymonitorandadjustimagequality.Forexample,anet-ManuscriptreceivedJanuary15,2003;revisedAugust18,2003.TheworkofZ.WangandE.P.SimoncelliwassupportedbytheHowardHughesMed-icalInstitute.TheworkofA.C.BovikandH.R.SheikhwassupportedbytheNationalScienceFoundationandtheTexasAdvancedResearchProgram.TheassociateeditorcoordinatingthereviewofthismanuscriptandapprovingitforpublicationwasDr.ReinerEschbach.Z.WangandE.P.SimoncelliarewiththeHowardHughesMedicalInstitute,theCenterforNeuralScienceandtheCourantInstituteforMathe-maticalSciences,NewYorkUniversity,NewYork,NY10012USA(e-mail:zhouwang@ieee.org;eero.simoncelli@nyu.edu).A.C.BovikandH.R.SheikharewiththeLaboratoryforImageandVideoEngineering(LIVE),DepartmentofElectricalandComputerEngi-neering,TheUniversityofTexasatAustin,Austin,TX78712USA(e-mail:bovik@ece.utexas.edu;hamid.sheikh@ieee.org).DigitalObjectIdentifier10.1109/TIP.2003.8198611AMATLABimplementationoftheproposedalgorithmisavailableonlineat~lcv/ssim/.workdigitalvideoservercanexaminethequalityofvideobeingtransmittedinordertocontrolandallocatestreamingresources.Second,itcanbeusedtooptimizealgorithmsandparametersettingsofimageprocessingsystems.Forinstance,inavisualcommunicationsystem,aqualitymetriccanassistintheop-timaldesignofprefilteringandbitassignmentalgorithmsattheencoderandofoptimalreconstruction,errorconcealment,andpostfilteringalgorithmsatthedecoder.Third,itcanbeusedtobenchmarkimageprocessingsystemsandalgorithms.Objectiveimagequalitymetricscanbeclassifiedaccordingtotheavailabilityofanoriginal(distortion-free)image,withwhichthedistortedimageistobecompared.Mostexistingapproachesareknownasfull-reference,meaningthatacom-pletereferenceimageisassumedtobeknown.Inmanypracticalapplications,however,thereferenceimageisnotavailable,andano-referenceorblindqualityassessmentapproachisdesir-able.Inathirdtypeofmethod,thereferenceimageisonlypar-tiallyavailable,intheformofasetofextractedfeaturesmadeavailableassideinformationtohelpevaluatethequalityofthedistortedimage.Thisisreferredtoasreduced-referencequalityassessment.Thispaperfocusesonfull-referenceimagequalityassessment.Thesimplestandmostwidelyusedfull-referencequalitymetricisthemeansquarederror(MSE),computedbyav-eragingthesquaredintensitydifferencesofdistortedandreferenceimagepixels,alongwiththerelatedquantityofpeaksignal-to-noiseratio(PSNR).Theseareappealingbecausetheyaresimpletocalculate,haveclearphysicalmeanings,andaremathematicallyconvenientinthecontextofoptimization.Buttheyarenotverywellmatchedtoperceivedvisualquality(e.g.,[1]–[9]).Inthelastthreedecades,agreatdealofefforthasgoneintothedevelopmentofqualityassessmentmethodsthattakeadvantageofknowncharacteristicsofthehumanvisualsystem(HVS).ThemajorityoftheproposedperceptualqualityassessmentmodelshavefollowedastrategyofmodifyingtheMSEmeasuresothaterrorsarepenalizedinaccordancewiththeirvisibility.SectionIIsummarizesthistypeoferror-sensi-tivityapproachanddiscussesitsdifficultiesandlimitations.InSectionIII,wedescribeanewparadigmforqualityassessment,basedonthehypothesisthattheHVSishighlyadaptedforextractingstructuralinformation.Asaspecificexample,wede-velopameasureofstructuralsimilarity(SSIM)thatcompareslocalpatternsofpixelintensitiesthathavebeennormalizedforluminanceandcontrast.InS