Automatic image annotation and retrieval using wei

AutomaticImageA[e-mail]@utdallas.eduDepartmentofComputerScience,UniversityofTexasatDallas,Richardson,Texas75083Abstract.Thedevelopmentoftechnologygenerateshugeamountsofnon-textualinformation,suchasimages.Anefficientimageannotationandretrievalsystemishighlydesired.Clusteringalgorithmsmakeitpossibletorepresentvisualfeaturesofimageswithfinitesymbols.Basedonthis,manystatisticalmodels,whichanalyzecorrespondencebetweenvisualfeaturesandwordsanddiscoverhiddensemantics,havebeenpublished.Thesemodelsimprovetheannotationandretrievaloflargeimagedatabases.However,imagedatausuallyhavealargenumberofdimensions.Traditionalclusteringalgorithmsassignequalweightstothesedimensions,andbecomeconfoundedintheprocessofdealingwiththesedimensions.Inthispaper,weproposeweightedfeatureselectionalgorithmasasolutiontothisproblem.Foragivencluster,wedeterminerelevantfeaturesbasedonhistogramanalysisandassigngreaterweighttorelevantfeaturesascomparedtolessrelevantfeatures.WehaveimplementedvariousdifferentmodelstolinkvisualtokenswithkeywordsbasedontheclusteringresultsofK-meansalgorithmwithweightedfeatureselectionandwithoutfeatureselection,andevaluatedperformanceusingprecision,recallandcorrespondenceaccuracyusingbenchmarkdataset.Theresultsshowthatweightedfeatureselectionisbetterthantraditionalonesforautomaticimageannotationandretrieval.Keywords:automaticimageannotation,subspaceclusteringalgorithm1.IntroductionImagesareamajorsourceofcontentontheInternet.Thedevelopmentoftechnologysuchasdigitalcamerasandmobiletelephonesequippedwithsuchdevicesgenerateshugeamountsofnon-textualinformation,suchasimages.Anefficientimageretrievalsystemisdesirablewheregivenalargedatabase,weneed,forexample,tofindtheimagesthathavetigers,orgivenanunseenimage,findkeywordsthatbestdescribeitscontent[Duygulu02].Hence,thesetechniquesraisethepossibilityofseveralinterestingapplicationssuchas:·Automaticimageannotation/description:inmanycasescollectionsofimagesarekeptforvarioususes.Newspapersmaywanttoretrieveimagesfromaso-calledmorgue[Markkula2000].Imageretrievalmightalsobeanimportantpartofintelligencegatheringandsurveillance.Withregardtoadatabankofimagesannotationthroughtheuseofkeywordsisoftenanuncertainproposition.Technicaladvancesinthefieldofautomaticimageannotationwouldbemostwelcome.·Afurtherapplicationofimageretrievaltoolsinvolvesarthistoryandpublicmuseums.Inthecaseofthelatterimagesareoftenpublishedontheweb.Whiletheentirecollectioncannotbepracticallyposteditwouldbeusefultoallowpatronsorstudentstoaccessanarchiveinsearchofparticularimages[Forst2000].Thismeansamethodtoorganizethecollectionthatsupportedbrowsingwouldbeattractiveandmademoresensetovisitors.Aggregatingimagesthatlookedsimilarandweresimilarlyannotatedwouldbeagoodstart.·Commercialimagecollectionscouldofferanattractiveserviceifsearchingthecollectioncouldbemadelessdifficultandexpensive.Illustratingtextwithimagescouldbemademucheasier,andcouldeventaketheformofauto-illustrationifreasonableresultscouldbecheaplyobtained.Content-basedimageretrieval(CBIR)computesrelevancebasedonthevisualsimilarityoflow-levelimagefeaturessuchascolorhistograms,textures,shapesandspatiallayoutetc.However,theproblemisthatvisualsimilarityisnotsemanticsimilarity.Thereisagapbetweenlow-levelvisualfeaturesandsemanticmeanings.Theso-calledsemanticgapisthemajorproblemthatneedstobesolvedformostCBIRapproaches.Forexample,aCBIRsystemmayansweraqueryrequestfor‘redball’withanimageofa‘redrose’.Ifweprovideannotationofimageswithkeywords,thentypicalwaytopublishanimagedatarepositoryistocreateakeyword-basedqueryinterfacetoanimagedatabase.Imagesareretrievediftheycontain(somecombinationofthe)keywordsspecifiedbytheuser.Ourgoalistoquerytheimagesnotonlybasedonentiretybutalsoontheindividualobjectsthatappearinimages.Forexample,theusercanspecifyaquerybysayingonly“tiger”objectandresultsetofobjectswillbetigerobject.Ontheotherhand,ausercanspecifythequerybyspecifyingonly“tiger”objectexcluding“river”objectintheimage.Inthatcase,resultsetofimageswillincludeimagesthatcontain“tiger”objectandnot“river”object.Toachieveallthesegoalsatthisfinegranularitythereareseveraltechnicalchallenges:ItisImportanttonotethatinthispaperobjectandvisualtokenswillbeusedinterchangeably.1.Segmentimagesintomeaningfulvisualsegments/tokens.2.Determinecorrelationbetweenassociatedkeywordsandvisualtokens.Withregardtothefirstproblemwerelyonnormalizedcutthatsegmentimagesintoanumberofvisualtokens[Shi97].Eachvisualtokenwillberepresentedbyavectorofcolors,textures,shapeetc.Therefore,visualtokenmeansasegmentedregionorobject,anditwillbedescribedbyasetoflowlevelfeatureslikecolor,texture,andshape.InFigure1wherefirstcolumncorrespondsimages,thesecondcolumnrepresentssegmentedimagesorasetofvisualtokens.Figure1.DemonstrationofCorrespondencebetweenImageobjectsandtheirKeywordWithregardtothesecondproblem,thereareseveraltasksonecouldattack.First,onecouldattempttopredictannotationsofentireimagesusingallinformationpresentwhichisannotationtask.Next,onemightatt