搜索
2012－04－212012－07－25Y11012692008C140631988－、。Hadoop王梅，朱信忠，赵建民，黄彩锋(浙江师范大学数理与信息工程学院，浙江金华321004)、。HadoopMapReduceHDFSHadoop。。HadoopMapReduceTP391A1673－629X201301－0204－04doi10．3969/j．issn．1673－629X．2013．01．050MassiveImagesRetrievalSystemBasedonHadoopWANGMeiZHUXin－zhongZHAOJian－minHUANGCai－fengCollegeofMathematicsandPhysicsandInformationEngineeringZhejiangNormalUniversityJinhua321004ChinaAbstractInatraditionalsingle－nodearchitectureimageretrievalsystemfacingtheproblemsofslowretrievalspeedpoorconcurrencyetcwhenretrievedmassiveimagedataproposedanimageretrievalmethodbasedontheHadoopcombiningimageretrievaltechnologywithMapReduceframeimagefeaturedatabasestoredinthedistributedfilesystemHDFScomputingnodesusingtheschedulingalgo-rithmbasedonHadoopdistributedstorageenhancedconcurrentprocessingcapabilityformultipledataatthesametimecompressedthecalculateddata．Testandexperimentresultsshowthatthemethodindealingwithlargedataretrievalcomparedwiththesingle－nodere-trievalsystemcaneffectivelyreducethesearchtimeimprovetheretrievalspeed．KeywordsHadoopMapReducedistributedcomputingimageretrieval0。CPU。、1。、、。HadoopApacheApacheSoftwareFoundation、PB23。Hadoop、、、。HadoopMapReduce4HDFSHadoop。11．1Hadoop、、。、23120131COMPUTERTECHNOLOGYANDDEVELOPMENTVol．23No．1Jan．20131。图1基于Hadoop的图像检索系统整体构架InternetWeb。。HDFSHDFS。、、。1．2HDFSHDFSMaster/Slave－NameNodeDataNode。Name-NodeDataNode。HDFS2。图2HDFS框架结构NameNodeHDFS。NameNodeHadoopNameNode。HadoopDataNode。DataNodeNameNodeHadoopDa-taNodeNameNode。NameNodeNameNodeHDFS、、、Ha-doop、HDFS。1．3MapReduceMapReduce5。MapReduce3。图3实现并行计算的MapReduce工作框图MapReduceHDFSDataNodeTaskTrack-erMapKey/ValueKeyval-ue。mapKey/Valuemap·502·1Hadoop、、Key/ValueKeyValue。MapReduce。ReduceMapHDFS。22．1HadoopPB。HadoopHadoop6。7。＜Blk_IDMetaData＞＜keyvalue＞Blk_IDMetaData。＜intbyte＞4。图4大文件分布式存储方法2．2、8。。。。9。RGBHSV。HSV72Hk=numk/numk=012…L－11。numkKnumL。。fxy()Lfxy()∈0L－1[]RSCMδθij()=cardx1y1x2y2[]∈S|fx1y1=i＆fx2y2=j{}cardS22i∈0L－1[]j∈0L－1[]x2=x1+dcosθy2=y1+dsinθcardSSCMδθij()。2410①。E=∑Di=1∑Dj=1mij23②。I=∑Di=1∑Dj=1i－j2·mij4③。S=－∑Di=1∑Dj=1mi－j·logmij5mij=0logmij=0。④。H=∑Di=1∑Dj=1mij1+i－j26D84F=F1F2F3F4[]。SQIQI·602·23SQI=w1·ScolarQI+w2·ScmQI77ScolarQIScolarQI=∑L－1k=0minHkQHkI()8ScolarQIScmQI=∑4i=1QFi－IFi()槡29w1w2w1+w2=10．5。7QISQI。2．3MapReduceMapReduceMapReduceMapReduceMapReduceTB。Map/Reduce3MapReduce5。图5MapReduce算法流程Hadoopmap、reduceMap/Reducerun1112。Map/Reducemap。mapMappermap＜KeyValue＞contextTaskTracker。re-duceReducerreduce13map。mapreduceMapperReducerRunMapReduce14。3PC1Mas-ter3Slave。CPU2．1GHzPentium2GB320GB100Mb/sRedHatLinux。B/S。6。图6不同节点下的检索时间对比611020Hadoop1B/S。HadoopMapReduceHadoop。2Hadoop40120B/SHadoop。4Hadoop·702·1HadoopHa-doopMapReduce。B/SB/S、、。MapReduce。1WangFeiVukEDavidBetal．Large－scalemulti－modelminingforhealthcarewithMapReduceC//Proceedingsofthe1stACMInternationalHealthInformaticsSymposium．s．l．s．n．2010479－483．2HDFSEB/OL．2011－12－08．http//hadoop．apache．org/．3WbiteT．HadooptheDefinitiveGuideM．s．l．O＇ReilllyMediaInc．2009．4MapReduceEB/OL．2011－12－08．http//had－oop．a-pache．org/mapreduce/．5ChuCTKimSKLinYAetal．Map－reduceformachinelearningonmulticoreM．s．l．theMITPress2007．6．J．20092992562－2566．7．J．201121499－102．8．J．20054162－164．9．J．200434209－211．10．J．200919438－40．11．HadoopD．2011．12．HadoopJ．201112242－245．13．J．20062791782－1787．14．J．2009193200－203．檪檪檪檪檪檪檪檪檪檪檪檪檪檪檪檪檪檪檪檪檪檪檪檪檪檪檪檪檪檪檪檪檪檪檪檪檪檪檪檪檪檪檪檪檪檪(上接第203页)LinuxWeb。。。1．WebServerJ．2009197213－215．2．ZigBeeJ．2007282404－408．3．Linux3GJ．2010209193－196．4．ZigBeeJ．200829123064－3067．5．LinuxGPRSD．2011．6．J．20116105－109．7．ZigBeeJ．20096588－591．8．ZigBeeJ．2010187117－119．9．ZigBeeLEDJ．2009322394－396．10．ARM－LinuxBootLoaderJ．20081810205－206．11JahnkeJHd＇EntremontMStierJ．Facilitatingtheprogram-mingofthesmarthomeJ．IEEEWirelessCommunications20029670－76．12FurberS．ARMSystem－on－ChipArchitectureD．USAAd-dison－WesleyPress2000．13HenkelJ．SelectiverevealinginopeninnovationprocessesThecaseofembeddedLinuxJ．ResearchPolicy2006357953－969．·802·23