机器学习及其算法和发展研究_张润

100024，，。，，，，，、，。;;;;TP181A1673－4793(2016)02－0010－10ResearchonMachineLearningwithAlgorithmandDevelopmentZHANGRunWANGYong-binCenterofComputerandNetworkCommunicationUniversityofChinaBeijing100024AbstractMachinelearningMLhasmadegreatprogressnowandisplayingamoreandmoreimportantroleinAIit’sneedfultoknowwellaboutML．ForthispurposethispaperattemptedtoprovidenoviceswithaguidesoitintroducedMLsystematically．FirstlythisarticlepresentedtheconceptbriefhistoryofdevelopmentandclassificationofML．SecondlywefocusedonthethreeclassicalalgorithmofMLinclu-dingBPCNNDP．ThirdlyitintroducedthelatestresearchvisionandapplicationofML．Finallyitcon-cludedthechallengesofML．KeywordsmachinelearningvisualcortexBPCNNdeeplearning1。TomM．Mitchell《MachineLearning》“”。。70。、、、、、。DeepMindAlphaGo41。[e-mail]:10.16196/j.cnki.issn.1673-4793.2016.02.002。、、1、。。。22．1。1943WarrenMcCullochWalterPitts2。1950“”“”。1957FrankRosenblattPerceptron。1959IBMA．M．Samuel8。。1962HubelWieselHubel－Wiesel。1969MarvinMinskySeymourPapert《Perceptron》XOR、、。1980·。1986《MachineLearning》。1982Hopfield3HopfieldHopfield。1986Rumelhart、HintonWilliams《》BP4BP15XORGate。。11989YannLeCunCNNBP。CNN、。90、、。611。2006。GeoffreyHintonRuslan-Salakhutdinov7。。2012HintonImageNet。Hinton、LeCunAndrewNg、、28SiriCortanaSmiletoPayAlphaGo。29。2．2。。。。。、、、。。AndrewNg10。、、。。311。3、。BP、。2133．1BPBP412、。4BP→→hWbx=f∑ni=1Wixi+bWibfWbxR→Rsig-moid、tanh、ReLUhWbx。→e=1n∑Ll=1∑ni=112hWbxil－yil2+λ2W2eyλ＞0λ2W2。BP。。BP。、。BP311234BP、、L－BFGSWolfeArmijo。BP、。BPChainRule。3．2CNN513、。CNN14。5615CNN、。。6CNNCNNBP/BP3．1。CNN。41CNN。7a～c8×83×36×6。。7d。CNN/。7。CNNAlphaGoDCNN。3．3。1943WarrenMcCullochWalterPitts8。。。816、、17。9、softmax18。Hinton———+19。519、。1．。2．。。3．。4．。5．。。AlphaGo280GPU1920CPU、BP、。20。44．1。、。4．1．1LiuYan21DDBNs。Luo22SGRBM、。YuDong23DCN。Huang24。Larochelle25RBMClassRBM。Zhou26CDN。Mrazova27CNN、、。KaimingHe28CNN。4．1．2Wong29Fisher、RD-FM、Fine－tuning。Collobert30GTN。Hinton3161。Duchi32。33Fisher。Zhou34ADN。Tom35。Schaul36。4．2AlphaGo41。、、、、。。、、、、。、、、、、、、、……5。。CNNYannLeCun2015IEEE37。、、。100。。、、、、。。6、。、、、。、。2013－2014、100“”。711．J．2015P60－62．2WarrenSMcCullochandWalterPitts．Alogicalcalculusoftheideasimmanentinnervousactivity．Thebulletinofmathematicalbiophysics194354115－133．3HopfieldJJ．NeuralNetworksandPhysicalSys-temswithEmergentCollectiveComputationalAbil-itiesProcNatlAcadSci．USA1982792254－2558．4DERumelhartGEHintonRJWilliams．Learn-inginternalrepresentationsbyerrorpropagation．Nature198632399533－536．5http//deeplearning．stanford．edu/wiki/index．php/UFLDL_Tutorial．6http//blog．csdn．net/datoubo/article/details/8577366．7GeofferyEHintonSalakhutdinovRR．Reducingthedimensionalityofdatawithneuralnetworks．Science20063135786504－7．8http//．ccf．org．cn/sites/ccf/xhdtnry．jspcontentId=2873667830199．9．J．2015P60－62．10http//tech．163．com/16/0229/07/BGVMTL-GA000915BF．html．11http//nkonst．com/machine－learning－ex-plained－simple－words/．12UFLDLTutorialhttp//deeplearning．stanford．edu/wiki/index．php/UFLDL_Tutorial．13http//blog．csdn．net/zouxy09/article/details/8781543/．14．J．201527－31．15http//．duzelong．com/wordpress/wp－con-tent/uploads/2015/10/CNN12．jpg．16MachineLearningandAIviaBrainSimulations_AndrewNg．pdf．17．20M．2015209－212．18http//deeplearning．stanford．edu/wiki/index．php/UFLDL_Tutorial．19GeoffreyEHintonSimonOsinderoYee－WhyeTeh．AFastLearningAlgorithmforDeepBeliefNetsJ．NeuralComputation20061871527－1554．20．J．2015P7．21LiuYanZhouShu－senChenQing－cai．Dis-criminativeDeepBeliefNetworksforVisualDataClassificationJ．PatternRecognition201044102287－2297．22LuoHShenRNiuCetal．SparseGroupRe-strictedBoltzmannMachinesC//ProceedingsoftheAAAI2011．23YuDongDengLi．DeepConvexnetaScalableArchitectureforSpeechPatternClassificationC//Procofthe12thAnnualConferenceofInterna-tionalSpeechCommnicationAssociation．20112296－2299．24HuangGBLeeHLEARNED－MILLERE．Learn-ingHierarchicalRepresentationsforFaceVerifica-tionwithConvolutionalDeepBeliefNet－worksC//ProcofIEEEConferenceonComputerVisionandPatternRecognition．20122518－2525．25LarochelleHMandelMPascanuRetal．Learn-ingAlgorithmfortheClassificationRestrictedBo-ltzmannMachineJ．JournalofMachineLearn-ingResearch201213643－669．26ZhouShu－senChenQing－caiWangXiao－long．ConvolutionalDeepNetworksforVisualDa-taClassificationJ．NeuralProcessLett20133817－27．27MrazovaIKukackaM．ImageClassificationwithGrowingNeuralNetworksJ．InternationalJour-nalofComputerTheory＆Engineering201353422－427．28HeKZhangXRenSetal．SpatialPyramidPoo-lingM//ComputerVision－ECCV201413thEuropeanConferenceZurichSwitzerlandSet6－122014ProceedingsPartⅢ2014346－361．2481x3k=3x4k=1。1．M．2012．2．M．2008．3．D．2012．4．J．1993028－13．5．．M．2006．6．J．200712248－260．7．J．201502110－116．8SchwartzSIFoinTC．ReviewACriticalReviewoftheSocialSystemsModelsofJayForresterJ．HumanEcology197209161－173．9RichE．ModelingtheDynamicsofOrganizationalKnowledgeJ．AmericanJournalofMedicine200202206－213．1829WongWKSunM．DeepLearningRegularizedFisherMappingsJ．IEEETransonNeuralNet-works201122101668－1675．30CollobertRTR．DeepLearningforEfficientDis-criminativeParsingC//Procofthe14thInter-nationalConferenceonArtificialIntelligenceandStatistics．2011224－232．31HintonG