机器学习当前研究进展

AFewUsefulThingstoKnowaboutMachineLearning11/10/2012Presentedby:YangSONG11/10/2012AFewUsefulThingstoKnowaboutMachineLearning2/71Author’sIntroductionPedroDomingosIntroductionTwelvekeylessonsConclusions11/10/2012AFewUsefulThingstoKnowaboutMachineLearning4/71OutlineIntroductionTwelvekeylessonsConclusions11/10/2012AFewUsefulThingstoKnowaboutMachineLearning5/71IntroductionMachineLearningAfewquotes“AbreakthroughinmachinelearningwouldbeworthtenMicrosofts”(BillGates,Chairman,Microsoft)Machinelearningisthehotnewthing”(JohnHennessy,President,Stanford)“Machinelearningistoday’sdiscontinuity”(JerryYang,Founder,Yahoo)“Machinelearningtodayisoneofthehottestaspectsofcomputerscience”(SteveBallmer,CEO,Microsoft)11/10/2012AFewUsefulThingstoKnowaboutMachineLearning6/71IntroductionMachineLearningTraditionalProgrammingMachineLearning11/10/2012AFewUsefulThingstoKnowaboutMachineLearning7/71IntroductionTypesofLearningSupervised(inductive)learningTrainingdataincludesdesiredoutputsUnsupervisedlearningTrainingdatadoesnotincludedesiredoutputsSemi-supervisedlearningTrainingdataincludesafewdesiredoutputsReinforcementlearningRewardsfromsequenceofactions11/10/2012AFewUsefulThingstoKnowaboutMachineLearning8/71IntroductionArecentreportfromtheMcKinseyGlobalInstitute,2011“machinelearning(a.k.a.dataminingorpredictiveanalytics)willbethedriverofthenextbigwaveofinnovation.”11/10/2012AFewUsefulThingstoKnowaboutMachineLearning9/71IntroductionPurposeofthisarticle“Severalfinetextbooksareavailabletointerestedpractitionersandresearchers.However,muchofthe‘folkknowledge’thatisneededtosuccessfullydevelopmachinelearningapplicationsisnotreadilyavailableinthem.”“Asaresult,manymachinelearningprojectstakemuchlongerthannecessaryorwindupproducinglessthanidealresults.Yetmuchofthisfolkknowledgeisfairlyeasytocommunicate.”“民科”Thisarticlewillfocusonclassificationproblem11/10/2012AFewUsefulThingstoKnowaboutMachineLearning10/71OutlineIntroductionTwelvekeylessonsConclusions11/10/2012AFewUsefulThingstoKnowaboutMachineLearning11/71OutlineIntroductionTwelvekeylessons1.Learning=representation+evaluation+optimization2.It’sgeneralizationthatcounts（泛化）3.Dataaloneisnotenough（先验知识）4.Overfittinghasmanyfaces（过拟合）5.Intuitionfailsinhighdimensions（高维）6.Theoreticalguaranteesarenotwhattheyseem（理论）Conclusions11/10/2012AFewUsefulThingstoKnowaboutMachineLearning12/71OutlineIntroductionTwelvekeylessons7.Featureengineeringisthekey（特征）8.Moredatabeatsaclevereralgorithm（数据）9.Learnmanymodels,notjustone（集成学习）10.Simplicitydoesnotimplyaccuracy（简单与精确）11.Representabledoesnotimplylearnable（可表示与可学习）12.Correlationdoesnotimplycausation（关联与因果）Conclusions11/10/2012AFewUsefulThingstoKnowaboutMachineLearning13/711.Learning=representation+evaluation+optimization11/10/2012AFewUsefulThingstoKnowaboutMachineLearning14/711.Learning=representation+evaluation+optimizationLearningalgorithmshavethreecomponents:Representation:Aclassifiermustberepresentedinsomeformallanguagethatthecomputercanhandle.Choosingarepresentationistantamounttochoosingthesetofclassifiersitcanpossiblylearn.Thissetiscalledthehypothesisspaceofthelearner.Evaluation:Anevaluationfunction(alsocalledobjectivefunctionorscoringfunction)isneededtodistinguishdifferentclassifiers.Optimization:Weneedamethodtosearchamongtheclassifiersforthehighest-scoringone.Thechoiceofoptimizationtechniqueiskeytotheefficiencyofthelearner.HypothesisspaceScoringfunctionSearchmethods11/10/2012AFewUsefulThingstoKnowaboutMachineLearning15/711.Learning=representation+evaluation+optimizationRepresentationInstancesKNN,SVMHyperplanesNaïveBayes,LogisticregressionDecisiontreesSetsofrulesPropositionalrules,LogicprogramsNeuralnetworksGraphicalmodelsBayesiannetworks,Conditionalrandomfields11/10/2012AFewUsefulThingstoKnowaboutMachineLearning16/711.Learning=representation+evaluation+optimizationEvaluationAccuracy/ErrorratePrecisionandrecallSquarederrorLikelihoodPosteriorprobabilityInformationgainK-LdivergenceCost/UtilityMargin11/10/2012AFewUsefulThingstoKnowaboutMachineLearning17/711.Learning=representation+evaluation+optimizationOptimizationCombinatorialoptimizationGreedysearch,Beamsearch,Branch-and-boundContinuousoptimization(convexoptimization)UnconstrainedGradientdescent,Conjugategradient,Quasi-NewtonmethodsConstrainedLinearprogramming,Quadraticprogramming11/10/2012AFewUsefulThingstoKnowaboutMachineLearning18/711.Learning=representation+evaluation+optimizationNotallcombinationsofonecomponentfromeachcolumninthetablemakeequalsenseE.g.discreterepresentationsnaturallygowithcombinatorialoptimization,continuouswithcontinuousThereisnosimplerecipeforchoosingeachcomponent,thenextsectionswilltouchonsomeofthekeyissues.11/10/2012AFewUsefulThingstoKnowaboutMachineLearning19/712.It’sgeneralizationthatcounts11/10/2012AFewUsefulThingstoKnowaboutMachineLearning20/712.It’sgeneralizationthatcountsThefundamentalg