机器学习简介

MachineLearning：AnOverview石立臣Outline•Whatismachinelearning(ML)•Typesofmachinelearning•Workflow•Popularmodels•Applications•FuturesWhatismachinelearningTrainingset(labelsknown)Testset(labelsunknown)f()=“apple”f()=“tomato”f()=“cow”Whatismachinelearning•Definition–Machinelearningreferstoasystemcapableoftheautonomousacquisitionandintegrationofknowledge–MachinelearningisprogrammingcomputerstooptimizeaperformancecriterionusingexampledataorpastexperienceComputerDataAlgorithmProgramKnowledgeKnowledge(new)Whatismachinelearning•Everymachinelearningalgorithmhasthreecomponents–Representation•Model(rules,statistics,instance;logic,KNN,SVM,DNN,…)–Evaluation•Performance(accuracy,mse,energy,entropy,…)–Optimization•Parameters–Combinatorialoptimization–Convexoptimization–ConstrainedoptimizationTypesofmachinelearning•Supervisedlearning–Trainingdataincludesdesiredoutputs•Unsupervisedlearning–Trainingdatadoesnotincludedesiredoutputs•Semi-supervisedlearning–Trainingdataincludesafewdesiredoutputs•Reinforcementlearning–RewardsfromsequenceofactionsTypesofmachinelearning•Supervisedlearning–Classification:discreteoutput–Regression:continuousoutputBias-varianceTrainingandValidationDataFullDataSetTrainingDataValidationDataIdea:traineachmodelonthe“trainingdata”andthentesteachmodel’saccuracyonthevalidationdataUnderfitting&OverfittingPredictiveErrorModelComplexityErroronTrainingDataErroronTestDataIdealRangeforModelComplexityOverfittingUnderfittingTypesofmachinelearning•Unsupervisedlearning–Clustering–Dimensionalityreduction–FactoranalysisTypesofmachinelearning•Semi-supervisedlearning–ClusteringorclassificationTypesofmachinelearning•Reinforcementlearning–Robot&controlWorkflowPredictionTrainingLabelsTrainingTrainingImageFeaturesImageFeaturesTestingTestImageLearnedmodelLearnedmodelSlidecredit:D.HoiemandL.LazebnikWorkflow•FeaturesWorkflow•Models–Logic,Rules–Statistical,Blackboxmodel•Static,dynamicmodel•Onlinelearning•EnsemblelearningWorkflow•ArchitectureModelFeatureHardwarePopularmodels•Linearmodel:logisticregression,lineardiscriminantanalysis,linearregression(withbasisfunction)Popularmodels•Nearestneighbor–Feature&distancePopularmodels•SupportvectormachinePopularmodels•ArtificialneuralnetworkPopularmodels•DecisiontreePopularmodels•CollaborativefilteringPopularmodels•Hierarchicalclustering•K-means•Spectralclustering•ManifoldlearningPopularmodels•Hiddenmarkovmodel•ConditionalrandomfieldsApplicationsApplicationsApplicationsApplicationsApplicationsApplicationsApplicationsApplicationsApplications•AttentionApplications•ImageclassificationApplicationsApplications•BrainmachineinterfaceApplicationsApplicationsApplicationsApplicationsApplications•Indirectillumination–RegressionApplications•Indirectillumination–kd-treeApplications•Thecoreisthedataset!!!•Others–features–model&optimizationFutures•Decision•Control•Knowledge•Prediction