问题为什么现在的计算机处理智能信息效率很低

Prof.LiqingZhangDept.ComputerScience&Engineering,ShanghaiJiaotongUniversityStatisticalLearning&Inference2019/9/5StatisticalLearningandInference2BooksandReferences–TrevorHastieRobertTibshiraniJeromeFriedman,TheElementsofstatisticalLearning:DataMining,Inference,andPrediction,2001,Springer-Verlag–VladimirN.Vapnik,TheNatureofStatisticalLearningTheory,2nded.,Springer,2000–S.Mendelson,AfewnotesonStatisticalLearningTheoryinAdvancedLecturesinMachineLearning:MachineLearningSummerSchool2002,S.MendelsonandA.J.Smola(eds),LectureNotesinComputerScience,2600,Springer,2003–M.Vidyasagar,Learningandgeneralization:withapplicationstoneuralnetworks,2nded.,Springer,20032019/9/5StatisticalLearningandInference3OverviewoftheCourseIntroductionOverviewofSupervisedLearningLinearMethodforRegressionandClassificationBasisExpansionsandRegularizationKernelMethodsModelSelectionsandInferenceSupportVectorMachineBayesianInferenceUnsupervisedLearning2019/9/5StatisticalLearningandInference4WhyStatisticalLearning?我门被信息淹没，但却缺乏知识。----R.Roger恬静的统计学家改变了我们的世界；不是通过发现新的事实或者开发新技术，而是通过改变我们的推理、实验和观点的形成方式。----I.Hacking问题：为什么现在的计算机处理智能信息效率很低？–图像、视频、音频–认知–语言2019/9/5StatisticalLearningandInference5ML:SARSRiskPredictionSARSRiskAgeGenderBloodPressureChestX-RayPre-HospitalAttributesAlbuminBloodpO2WhiteCountRBCCountIn-HospitalAttributes2019/9/5StatisticalLearningandInference6ML:AutoVehicleNavigationSteeringDirection2019/9/5StatisticalLearningandInference7ProteinFolding2019/9/5StatisticalLearningandInference8TheScaleofBiomedicalData2019/9/5StatisticalLearningandInference9计算科学与脑科学计算机信息处理–基于逻辑的计算–ＣＰＵ和数据分离–数据处理与存储简单–智能信息处理复杂、慢–认知能力弱–信息处理模式：逻辑－概念－统计信息大脑信息处理–基于统计信息的计算–计算和数据集成一体–数据处理与存储未知–智能信息处理简单、快速–认知能力强–信息处理模式：统计信息－概念－逻辑2019/9/5StatisticalLearningandInference10FunctionEstimationModelTheFunctionEstimationModeloflearningexamples:–Generator(G)generatesobservationsx(typicallyinRn),independentlydrawnfromsomefixeddistributionF(x)–Supervisor(S)labelseachinputxwithanoutputvalueyaccordingtosomefixeddistributionF(y|x)–LearningMachine(LM)“learns”fromani.i.d.l-sampleof(x,y)-pairsoutputfromGandS,bychoosingafunctionthatbestapproximatesSfromaparameterisedfunctionclassf(x,),whereisintheparameterset2019/9/5StatisticalLearningandInference11FunctionEstimationModelKeyconcepts:F(x,y),ani.i.d.k-sampleonF,functionsf(x,)andtheequivalentrepresentationofeachfusingitsindexxGSLMyy^2019/9/5StatisticalLearningandInference12Thelossfunctional(L,Q)–theerrorofagivenfunctiononagivenexampleTheriskfunctional(R)–theexpectedlossofagivenfunctiononanexampledrawnfromF(x,y)–the(usualconceptof)generalisationerrorofagivenfunctionTheProblemofRiskMinimization,,,:,,,,:xyzfzLzQxfyLfyxLzdFzQR,2019/9/5StatisticalLearningandInference13TheProblemofRiskMinimizationThreeMainLearningProblems–PatternRecognition:–RegressionEstimation:–DensityEstimation:,,,and1,0xfyxfyLy12,,,andxfyxfyLy,log,and1,0xpxpLy2019/9/5StatisticalLearningandInference14GeneralFormulationTheGoalofLearning–Givenani.i.d.k-samplez1,…,zkdrawnfromafixeddistributionF(z)–Forafunctionclass’lossfunctionalsQ(z,),within–Wewishtominimisetherisk,findingafunction*Rminarg*2019/9/5StatisticalLearningandInference15GeneralFormulationTheEmpiricalRiskMinimization(ERM)InductivePrinciple–Definetheempiricalrisk(sample/trainingerror):–Definetheempiricalriskminimiser:–ERMapproximatesQ(z,*)withQ(z,k)theRempminimiser…thatisERMapproximates*withk–Least-squaresandMaximum-likelihoodarerealisationsofERMkiizQkR1emp,1empminargRk2019/9/5StatisticalLearningandInference164IssuesofLearningTheory1.Theoryofconsistencyoflearningprocesses•Whatare(necessaryandsufficient)conditionsforconsistency(convergenceofRemptoR)ofalearningprocessbasedontheERMPrinciple?2.Non-asymptotictheoryoftherateofconvergenceoflearningprocesses•Howfastistherateofconvergenceofalearningprocess?3.Generalizationabilityoflearningprocesses•Howcanonecontroltherateofconvergence(thegeneralizationability)ofalearningprocess?4.Constructinglearningalgorithms(i.e.theSVM)•Howcanoneconstructalgorithmsthatcancontrolthegeneralizationability?2019/9/5StatisticalLearningandInference17ChangeinScientificMethodologyTRADITIONALFormulatehypothesisDesignexperimentCollectdataAnalyzeresultsReviewhypothesisRepeat/PublishNEWDesignlargeexperimentsCollectlargedataPutdatainlargedatabaseFormulatehypothesisEvaluatehypothesisondatabaseRunlimitedexperimentsReviewhypothesisRepeat/Publish2019/9/5StatisticalLearningandInference18Learning&AdaptationInthebroadestsense,anymethodthatincorporatesinformationfromtrainingsamplesinthedesignofaclassifieremployslearning.Duetocomplexityofclassificationproblems,wecannotguessthebestclassificationdecisionaheadoftime,weneedtolearnit.Creatingclassifierstheninvolvespositingsomegeneralformofmodel,orformoftheclassifier,andusingexamplestolearnthecompleteclassifier.2019/9/5StatisticalLearningandInference19SupervisedlearningInsupervisedlearning,ateacherprovidesacategorylabelforeachpatternin