机器学习课件

MachineLearningLecture1Introduction1MotivatingProblems•HandwrittenCharacterRecognition2MotivatingProblems•FingerprintRecognition(e.g.,bordercontrol)3MotivatingProblems•FaceRecognition(securityaccesstobuildingsetc)4CanMachinesLearntoSolveTheseProblems?Or,tobemoreprecise–Canweprogrammachinestolearntodothesetasks?5DefinitionofLearning•AcomputerprogramissaidtolearnfromexperienceEwithrespecttosomeclassoftasksTandperformancemeasureP,ifitsperformanceattasksinT,asmeasuredbyP,improveswithexperienceE(Mitchell,MachineLearning,McGraw-Hill,1997)6DefinitionofLearning•Whatdoesthismeanexactly?–Handwritingrecognitionproblem•TaskT:Recognizinghandwrittencharacters•PerformancemeasureP:percentofcharacterscorrectlyclassified•TrainingexperienceE:adatabaseofhandwrittencharacterswithgivenclassifications7DesignaLearningSystem•WeshallusehandwrittenCharacterrecognitionasanexampletoillustratethedesignissuesandapproaches8DesignaLearningSystemStep0:–Letstreatthelearningsystemasablackbox9LearningSystemZDesignaLearningSystemStep1:CollectTrainingExamples(Experience).–Withoutexamples,oursystemwillnotlearn(so-calledlearningfromexamples)10236789DesignaLearningSystemStep2:RepresentingExperience–Choosearepresentationschemefortheexperience/examples•Thesensorinputrepresentedbyann-dvector,calledthefeaturevector,X=(x1,x2,x3,…,xn)11(1,1,0,1,1,1,1,1,1,1,0,0,0,0,1,1,1,1,1,0,….,1)64-dVector(1,1,1,1,1,1,1,1,1,1,0,0,1,1,1,1,1,1,1,0,….,1)64-dVectorDesignaLearningSystemStep2:RepresentingExperience–Choosearepresentationschemefortheexperience/examples•Thesensorinputrepresentedbyann-dvector,calledthefeaturevector,X=(x1,x2,x3,…,xn)•Torepresenttheexperience,weneedtoknowwhatXis.•SoweneedacorrespondingvectorD,whichwillrecordourknowledge(experience)aboutX•TheexperienceEisapairofvectorsE=(X,D)12DesignaLearningSystemStep2:RepresentingExperience–Choosearepresentationschemefortheexperience/examples•TheexperienceEisapairofvectorsE=(X,D)–So,whatwouldDbelike?Therearemanypossibilities.13DesignaLearningSystemStep2:RepresentingExperience–So,whatwouldDbelike?Therearemanypossibilities.–Assumingoursystemistorecognise10digitsonly,thenDcanbea10-dbinaryvector;eachcorrespondtooneofthedigits14D=(d0,d1,d2,d3,d4,d5,d6,d7,d8,d9)e.g,ifXisdigit5,thend5=1;allothers=0IfXisdigit9,thend9=1;allothers=0DesignaLearningSystemStep2:RepresentingExperience–So,whatwouldDbelike?Therearemanypossibilities.–Assumingoursystemistorecognise10digitsonly,thenDcanbea10-dbinaryvector;eachcorrespondtooneofthedigits15X=(1,1,0,1,1,1,1,1,1,1,0,0,0,0,1,1,1,1,1,0,….,1);64-dVectorD=(0,0,0,0,0,1,0,0,0,0)X=(1,1,1,1,1,1,1,1,1,1,0,0,1,1,1,1,1,1,1,0,….,1);64-dVectorD=(0,0,0,0,0,0,0,0,1,0)D=(d0,d1,d2,d3,d4,d5,d6,d7,d8,d9)DesignaLearningSystemStep3:ChooseaRepresentationfortheBlackBox–WeneedtochooseafunctionFtoapproximatetheblockbox.ForagivenX,thevalueofFwillgivetheclassificationofX.ThereareconsiderableflexibilitiesinchoosingF16LearningSystemFF(X)XDesignaLearningSystemStep3:ChooseaRepresentationfortheBlackBox–Fwillbeafunctionofsomeadjustableparameters,orweights,W=(w1,w2,w3,…wN),whichthelearningalgorithmcanmodifyorlearn17LearningSystemF(W)F(W,X)XDesignaLearningSystemStep4:Learning/AdjustingtheWeights–Weneedalearningalgorithmtoadjusttheweightssuchthattheexperience/priorknowledgefromthetrainingdatacanbelearnedintothesystem:18E=(X,D)F(W,X)=DDesignaLearningSystemStep4:Learning/AdjustingtheWeights19LearningSystemF(W)F(W,X)XDE=(X,D)Error=D-F(W,X)AdjustWDesignaLearningSystemStep5:Use/TesttheSystem–Oncelearningiscompleted,allparametersarefixed.AnunknowninputXispresentedtothesystem,thesystemcomputesitsansweraccordingtoF(W,X)20LearningSystemF(W)F(W,X)XAnswerRevisionofSomeBasicMaths•VectorandMatrix–Rowvector/columnvector/vectortransposition–Vectorlength/norm–Inner/dotproduct内积/点积–Matrix(vector)multiplication–Linearalgebra线性代数–Euclideanspace欧式空间•BasicCalculus基本微积分–Partialderivatives–Gradient–Chainrule21RevisionofSomeBasicMaths•Inner/dotproductx=[x1,x1,…,xn]T,y=[y1,y1,…,yn]TInner/dotproductofxandy,xTy•Matrix/Vectormultiplication22niiinnTyxyxyxyxyx12211RevisionofSomeBasicMaths•Vectorspace/Euclideanspace•AvectorspaceVisasetthatisclosedunderfinitevectoradditionandscalarmultiplication.•Thebasicexampleisn-dimensionalEuclideanspace,whereeveryelementisrepresentedbyalistofnrealnumbers•Ann-dimensionalrealvectorcorrespondstoapointintheEuclideanspace.[1,3]isapointin2-dimensionalspace[2,4,6]ispointin3-dimensionalspace23RevisionofSomeBasicMaths•Vectorspace/Euclideanspace–Euclideanspace(Euclideandistance)–Dot/innerproductandEuclideandistance•Letxandyaretwonormalizednvectors,||x||=1,||y||=1,wecanwrite•MinimizationofEuclideandistancebetweentwovectorscorrespondstomaximizationoftheirinnerproduct.–Euclideandistance/innerproductassimilaritymeasure242222211nnyxyxyxYXYXYXYXYXTT222RevisionofSomeBasicMaths•BasicCalculus–Multivariablefunction:–Partialderivative:givesthedirectionandspeedofchangeofy,withrespecttoxi–Gradient–Chainrule:Lety=f(g(x)),u=g(x),then–Letz=f(x,y),x=g(t),y=h(t),then25)...,,,()(21nxxxfxynxfxff......