模式识别与人工智能

2006-Learningwithoutthoughtmeanslaborlost;thoughtwithoutlearningisperilous.--Knowledgeisoftwokinds,weknowasubjectourselves,orweknowwecanfindinformationuponit.-3.1nnnn3.1.1nn1.1nnnnn3.1.1n3.11.13.1.1nP(Xi)=(1/2π|Σi|1/2)exp(-γi2/2);i=1,2;P(Xi)Σi2*2;|Σi|Σiγi2=(Xi-μi)TΣi-1(Xi-μi);μiXiμiΣi3.1.2nA,B,Cxxnxxn3.1.2n(MahalanobisDistance)dij=[(xi–xj)’Σ-1(xi–xj)]1/2n3.1.3nXμ(1)μ(2)d2(X,μ(1))-d2(X,μ(2))=(X-μ(1))’Σ-1(X-μ(1))-(X-μ(2))’Σ-1(X-μ(2))=-2W(X);W(X)=[X–(μ(1)+μ(2))/2]’Σ-1(μ(1)-μ(2))W(X)0Xμ(1)W(X)0Xμ(2)W(X)=0/3.1.3nXW(X)=(X-μ(1))’Σ1-1(X-μ(1))-(X-μ(2))’Σ2-1(X-μ(2))nWij(X)=[X–(μ(i)+μ(j))/2]’Σ-1(μ(i)-μ(j))Wij(X)0∀i≠jXμ(i)nXXd2(X,μ(i))=minkd2(X,μ(k)),Xμ(i)3.1.4nnXi(1),Xj(2)i=1,2,…,n1;j=1,2,…,n212E(1)=(1/n1)Σ1n1Xi;E(2)=(1/n2)Σ1n2XjV(1)=(1/(n1-1))Σ1n1(Xi(1)–E(1))(Xi(1)–E(1))’=(1/(n1-1))A1;V(2)=(1/(n2-1))Σ1n2(Xi(2)–E(2))(Xi(2)–E(2))’=(1/(n2-1))A23.1.4nV=(1/(n1+n2-2))(A1+A2)nW(X)=[X–(E(1)+E(2))/2]’V-1(E(1)–E(2))W(X)0X(1)W(X)0X(2)W(X)=0/3.1.5nn3.1.63.11.1nE(1)=[173.6,66]’;E(2)=[162,51]’;nW(X)=x1+72.52x2–4410nX(3)1/10=10%3.1.63.1nnnnn10%0nX(3)n3.1.63.1n3.1.7nnn3.1.7nnnnBayesn-3.2BayesnBayesFisherBayesnBayesn3.2.1Bayescω1ω2…,ωi…,ωcP(ω1),P(ω2),…,P(ωi),…,P(ωc)XP(X/ωi)i=1,2,…,CBayes(3.1)Bayesn1)gi(X)=P(ωi/X)2)gi(X)=P(X/ωi)P(ωi)3)gi(X)=logP(X/ωi)+logP(ωi)4)Lij(X)=P(X/ωi)/P(X/ωj)Lij(X)=1/Lji(X)BayesX∈ωi,Lij(X)P(ωj)/P(ωi)(3.3)BayesnBayesBayesnP1(e)Xω1,ω2P2(e)Xω2,ω1P(e)=P(ω1)P1(e)+P(ω2)P2(e)BayesnnL(λij)X∈ωiωj:ω1ω2nX∈ωi,L(X)[(λ21-λ22)/(λ12-λ11)][P(ω2)/P(ω1)](3.4)L(X)=P(X/ω1)/P(X/ω2)nP.114-116nNeyman-PearsonP.117-119nP.119-125nP.125,4.1;4.2;4.4;4.5;P.126,4.7;4.9n1.1BayesBayesng(X),cBayes1gi(X)=P(ωi/X)=P(X/ωi)P(ωi)i=1,2,…,c2gi(X)=Lij(X)-θijθij=P(ωj)/P(ωi)3gi(X)=Lij(X)-θijLij(X)=P(X/ωi)/P(X/ωj)θij=[(λij-λjj)/(λji-λii][P(ωj)/P(ωi)]n(i=1,2)g(X)g(X)=g1(X)–g2(X):X∈ω1(ω2),g(X)()0:g(X)=0:3.2nxGaussiansσ2=½01P(x|ω1)=(1/π1/2)exp(-x2);P(x|ω2)=(1/π1/2)exp(-(x-1)2)P(ω1)=P(ω2)=½x0(a)(b)L=(00.5;1.00)n:(a)x0x0:exp(-x2)=exp(-(x-1)2)lnx0=½(b)λ11=0,λ12=0.5,λ21=1.0,λ22=0;(λ21-λ22)/(λ12-λ11)=1.0/0.5=2x0:exp(-x2)=2exp(-(x-1)2)lnx0=(1–ln2)/2½xoxoω2xongi(X)=logP(X/ωi)+logP(ωi)P(X|ωi)=N(μi,Σi)gi(X)=lnP(ωi)-(n/2)ln2π-½ln|Σi|-½(X-μi)T(Σi)-1(X-μi)ni=1,2,…,cigi(X)gi(X)=lnP(ωi)-½ln|Σi|-½(X-μi)T(Σi)-1(X-μi)(3.3)§1Σi=Σ2Σi≠Σj1nΣi=Σ,(3.3)gi(X)=lnP(ωi)-½X’Σ-1X-½μi’Σ-1μi+μi’Σ-1X(3.4)nωiωjWij’(X–X0)=0Wij=Σ-1(μi-μj),X0=½(μi+μj)–(Pij)(μi-μj)/(μi-μj)’Σ-1(μi-μj)Pij=ln[P(ωi)/P(ωj)]n1X02X03P(ωi)=P(ωj)X04Σi=Σ=σ2IIσ22nΣi≠Σj,(3.3)gi(X)=X’WiX+wi’X+wi0(3.5)Wi=-1/2Σi-1wi=Σi-1μiwi0=lnP(ωi)-½ln|Σi|-½μi’Σ-1μinωiωjX’(Wi-Wj)X+(wi-wj)’X+(wi0-wjo)=0n3.2.3nBayesnnN→∞NN15nnnμi1xnΣinxnnn3.3n1.1Bayesnnω1ω2μ1=[173.6,66]’;μ2=[162,51]’Σ1=[29.8414.4;14.414];Σ2=[2613;1334]|Σ1|=210.4;|Σ2|=715Σ1-1=[0.06654-0.06844;-0.068440.141825]Σ2-1=[0.047552-0.01818;-0.018180.036364]nΣ1≠Σ2(3.5)P.96W=W1–W2=-0.5(Σ1-1-Σ2-1)=[-0.0094940.02513;0.02513-0.05273]w=w1–w2=Σ1-1μ1-Σ2-1μ2=[7.034221;-2.52091]–[6.776244;-1.09091]=[0.257997;-1.43]3.3w0=w10–w20=-530.748+525.035=-5.71265g(X)=X’WX+w’X+w0Xng(X1)=3.4048840;g(X6)=-0.614930g(X2)=5.886050;g(X7)=-14.38190g(X3)=0.3255010;g(X8)=-16.73620g(X4)=5.4252780;g(X9)=-10.29840g(X5)=3.4048840;g(X10)=-4.45893010%,nnnnnnParzenkp.134-152)nN→∞nn3.2.4p.158-171)nN→∞nnnNN≤n+1N=2(n+1)4)a)b)c)d)e)nP.171,5.12;Bayesn=/×100%nn1(n=/n×100%2(n–1)nnn=/n×100%3mm=mm=143.3FishernFishernXFisherNnXkω1ω2{X1}{X2}N1N2N=N1+N2yk=W’Xk,k=1,2,…,NWnXkykW’XkykykXkFisherFisherJF(W)=|m1–m2|2/(S12+S22)miSiW*FisherW*=Sw-1(m1–m2)mi=(Σk=1,Nixk)/Ni,i=1,2Sw=S1+S2Si=Σk=1,Ni(xk–mi)(xk–mi)’p.53-56)Fisher1ω1ω2{X1}{X2}2m1m23S1S24Sw5SwSw-16FisherW*=Sw-1(m1–m2)7Fisheryk=W*’Xk8y0y0=W*’(N1m1+N2m2)/(N1+N2)9yky0Xkω1yky0Xkω2yk=y03.4Fisher1.1Fishernω1ω2nmim1=[173.6;66];m2=[162;51]nSiS1=[149.272;7270];S2=[13065;65170]nSw=S1+S2=[243.2137;137240]nSw-1=[0.006061-0.00346;-0.003460.006142]nW*=Sw-1(m1–m2)=[0.01841;0.051991]nFisheryk=[0.018410.051991]Xk:y1=6.509053y2=6.861057y3=6.157049y4=6.693149y5=6.916286y6=6.065002y7=5.193086y8=5.285133y9=5.637137y10=5.9891413.4Fisherny0y0=W*’(N1m1+N2m2)/(N1+N2)=6.130609ny1~y56.130609ω1y6~y106.130609ω2Fishernn/n(2)FishernFishernFishernFishernnnnFisherBayesnnP.126,4.6;FisherP.171,5.12;FisherKnowledgeaccumulationismuchmoredifficultthanreplier.Amillionairecouldbecreatedatonenight,butnotascholar.3.4nBayesFishernnn3.4.1n1-NNcωi(i=1,2,…,c)nXj(I)(j=1,2,…,nXndi(X)=minj||X–Xj(I)||j=1,2,…,n1-NNX∈ωmdm(X)=minidi(X)i=1,2,…,cnk-NNk1,k2,…,kcXkω1,ω2,…,ωcωidi(X)=kik-NNX∈ωmdm(X)=maxidi(X)i=1,2,…,ck-NN3.4.2n1-NN[0,(c-1)/c]nk-NNkBayesp.173-1783.4.3n3.4.3n1NRNTNR+NT=NNNRNTNTNTENTE=NTNTEk–NNk–NN3.4.32)p.180-182nNBayesk–NNkp.184-1883.5n1.11-NN3-NNX(16758)ndi(X)=(Σi(X–Xi)2)1/2(a)1-NNXω1d1(X)=7.615773ω2d6(X)=7.28011d2(X)=14.42221d7(X)=17.69181d3(X)=2.828427d8(X)=14.76482d4(X)=14.76482d9(X)=8.246211d5(X)=16.27882d10(X)=4.242641k-NNdm(X)=minidi(X)=d3(X)=2.828427X(16758)∈ω1(b)3-NNXX3,X10,X6k1=1,k2=2k-NNdm(X)=maxidi(X)=maxi(k1,k2)=k2X(16758)∈ω23.6n1.11-NNX(16758)nXNR={X1,X3,X5,X7,X9}XNT={X2,X4,X6,X8,X10}X2d212=50,d232=200,d252=9,d272=1025,d292=500,9X2X5X2X4d452X5X4X6d632X3X6X8d872X7X8X10X3X9X10XNTE={X2,X4,