搜索
Z000vol`72VC”ResearehoftheCVDmonsonComputatoznofNeraulNetworks`z,(100084)’(210093)2AbstractThere,5somerelatlonshipbetweenthearehteetureandgeneralizationoftheneuralnet-work,andthesizeofthetralningset.AsthekernelofPAClearnngandthemeasureofthelearninga-bllityoflearnlngmaehlne,VCdlmensionplaysanimportantrolentherelationshxp·50.theresearehofVCdrmenslonwllhghyprofitneuralnetworktheoryandteehnology.Inthispaper,bsedonthestatis-ticalIearningtheory,thetotheminuteaehievementsandresearchevolvementoftheVCdlmensloneomPutatlonofneuralnetworksareintrodueed,andsomefutureresearehissuesarealsodseussed.KeywordsVCdimension,Neuralnetworks,Patternreeogniton,Maehinelearning,Computationallearningtheory,l1[,,。,,。,,“〕。PAe(ProbablyApproximateyeorreet,)。PAC,vc(vaP-nikChervonenkLSdimensLon)、、。VC,;,,。Horink,“,NP”〕,,vC。PAC—,)(:69625103),VC,VC,。2,〔,」。,,。,。Vapn;k[5〕,,,VC。,VC。,。{(x,,y:),(xZ,yZ),…,(x,,y,)},x〔R”,yo,l}。,x,n,y。(x,。,{f(x,a),a,af(·),a。。。’,f(x,59a)y,。,(1):(。){r,(,a)d(,,)(l)JZ`(x,y)。(x,,R(a)。,,(Emp,r,ealR;sk)R`,(a),(2):;。,,,(。)l,f(,。)I“,~1(2)(x,y),l,。,,R`,,(a)R(a)。,,,(a)R(a),R,,,(a)f(x,a,)R(a)f(x,a。)。,,,(a)(a),,。(a)(a)。Vapn;kChervonenkis,」,{f(x,a).aA}VC。vink,R(。),。1,1:(。)`,,(。)+〔(In(21/)+l)In(,/4)〕l(3)h{f(x,a),aA}VC,l。(3)VC。(3),VC,.R,,(a),R(a)。,,VC。,。,VC。,、。。,m,f:R·~R,。a、,A,a`f(,a’)。,f(·)VC,VC。,,VC。VC,,,VC`,VC,;,,VC,。,VC。3VCVCl0[〕:FnX{。,1},FVCXE,E:SE,F,xS(x)1,xSxE(x)0。VC,,,F(,,VCVC)〔。,{f(x,a),aA}。,f(x,a),。A}vC,VC。,60.4VCVCCvoer1[2〕,,Vink,Bltlme:「”,VallantPAC,vC,。VC。VidyasagarVCl[`〕。,VC,VC。,、iSgoiod。,。,:O、。fg,`:o`,g,f=O(g);eZo》cZg,f=。(g);f0(g)f(g),f=(g)。(l)Cvoer〕aBum,,w,Ve0(wlogw),109(·)z。1994,MaassVC`SJ,l()、;d3(),r。,`((),(,,)1,41ogn2,r。(),VCVC(p。)(。2109。)Maass,,,w,VC。(wlogw)。,sakura,1995[’6〕,,,VC。(wlog)。1999,CarterOxley`7」ve〔`,」,Po`neare,,]VC,2:2v;,vZ,4,v〔Rd,t:,tZ,t,,、,,2,…,w,R,r,,:F.,{f:R`~RIf()w;(。;,`),v;,。`,t,}VC:(4)、、、.`k.1`、J.ve(N)ve(:、.)(5)2,,。,。,VC。,1998K。:ranoSntag20[〕,,VC。(wlog(k/w)),。(min{wklogwk,2+wlogwk})。k。(2)1995,GoldbergJerrum〔,,VC:VC(N)c、(wZ+wk)O(wZ)(6)w,k。1997,Ko,ranSontag〔2,]VC(w,),3:3。」,,l1,,。、O(;,)。VC,:a)。.VCO(,ll09;b)。〔afifne),VCO(n);c),a。b。(x)~“xb,VCO(),VC(09,l)。1998,Bardett〕VC,45:4w,k(,Lw.l,,w,Lk,,,l,。F,VC(sgn(F))2Llog(ZeLPk)+2JLZlog(l+l)+ZL(7)LkO(),l,:VC(sgn(F))=O(wLlogw+wLZ)(8)Sf:R~R:a)lmf(a)1,1imf(a)=0;b)fx。f(x。)。。L1,w)loL14,Lw,,f,:VC(sgn(F))tL/2w/2」(9)“。45,L~0(w),VC(wL);L,VCO(wlogw),GoldbergJerrum。,1998KioranoSntag〕,,VCwk。,VC(wk),O(Zk)。k。(3)Sig.oidSmoid,KoiranSontag2997〔vC(,),3。Bartlett[23〕VC(wL),5,w,.61.l。1997,Karp;nsklMae,ntyre`2J,VCO(wZ),k。,1998KolarnSontag`0」,s,gmo:d,VC(wk),O(w`)。k。51994,MaassVC,。,:(l),VC,,VCwlogw。,,Sm。Gaussian。,VC。Sigmiod,VCO(wZ)(wZ),。,。(2),VC、。、,、。,VC,。,earte:oxley[,’〕。(3),、,。,。,VC,。,。Kioarnosnta。(4)fL,(neuralnetworken-semble).27〕.,、。,,。,·62·VC,,。〔5)VC,“”,。,VC,“”,。,。,VC。VC,,。,Haussler[“〕Takahashi·,。1,..;,19982WolpertD.TheMathematiesofGeneralization:In:Proc.ofthe1992SFI/CNLSWorkshoponForma1AproaehestoSupervisedLearning,Reading,MA:Addison-Wesley,19953VaIiantLG.ATheoryoftheLearnable.CommunieationsoftheACM,1984,27(11):1134~11424HornikKM,StineheombeM,WhiteH.MultilayerFeed-forwardNetworkareUniversalAPProximators.NeuralNetworks,1989.2:359~3665VapnikV.EstlmarionofeDPendene`esaBsedonEmpiriealaDta.NewYorkspringerVerlag.19826aBumEB.HausslerD.WhatSizeNetGivesVa1idGener_alization?NeuralComputation,1989,1:151~1607LeCunY,etal.OptimalBrainDamage·In:TouretzkyD,ed,AdvaneesInNeuralInforatntionProeessingSystems(VolumeZ).Denver.CO:MorganKaufmann.19908Guyonl,etalStrueturalRiskMinimiaztionforCharaeterReeogn:t:on·In:MoodyJ,HansonS,LippmannR,eds.AdvaneesinNeuralInformationProcessingSy,tems(Volume4).eDnver,CO:MorganKaufamnn,19929VapnikVN.ChervonenkisAY.0ntheUniofrmConver-geneeofRelatxveFrequeneiesofEventstoTheirProbabil-ities.TheoryofProbabilityandItsApPlieations.1971,16(2):26428010AnthonyM·ProbabilistieAnalysisofLearningxnArtifi-ealNeuralNetworks:ThePACModelandItsVarants.NeuralComputingSurveys,1997,l:l~4711VapnikV.TheNatureofStatistiealLearningTheo-ry.NewYork:SpringerVerlag,199512CoverTM.GeometryandStatistiealPropertiesofSys-tetnsofLinearInequalitleswithAPplieationsinPatternReeognition.IEEEtTan,aetionsonEleetronieOCmPuters.·1965.6:326~33413lBumerA,etal.LearnabilityandtheVapnikChervo-nenk,Dimenson.J.oftheACM,1989.36:929~96514VidyasagarM·AhTeoryofLearningadnGeneraliza-tlonLondon:SpringerVerlag.199715MaassW.NeuralNetswithSurliarVCmen-sion.NeuralComputation,1994,6:877~884(78)0.6}x()’\\M\、/`、\ùùOùùU,JJ.....l.esL..`,.`1..1`1......!,lì......-:“nnUU6llUnóǎU“ù“U、ó·.,...`O…nUU”\//\/l///13060708090100110120130lrlll40,、ùq,、ó,ó1nU1B,lAA,。。。23451ChengLIChun,oBxingPeng,TheFuzyRelationEquationWithUnionorInterseetionPerseveringOPerator.FuzzyeStsandSystem,1988,25:191~204,..:,1998.:〔〕.,2996MizumotoM,ZimmermannHComparisonofFuzzyRea-soningMethod.FuzzyeStsandSystem.1982,8:253283.I.(E).1999,29(1):43~53(62)16aSkuraiA.TghterBoundsontheVCDimenslonofThreeLayerNetworks.In:Proc,oftheWorldCongressonNeuralNetworks.Hillsdale,NJ:Erlbaum,1993.54054317CarterMA,OxleyME.EvaluatingtheVapnikChervo-nenklsDimensionofArtifieialNeuralNetworksUsing