卷积神经网络

1ConvolutionalneuralnetworksAbin-Roozgard2IntroductionDrawbacksofpreviousneuralnetworksConvolutionalneuralnetworksLeNet5ComparisonDisadvantageApplicationPresentationlayout3introduction4IntroductionConvolutionalneuralnetworksSignalprocessing,Imageprocessingimprovementoverthemultilayerperceptronperformance,accuracyandsomedegreeofinvariancetodistortionsintheinputimages5Drawbacks6StructureTypesofDecisionRegionsExclusive-ORProblemClasseswithMeshedregionsMostGeneralRegionShapesSingle-LayerTwo-LayerThree-LayerHalfPlaneBoundedByHyperplaneConvexOpenOrClosedRegionsArbitrary(ComplexityLimitedbyNo.ofNodes)AABBAABBAABBBABABABehaviorofmultilayerneuralnetworks7Multi-layerperceptronandimageprocessingOneormorehiddenlayersSigmoidactivationsfunctions8thenumberoftrainableparametersbecomesextremelylargeDrawbacksofpreviousneuralnetworks9Littleornoinvariancetoshifting,scaling,andotherformsofdistortionDrawbacksofpreviousneuralnetworks10Littleornoinvariancetoshifting,scaling,andotherformsofdistortionDrawbacksofpreviousneuralnetworksShiftleft11Drawbacksofpreviousneuralnetworks154inputchangefrom2shiftleft77:blacktowhite77:whitetoblack12scaling,andotherformsofdistortionDrawbacksofpreviousneuralnetworks13thetopologyoftheinputdataiscompletelyignoredworkwithrawdata.DrawbacksofpreviousneuralnetworksFeature1Feature214DrawbacksofpreviousneuralnetworksBlackandwhitepatterns:Grayscalepatterns:32*3210242232*32102425625632*32inputimage15AADrawbacksofpreviousneuralnetworks16ImprovementFullyconnectednetworkofsufficientsizecanproduceoutputsthatareinvariantwithrespecttosuchvariations.TrainingtimeNetworksizeFreeparameters17Convolutionalneuralnetwork(CNN)18In1995,YannLeCunandYoshuaBengiointroducedtheconceptofconvolutionalneuralnetworks.HistoryYannLeCun,ProfessorofComputerScienceTheCourantInstituteofMathematicalSciencesNewYorkUniversityRoom1220,715Broadway,NewYork,NY10003,USA.(212)[e-mail]19CNN’sWereneurobiologicallymotivatedbythefindingsoflocallysensitiveandorientation-selectivenervecellsinthevisualcortex.Theydesignedanetworkstructurethatimplicitlyextractsrelevantfeatures.ConvolutionalNeuralNetworksareaspecialkindofmulti-layerneuralnetworks.AboutCNN’s20CNNisafeed-forwardnetworkthatcanextracttopologicalpropertiesfromanimage.Likealmosteveryotherneuralnetworkstheyaretrainedwithaversionoftheback-propagationalgorithm.ConvolutionalNeuralNetworksaredesignedtorecognizevisualpatternsdirectlyfrompixelimageswithminimalpreprocessing.Theycanrecognizepatternswithextremevariability(suchashandwrittencharacters).AboutCNN’s21ClassificationConvolutionalneuralnetworkClassificationClassificationPre-processingforfeatureextractionInputf1…fnoutputf1f2outputFeatureextractionInputShiftanddistortioninvarianceclassification22CNN’sTopologyFeatureextractionlayerConvolutionlayerShiftanddistortioninvarianceorSubsamplinglayerCSFeaturemaps23FeatureextractionlayerorConvolutionlayerfeaturesdetectthesamefeatureatdifferentpositionsintheinputimage.24Featureextractionw13w12w11w23w22w21w33w32w3110-110-110-1ConvolvewithThresholdw13w12w11w23w22w21w33w32w3125FeatureextractionInputsCSSharedweights:allneuronsinafeaturesharethesameweights(butnotthebiases).Inthiswayallneuronsdetectthesamefeatureatdifferentpositionsintheinputimage.Reducethenumberoffreeparameters.26FeatureextractionIfaneuroninthefeaturemapfires,thiscorrespondstoamatchwiththetemplate.27SubsamplinglayerthesubsamplinglayersreducethespatialresolutionofeachfeaturemapByreducingthespatialresolutionofthefeaturemap,acertaindegreeofshiftanddistortioninvarianceisachieved.28thesubsamplinglayersreducethespatialresolutionofeachfeaturemapSubsamplinglayer29Subsamplinglayer30SubsamplinglayerTheweightsharingisalsoappliedinsubsamplinglayers.31theweightsharingisalsoappliedinsubsamplinglayersreducetheeffectofnoisesandshiftordistortionFeaturemapSubsamplinglayer32Uptonow…33LeNet534LeNet5IntroducedbyLeCun.rawimageof32×32pixelsasinput.35LeNet5C1,C3,C5:Convolutionallayer.5×5Convolutionmatrix.S2,S4:Subsamplinglayer.Subsamplingbyfactor2.F6:Fullyconnectedlayer.36LeNet5AlltheunitsofthelayersuptoF6haveasigmoidalactivationfunctionofthetype:()tanh()jjjyvASv370F1F2F84W1W2W84Y0LeNet58421(),0,...,9jiijiYFWj+1+1+138LeNet5About187,000connection.About14,000trainableweight.39LeNet540LeNet541Comparison42Database:MNIST(60,000handwrittendigits)Affinedistortion:translation,rotation.elasticdeformations:correspondingtouncontrolledoscillationsofthehandmuscles.MLP(thispaper):has800hiddenunit.Comparison43“Thispaper”refertoreference[3]onreferencesslide.Comparison44Disadvantages45DisadvantagesFromamemoryandcapacitystandpointtheCNNisnotmuchbiggerthanaregulartwolayernetwork.Atruntimetheconvolutionoperationsarecomputationallyexpensiveandtakeupabout67%ofthetime.CNN’sareabout3Xslowerthantheirfullyconnectedequivalents(size-wise).46DisadvantagesConvolutionoperation4nestedloops(2loopsoninputimage&2loopsonkernel)SmallkernelsizemaketheinnerloopsveryinefficientastheyfrequentlyJMP.Cashunfriendlymemoryacce