卷积神经网络研究综述-李彦冬

2016-03-302016-04-20。2012BAH44F02M17010601CXY2011057。1984—、1977—、1960—。1001-9081201609-2508-08doi10．11772/j．issn．1001-9081．2016．09．2508*，，(，610054)(*[e-mail])、、。、、、。TP181ASurveyofconvolutionalneuralnetworkLIYandong*，HAOZongbo，LEIHang(SchoolofInformationandSoftwareEngineering，UniversityofElectronicScienceandTechnologyofChina，ChengduSichuan610054，China)Abstract:Inrecentyears，ConvolutionalNeuralNetwork(CNN)hasmadeaseriesofbreakthroughresearchresultsinthefieldsofimageclassification，objectdetection，semanticsegmentationandsoon．ThepowerfulabilityofCNNforfeaturelearningandclassificationattractswideattention，itisofgreatvaluetoreviewtheworksinthisresearchfield．AbriefhistoryandbasicframeworkofCNNwereintroduced．ＲecentresearchesonCNNwerethoroughlysummarizedandanalyzedinfouraspects:over-fittingproblem，networkstructure，transferlearningandtheoreticanalysis．State-of-the-artCNNbasedmethodsforvariousapplicationswereconcludedanddiscussed．Atlast，someshortcomingsofthecurrentresearchonCNNwerepointedoutandsomenewinsightsforthefutureresearchofCNNwerepresented．Keywords:ConvolutionalNeuralNetwork(CNN);deeplearning;featurerepresentation;neuralnetwork;transferlearning。ConvolutionalNeuralNetworkCNN。。。LeNet-51、。。DeepBeliefNetworkDBN2ConvolutionalDeepBeliefNetworkCDBN34AlexNet5Ｒ-CNNＲegionswithCNN6FullyConvolutionalNetworkFCN7。8－10。1112－13。。11．1、。1。2060Hubel14ＲeceptiveField。1980FukushimaNeocognitron15。Neocognitron、。NeocognitronJournalofComputerApplications，2016，36(9):2508－2515，2565ISSN1001-9081CODENJYIIDU2016-09-10．joca．cn。2。1998Lecun1LeNet-5。。。LeNet-5。16、17、18。3。2012Krizhevsky5AlexNetImageNet1911%。AlexNetVGGVisualGeometryGroup8、GoogleGoogLeNet9、ＲesNet10AlexNetImageNet。。ＲecurrentNeuralNetworkＲNN20－2122－2324———3D25。1．2、、。1。。GoogLeNetInception、VGGＲesNetshortconnection。。。。2。。SIFTScale-InvariantFeatureTransform26、HOGHistogramofOrientedGradient27。。28－2924。30－31。、、。3。132、2533、34。35－36、37。GoogleAlphago38。。22．11、、、。1X。HiiH0=X。HiHiHi=fHi－1Wi+bi1Wii“”i－1ibifxiHi。39。12。HiHi=subsamplingHi－12Ylii。3H0Y。Yi=PL=li|H0Wb3LWb。H0“”。MeanSquaredErrorMSENegativeLogLikelihoodNLL40MSEWb=1|Y|∑|Y|i=1Yi－^Yi24NLLWb=－∑|Y|i=1logYi5L2λweightdecay90529EWb=LWb+λ2WTW6。Wb。ηWi=Wi－ηEWbWi7bi=bi－ηEWbbi82．22．1、1。、λ、η。810、2441、42－43。44。。2。。45。5846。。3。。30－31。。31234。3．1over-fitting40。、。。Hinton47Dropout。Dropout。Dropout。DropoutWan48DropConnect。DropoutDropConnect。DropConnectDropout。DropConnectGoodfellow42Maxout。DropConnectMaxout。Goodfellow42Maxout。2Dropout、DropConnectMaxout。2Dropout、DropConnect、MaxoutLin43MaxoutNINNetworkinNetwork。NINGlobalaveragepoolingNIN“”microneuralnetworkMaxout。。Zeiler39Stochasticpooling。AveragepoolingMaxpoolingStochasticpooling。。10152362、、、。3．2Lecun1LeNet-5。LeNet-512LeNet-53。。。LeNet-5Krizhevsky5AlexNet。AlexNet5656000LeNet-5。AlexNetImageNet19。ImageNet1000120。AlexNetdropout。AlexNetGPUCPUGPU。AlexNetImageNet201211%。AlexNet。Simonyan8AlexNetVGG。VGG3×3Simonyan。49。VGG16～19。He10。。HeＲesNet。ＲesNetshortconnectionsx。FxshortconnectionFx+x。HeFx+xFx。xshortconnection0。shortconnection。ＲesNetVGG152ＲesNetVGG。ＲesNet123。Szegedy9。Inception。3Inception1×13×35×512。InceptionGoogLeNetAlexNet1/12ImageNetAlexNet10%。3InceptionSpringenberg50“”。“”。12。VGG、ＲesNet。GoogLeNet、。3．3。。Donahue30t-SNE51。t-SNE。t-SNEDonahueGISTGIST52LLCLocality-constrainedLinearCoding53t-SNE11529t-SNE。Donahue12Donahue3t-SNE。4Zeiler24t-SNE。DeConvNet54。4。、。ZeilerAlexNet。、。Zeiler。Nguyen55。5Nguyen56。Nguyen。。5“”1、2Nguyen55“”。。3．4“”5711。“”。61ImageNet2Caltech3。6Zeiler24ImageNetCaltech-10158Caltech-2565940%。ImageNetCaltech。Donahue30ZeilerImageNetdomainadaption、subcategoryrecognitionscenerecognition。Ｒazavian31。Zhou60ImageNetPlaces60。ImageNetPlaces。122152363。123。4。1。AlexNetImagNet84．7%AlexNetVGG8、GoogLeNet9、PＲeLU-net46BN-inception61。ＲesNet10ImageNet96．4%ＲesNetAlexNet。。2。。。Gishick662Ｒen63Ｒ-CNN6、FastＲ-CNN62FasterＲ-CNN63。Ｒ-CNNCNNregionproposals64。Ｒ-CNNregionproposals。FastＲ-CNNregionproposalsregionproposalsregionproposalsFastＲ-CNN。FasterＲ-CNN7regionproposals64。。3。Khan6566－67Levi68Long7FCNZhou60Ji253D。。4。AlphaGo38Abdel-Hamid37HiddenMarkovModelHMMKalchbrenner35Donahue20LＲCNLong-termＲecurrentConvolutionalNetwork。。、。51。。。2。。。3。4。。、。5FasterＲ-CNNFCN。6。。6、、、、31529。。。［1］LECUNY，BOTTOUL，BENGIOY，etal．Gradient-basedlearningappliedtodocumentrecognition［J］．ProceedingsoftheIEEE，1998，86(11):2278－2324．［2］HINTONGE，OSINDEＲOS，TEHYW．Afastlearningalgorithmfordeepbeliefnets［J］．NeuralComputation，2006，18(7):1527－1554．［3］LEEH，GＲOSSEＲ，ＲANGANATHＲ，etal．Convolutionaldeepbeliefnetworksforscalableunsupervisedlearningofhierarchicalrep-resentations［C］//ICML'09:Proceedingsofthe26thAnnualInter-nationalConferenceonMachineLearning．NewYork:ACM，2009:609－616．［4］HUANGGB，LEEH，EＲIKG．Learninghierarchicalrepresenta-tionsforfaceverificationwithconvolutionaldeepbeliefnetworks［C］//CVPＲ'12:Proceedingsofthe2012IEEEConferenceonComputerVisionandPatternＲecognition．Washington，DC:IEEEComputerSociety，2012:2518－2525．［5］KＲIZHEVSKYA，SUTSKEVEＲI，HINTONGE．ImageNetclassi-ficationwithdeepconvolutionalneuralnetworks［C］//ProceedingsofAdvancesinNeuralInformationProcessingSystems．Ca