基于预训练神经网络的小样本图像分类(IJEM-V8-N4-5)

I.J.EngineeringandManufacturing,2018,4,40-55PublishedOnlineJuly2018inMECS()DOI:10.5815/ijem.2018.04.05Availableonlineat“GeneralMihailoApostolski”,University“GoceDelcev”,1000Skopje,RMacedoniabFacultyofComputerScience,University“GoceDelcev”,2000Stip,RMacedoniaReceived:12March2018;Accepted:19April2018;Published:08July2018AbstractNowadaystheriseoftheartificialintelligenceiswithhighspeed.Evenwearefarawayfromthemomentwhenmachinesaregoingtomakedecisionsinsteadofhumanbeings,thedevelopmentinsomefieldsofartificialintelligenceisastonishing.Deepneuralnetworksaresuchafiled.Theyareinabigexpansioninanewmillennium.Theirapplicationiswide:theyareusedinprocessingimages,video,speech,audio,andtext.Inthelastdecade,researchesputspecialattentionandresourcesinthedevelopmentofspecialkindofneuralnetworks,convolutionalneuralnetworks.Thesenetworkshavebeenwidelyappliedtoavarietyofpatternrecognitionproblems.Convolutionalneuralnetworksweretrainedonmillionsofimagesanditisdifficulttooutperformtheaccuraciesthathavebeenachieved.Ontheotherhand,whenwehaveasmalldatasettotrainthenetwork,thereisnosuccesstodoitfromascratch.Thisarticleexploitsthetechniqueoftransferlearningforclassifyingtheimagesofsmalldatasets.Itconsistsfine-tuningofthepre-trainedneuralnetwork.Hereindetailsispresentedtheselectionofhyperparametersinsuchnetworks,inordertomaximizetheclassificationaccuracy.Intheend,thedirectionshavebeenproposedfortheselectionofthehyperparametersandofthepre-trainednetworkwhichcanbesuitablefortransferlearning.IndexTerms:Pre-trainedneuralnetworks,deeplearning,transferlearning,accuracy,hyperparameters,smalldatasets.©2018PublishedbyMECSPublisher.Selectionand/orpeerreviewunderresponsibilityoftheResearchAssociationofModernEducationandComputerScience.1.IntroductionArtificialintelligence(AI),deeplearning(DL),andneuralnetworks(NN)arepowerfulmachinelearning-basedtechniques.Thesetechniquesareincrediblyexcitingandusedtosolvemanyreal-worldproblems.*Correspondingauthor.E-mailaddress:ClassificationofSmallSetsofImageswithPre-trainedNeuralNetworks41Ononehand,human-likedeductivereasoninganddecision-makingbyacomputerisstillalongtimeaway.Butontheotherhand,therehavebeenremarkablegainsintheapplicationofAItechniquesandassociatedalgorithmsofAI.PopularexamplesofanAIsolutionincludesIBM’sWatson,Apple’sSiriandAmazon’sAlexa.WatsonwasmadefamousbybeatingthetwogreatestJeopardychampionsinhistory.Itisnowbeingusedasaquestionansweringcomputingsystemforcommercialapplications[29].SofarAIhasbeenusedforspeechrecognitionandnaturallanguageapplications(processing,generation,andunderstanding).Itisalsousedforotherrecognitiontasks(pattern,text,image,video,audio,facial…),autonomousvehicles,medicaldiagnoses,gaming,searchengines,robotics,spamfiltering,crimefighting,marketing,remotesensing,transportation,classification,etc.TherearemanydifferentgoalsofAIasmentioned,withdifferenttechniquesusedforeach.Theprimarytopicsofthisarticlearedeepneuralnetworks,especiallyonecertainkindofthem–convolutionneuralnetworksandtheirusefortransferlearning.Atthetimeofthiswriting,therearealotofpre-trainedconvolutionalneuralnetworks,developedbyscientistsorbigcorporations.Themainpurposeofthesenetworksistosolveimageclassificationproblems.EventhattheabovementionedCNNweretrainedoncertainimagesets,theycanbeusedforimageclassificationsonothersetsofimages.Thisissocalled‘transferlearning’.Transferlearningisatechniqueofoptimizationofpre-trainedCNNinordertoclassifyasetofimagesonwhichitwasnottrainedbefore.Optimizationactuallyconsistsselectionofthehyperparametersoftheneuralnetwork.NomenclatureAIArtificialIntelligenceDLDeepLearningNNNeuralNetworkCNNConvolutionalNeuralNetworkBPBackpropagationSLSupervisedLearningULUnsupervisedLearningRLReinforcementLearningRNNRecurrentNeuralNetworkMLPMultilayerPerceptronKSHKrizhevsky,Sutskever,andHintonILSVRCImageNetLarge-ScaleVisualRecognitionChallengeGPUGraphicProcessorUnitRGBRedGreenBlueReLURectifiedLinerUnit2.NeuralNetworksandDeepLearningAstandardneuralnetwork(NN)consistsofmanyneurons.Theyaresimple,connectedprocessors,eachproducingasequenceofreal-valuedactivations.Inputneuronsdifferfromtheotherneuronsoftheneuralnetwork.Theygetactivatedthroughsensorsperceivingtheenvironment.Otherneurons(hiddenandoutput)getactivatedthroughweightedconnectionsfrompreviouslyactiveneurons.Asinputneuronsareaffectedbythesurrounding,alsosomeneuronsmayinfluencetheenvironment.LearningisaboutfindingweightsthatmaketheNNbehavesinthedesiredway,suchasdrivingacar.Suchbehaviormayrequirelongcausalchainsofcomputationalstages.Itdependsontheproblemandhowtheneuronsareconnected.Eachstagetransformstheaggregateactivationofthenetwork.Asmentionedinreference[1]DeepLearningisaboutaccuratelyassigningweightsacrossmanysuchstages.Inthehistoryofneuralnetworksfirst,shallowNN-likemodelsappeared.ShallowNNswithseveral42ClassificationofSmallSetsofImageswithPre-trainedNeuralN