SNN与Deep-Learning区别

Shortanswer:Strictlyspeaking,DeepandSpikingrefertotwodifferentaspectsofaneuralnetwork:Spikingreferstotheactivationofindividualneurons,whileDeepreferstotheoverallnetworkarchitecture.Thusinprinciplethereisnothingcontradictoryaboutaspiking,deepneuralnetwork(infact,thebrainisarguablysuchasystem).However,inpracticethecurrentapproachestoDLandSNNdon'tworkwelltogether.Specifically,DeepLearningascurrentlypracticedtypicallyreliesonadifferentiableactivationfunctionandthusdoesn'thandlediscretespiketrainswell.Furtherdetails:Realneuronscommunicateviadiscretespikesofvoltage.Whenbuildinghardware,spikinghassomeadvantagesinpowerconsumption,andyoucanroutespikeslikedatapackets(AddressEventRepresentationorAER)toemulatetheconnectivityfoundinthebrain.However,spikingisanoisyprocess;generallyasinglespikedoesn'tmeanmuch,soitiscommoninsoftwaretoabstractawaythespikingdetailsandmodelasinglescalarspikerate.Thissimplifiesalotofthings,especiallyifyourgoalismachinelearningandnotbiologicalmodeling.ThekeyideaofDeepLearningistohavemultiplelayersofneurons,witheachlayerlearningincreasingly-complexfeaturesbasedonthepreviouslayer.Forexample,inavisionsetting,thelowestlevellearnssimplepatternslikelinesandedges,thenextlayermaylearncompositionsofthelinesandedges(cornersandcurves),thenextlayermaylearnsimpleshapes,andsoonupthehierarchy.Upperlevelsthenlearncomplexcategories(people,cats,cars)orevenspecificinstances(yourboss,yourcat,thebatmobile).Oneadvantageofthisisthatthelowest-levelfeaturesaregenericenoughtoapplytolotsofsituationswhiletheupperlevelscangetveryspecific.ThecanonicalwaytotrainspikingnetworksissomeformofSpikeTimingDependentPlasticity(STDP),whichlocallyreinforcesconnectionsbasedoncorrelatedactivity.ThecanonicalwaytotrainaDeepNeuralNetworkissomeformofgradientdescentback-propagation,whichadjustsallweightsbasedontheglobalbehaviorofthenetwork.Gradientdescenthasproblemswithnon-differentiableactivationfunctions(likediscretestochasticspikes).Ifyoudon'tcareaboutlearning,itshouldbeeasiertocombinetheapproaches.Onecouldpresumablytakeapre-traineddeepnetworkandimplementjustthefeed-forwardpart(nofurtherlearning)asaspikingneuralnet(perhapstoputitonachip).Theresultingchipwouldnotlearnfromnewdatabutshouldimplementwhateverfunctiontheoriginalnetworkhadbeentrainedtodo.