Dynamics of random neural network with synaptic de

DynamicsofaRandomNeuralNetworkwithSynapticDepressionW.Senn,K.Wyler,J.Streity,M.Larkumy,H.-R.Luschery,F.Merz,H.Meyz,L.Muller,D.Stainhauser,K.VogtyandTh.WannieryDecember7,1994AbstractWeconsiderarandomlyconnectedneuralnetworkwithlinearthresholdelementswhichupdateindiscretetimesteps.Thetwomainfeaturesofthenetworkare:1)equallydistributedandpurelyexcitatoryconnectionsand2)synapticdepressionafterrepetitivering.Wefocusonthetimeevolutionoftheexpectednetworkactivity.The4typesofqualitativebehaviorareinvestigated:Singularexcitation,convergencetoaconstantactivity,oscillationandchaos.Theiroccurrenceisdiscussedasafunctionoftheaveragenumberofconnectionsandthesynapticdepressiontime.Ourmodelreliesonexperimentswithaslicecultureofdisinhibitedembryonicratspinalcord.Thedynamicsofthesenetworksessentiallydependsonthefollowingcharacteristics:thelownon-structuredconnectivity,thehighsynapticdepressiontimeandthelargeEPSPwithrespecttothethresholdvalue.KeywordsRandomneuralnetwork,Thresholdelement,Synapticdepression,Aver-ageactivity,Dynamics,Oscillation,Disinhibitedconnections,Embryonicspinalcord.1IntroductionArticialneuralnetworksdesignedasmodelstounderstandthefunctioningofbiologicalnetworksareusuallybasedonexperimentaldatafromadultoratleastpostnatalanimals.Suchnetworksconsistofhighlystructuredexcitatoryandinhibitorysynapticconnections,ahighdegreeofconvergenceanddivergencewithcorrespondinglowweightsofsingleunitconnectionsandthusofneuronswithamanifoldrepertoireofintegrativepropertiesandawiderangeoffrequenciesofoutputpatterns.Theanalysisofsuchnetworksisusuallytask-oriented.Thatis,itsarticialmodelisdesignedtoperformthesametaskasthebiologicalnetworkisbelievedtodo.Thedisadvantageofsuchanapproachisthatacertainbiologicaltaskmaybeperformedbyavarietyofdierentlystructurednetworksandthereforeasimpleanalogycannotbeconclusiveconcerningtheactualbiologicalmechanism.InotherwordsitisquestionablewhetherthestructureofanetworkisstrictlyInstitutfurInformatikundangewandteMathematik,UniversitatBernyPhysiologischesInstitut,UniversitatBernzAscomTechAG1determinedbyitsfunction.Moreprobably,anadditionaldimensionhastobeconsideredinordertounderstandanetworksstructureandthisisitsdevelopmentalhistory.Duringdevelopmentthepropertiesofneuronsandneuronalnetworksundergodramaticchanges:Embryonicnetworksoftenlackfunctionalinhibitorycontacts(Oppenheim,1975).Thedegreeofconvergenceanddivergenceislowwithcorrespondinghighweightsofsingleunitconnections(Streitetal.,1991)andconsequentlytheintegrativepropertiesoftheneuronsareoflowimportance.Inaddition,therangeofoutputfrequenciesofneuronsisstronglylimitedbythepoorfrequencyresponseofimpulseconductionandsynaptictransmission.Inconsequence,synapticecacyrapidlydecreasesatfrequenciesabove1Hz(Streitetal.,1992).Inordertolearnsomethingaboutthepossibilitiesofactivity-dependentself-organizationofembryonicnetworksduringperiodsofspontaneousactivity,thedynamicsofrandomlyconnectednetworkswithhighindividualweights,alowdegreeofconnectivityandstrongsynapticdepressionwereanalyzedfortheircapabilityofprovidingastableoscillatoryoutput.Suchnetworkswithastableoscillatoryoutputbasedonendogenouspropertiesweredescribedinaratspinalcordculturesystem(Streit,1993),andarebelievedtobeinvolvedintheformationofspinalpatterngeneratorsforlocomotion(O’Donovanetal.,1992;Grillner,1985).Itisshownthatstableoscillationsofactivitycanariseinsuchnetworkssimplyasaresultoftheendogenouspropertiesmentionedaboveregardlessoftheparticularnetworkstructure.Furthermorethepatternsofactivitycanbealteredbychangingsingleparametersliketheaveragesynapticweightorthemeanrateofconnec-tivity.ThedynamicsofahomogeneousrandomnetworksimilartoourswasrstinvestigatedtheoreticallybyS.Amari(Amari,1971).Incontrasttohisworkweconsiderinadditionthesynapticdepressionafterring.Intypicaltime-discreterandomnetworksthesynapticdepressionismodeledbyanabsoluterefractoryperiodlyingbetweenonceandtwicetheinterneuraldelay,seee.g.(Anninosetal.,1970;Fournouetal.,1993).However,thisisaverystrongrestriction.Inourapproachanarbitrarysynapticdepressiontimeisallowedandwehaveacontinuousregenerationofthesynapticecacy.OscillationsofanexcitatorynetworkwithinterneuraldelaysandphenomenologicallydescribedneuronswereinvestigatedbyJ.-F.Vibertandrecentlyappearedinthisjournal(Vibertetal.,1994).Althoughwedonotconsidersuchdelays,thevariouspatternsofnetworkactivityasoscillations,chaosoraconstantactivitycanbeexplained.Withappropriatemodicationstotheneuronmodelweareabletogivesomerigorousmathematicalstatementsonthelong-termbehaviorofnetworkactivity.WepickupVibert’squestionofwhetherrhythmicpatternsrelyonpacemakercellsorwhethertheyareanemergingpropertyofthenetwork(Vibertetal.,1994,p.589).Inthecaseoftheembryonicnetworks,ourresultssuggestthatoscillationsindeedoriginatefromthetypicalrangeofparametervaluesmeasuredinthesliceculturesofembryonicratspinalcord.Inparticular,pacemakercellsseemnottobenecessary.Thepaperisorganizedasfollows:Section2statesthemathematicalmodeltogetherwiththebiologicalassumptions.Twoversionsofthesynapticdepressionar