Effective neural response function for collective

January12,1999EectiveneuralresponsefunctionforcollectivepopulationstatesDanielJ.Amit1andMassimoMascaroDipartimentodiFisica,UniversitadiRoma\LaSapienza,P.leAldoMoro,RomaAbstractCollectivebehaviorofneuralnetworksoftendividestheensembleofneuronsintosub-classesbydierentneurontypes;byselectivesynap-ticpotentiation;orbymodesofstimulation.Whenthenumberofclassesbecomeslargerthantwo,theanalysis,eveninamean-eldtheory,loosesitsintuitiveaspectbecauseofthenumberofdimen-sionsofthespaceofdynamicalvariables.Oftenoneisinterestedinthebehaviorofareducedsetofsub-populations(infocus)andintheirdependenceonthesystem’sparameters,asinsearchingforcoex-istenceofspontaneousactivityandworkingmemory;inthecompeti-tionbetweendierentworkingmemories;inthecompetitionbetweenworkingmemoryandanewstimulus;orintheinteractionbetweenselectiveactivityintwodierentneuralmodules.Forsuchcaseswepresentamethodforreducingthedimensionalityofthesystemtooneortwodimensions,evenwhenthetotalnumberofpopulationsinvolvedishigher.Inthereducedsystemthefamiliarintuitivetoolsapplyandtheanalysisofthedependenceofdierentnetworkstatesonambientparametersbecomestransparent.More-over,whenthecodingofstatesinfocusissparse,thecomputationalcomplexityismuchreduced.Beyondtheanalysiswepresentasetofdetailedexamples.Weconcludebyadiscussionofquestionsofstabilityinthereducedsystem.1IntroductionInrecurrentneuralnetworkswithattractordynamics(withintensivefeedback)onelooksfortheemergingselectivecollectivebehaviorofsubsetsofneurons,such1andRacahInstituteofPhysicsHebrewUniversity,Jerusalem1asthoseaectedbystimuliorsubjecttolearning.Thesearchissimpliedusingtheextendedmeaneld(MF)approximation,startingfromthedynamicsoftheindividualspikingcells.DespitethesimplicationbroughtaboutbytheMFap-proximation,whenthenumberofdierentsub-populationsincreases,onemuststillsolvealargenumberofcoupled,non-linearequationswithmanyfreeparam-eters.Apartfromthecomputationalcomplexityofthesearchforsolutionswithinterestingstructure,thereisthetopologicalcomplexityofamulti-dimensionalspace(ofallthepopulations).Moreover,thestabilityanalysisofthesolutionsisrathercumbersome.Thedicultyisfurtherincreasedifthesystemiscomposedofmorethanonemodule,aswouldbeinferredbyexperimentssuchasthoseofDesimoneetal.[1].Herewepresentamethodforstudyingthebehaviorofsmallsub-populationsintheMFapproximationbyextractinganeectivesingleneuronresponsefunc-tionofatypicalneuroninasub-populationunderstudy.Theresponsefunctioncontainsallthefeed-backeectsofalltheothersub-populationstovariationsinthespikerateoftheneuronobserved,sothattheresultingstationarystatesareexactstationarystatesofthecompletesystem.Intheprocess,thedimension-alityoftheproblemismuchreduced,allowingdirect,intuitiveinsightsintothepotentialstationarystatesaswellasintotheirstability.Stepsinthisdirectionhavebeenproposedinspecicsituations(seee.g.[15,16]).Herewegeneralizetheapproach,analyzefurtheritspropertiesandapplyittomorecomplicatedsituations.Themethodapplieswheneverthedynamicalchangeinthepopulationinfocusdoesnotdestabilizethestationarystateoftheambientpopulationspre-existenttothechange.Thisconditioncanbeshowntobesatisedincasesofsparsecoding,inwhichthemethodalsoreducesverysignicantlythecomputationalcomplexity.Butevenwhenthecomputationalloadisnotreduced,theeectiveresponsefunctionturnsouttobeusefulforexposingtheeectoftheglobalparametersonthenetworkproperties,asshowninanexampleattheendofthisintroduction.Forasinglepopulationinfocusonerecoversthefamiliar,one-dimensional,MFpictureinwhichattractorstatesofdelayactivityarecapturedbyintersec-tionsoftheeectiveresponsefunctionwithastraightline,thoseintersectionsatwhichtheslopeofthetransferfunctionissmallerthanthatoftheline.Fortwopopulationsinfocus,onecanstillexploita2-dimensionalowdiagramwhichidentiesattractorstationarystates.Suchsituationsapplyinthestudyoftheinteractionbetweentwocompetingselectivedelayactivitystatesinthesamemodule,orwhenselectiveactivityintwodierentmodulesinteract.Therstmaydescribethecasewhenanewstimuluscompeteswithanelicitedactivememory,asinselectiveattention[2].Thesecondmayberequiredfordescrib-ingtheinteractionbetweeninferotemporalandprefrontalcortex[1],orbetweeninferotemporalcortexandaretinotopiccorticalcomponentsuchasV4[2].Themethodisillustratedindetailforarecurrentnetworkofintegrate-and-re2(IF)neurons,abletosustainbothspontaneousandselectiveactivity.Itisthenshownhowinthesparsecodingcasetheeectiveresponsefunctiondecouplesthepopulationinfocusfromtheotherpopulations,leadingtoabona-de1-dimensionalsystem.Theresultingsystemhasthebareresponsefunctionwithrenormalizedlearningparameterandabiasfortheaerentcurrent.Nextweexhibitthemethodforacaseoftwopopulationsinfocusandconcludewithadiscussionoftheconnectionofthestabilityofthesolutionsofthereducedsystemtothatofthefullsystem.Thisissueisdiscussedinfulldetailforsparselycodedpopulations.1.0.1AnexampleToillustratetheusefulnessoftheeectiveresponsefunctionwewillshowtheresultsoftheanalysisofthesystemtreatedin[5].ThisanalysiswillbefurtherextendedinSec.3.Consideranetworkcomp