第六章 自组织映射网

第六章自组织映射网IntroductionSelf-organizingmap(SOM)LearningVectorQuantization(LVQ)NeuralGas(NG)GrowingNeuralGas(GNG)机器人智能与神经计算实验室(RINC.LAB)Manytimesthereisno“teacher”totellushowtodothingsAbabythatlearnshowtowalkGroupingofeventsintoameaningfulscene(makingsenseoftheworld)DevelopmentofoculardominanceandorientationselectivityinourvisualsystemSelfOrganizationNetworkOrganizationisfundamentaltothebrainFunctionalstructureLayeredstructureBothparallelprocessingandserialprocessingrequireorganizationofthebrainSelfOrganizingNetworksDiscoversignificantpatternsorfeaturesintheinputdataDiscoveryisdonewithoutateacherSynapticweightsarechangedaccordingtolocalrulesThechangesaffectaneuron’simmediateenvironmentuntilafinalconfigurationdevelopsQuestionHowcanausefulconfigurationdevelopfromselforganization?Canrandomactivityproducecoherentstructure?Answer:biologicallyThereareselforganizedstructuresinthebrainNeuronalnetworksgrowandevolvetobecomputationallyefficientbothinvitroandinvivoRandomactivationofthevisualsystemcanleadtolayeredandstructuredorganizationAnswer:PhysicallyLocalinteractionscanleadtoglobalordermagneticmaterialsElectriccircuitssynchronizationofcoupledoscillatorsMathematicallyA.Turing:GlobalordercanarisefromlocalinteractionsRandomlocalinteractionsbetweenneighboringneuronscancoalesceintostatesofglobalorder,andleadtocoherentspatiotemporalbehaviorMathematically,CntdNetworkorganizationtakesplaceat2levelsthatinteractwitheachother:Activity:certainactivitypatternsareproducedbyagivennetworkinresponsetoinputsignalsConnectivity:synapticweightsaremodifiedinresponsetoneuronalsignalsintheactivitypatternsSelfOrganizationisachievedifthereispositivefeedbackbetweenchangesinsynapticweightsandactivitypatternsPrinciplesofSelfOrganization1.Modificationsinsynapticweightstendtoselfamplify2.Limitationofresourcesleadtocompetitionamongsynapses3.Modificationsinsynapticweightstendtocooperate4.Orderandstructureinactivationpatternsrepresentredundantinformationthatistransformedintoknowledgebythenetwork-6-4-202468101214-4-3-2-10123RedundancyUnsupervisedlearningdependsonredundancyinthedataLearningisbasedonfindingpatternsandextractingfeaturesfromthedataTypesofInformationFamiliarity–thenetlearnshowsimilarisagivennewinputtothetypical(average)patternithasseenbeforeThenetfindsPrincipalComponentsinthedataClustering–thenetfindstheappropriatecategoriesbasedoncorrelationsinthedataEncoding–theoutputrepresentstheinput,usingasmalleramountofbitsFeatureMapping–thenetformsatopographicmapoftheinputPossibleApplicationsFamiliarityandPCAcanbeusedtoanalyzeunknowndataPCAisusedfordimensionreductionEncodingisusedforvectorquantizationClusteringisappliedonanytypesofdataFeaturemappingisimportantfordimensionreductionandforfunctionality(asinthebrain)SimpleModelsNetworkhasinputsandoutputsThereisnofeedbackfromtheenvironmentnosupervisionThenetworkupdatestheweightsfollowingsomelearningrule,andfindspatterns,featuresorcategorieswithintheinputspresentedtothenetworkFeatureMappingGeometricalarrangementofoutputunitsNearbyoutputscorrespondtonearbyinputpatternsFeatureMapTopologypreservingmap机器人智能与神经计算实验室(RINC.LAB)Determinethewinner(theneuronofwhichtheweightvectorhasthesmallestdistancetotheinputvector)Movetheweightvectorwofthewinningneurontowardstheinputi机器人智能与神经计算实验室(RINC.LAB)Imposeatopologicalorderontothecompetitiveneurons(e.g.,rectangularmap)Letneighborsofthewinnersharethe“prize”(The“postcodelottery”principle)Afterlearning,neuronswithsimilarweightstendtoclusteronthemap机器人智能与神经计算实验室(RINC.LAB)机器人智能与神经计算实验室(RINC.LAB)•“从很多角度来看，神经元功能的空间结构和自组织是很重要的，应该在所有的神经理论模型中反映这一事实。然而奇怪的是，传统的神经网络模型并没有注意这样一种空间结构，因此，自组织模型在神经网络理论中占据着重要的位置。”(Kohonen)•“把多种算法（FuzzyC-Means,FuzzyART,FuzzyARTforFuzzyClusters,FuzzyMax-Min,andtheKohonenneuralnetwork）应用在文档聚类的问题时。Kohonen‘sSOM算法聚类最好，而且形成了聚类之间的有序拓扑结构。”(Vicente,etc.)SOM模型评述Self-OrganizingMapsProjectionofpdimensionalobservationstoatwo(orone)dimensionalgridspaceConstraintversionofK-meansclusteringPrototypeslieinaone-ortwo-dimensionalmanifold(constrainedtopologicalmap;TeuvoKohonen,1993)Kprototypes:Rectangulargrid,hexagonalgridIntegerpairljQ1xQ2,whereQ1=1,…,q1&Q2=1,…,q2(K=q1xq2)High-dimensionalobservationsprojectedtothetwo-dimensionalcoordinatesystem机器人智能与神经计算实验室(RINC.LAB)机器人智能与神经计算实验室(RINC.LAB)’sself-organizingmapdefinesamappingfromtheinputdataspaceRnontoa2-darrayofnodes,itcanconvertcomplex,nonlinearstatisticalrelationshipsbetweenhigh-dimensionaldataitemsintosimplegeometricrelationshipsonalow-dimensionaldisplay.机器人智能与神经计算实验室(RINC.LAB)’sartificialneuralnetworkKohonenlayerInputlayerOutputlayer机器人智能与神经计算实验室(RINC.LAB)机器人智能与神经计算实验室(RINC.LAB)