Distance-measure A SELF-ORGANIZING 2-D FEATURE MAP

NNets—L.10April17,200210Self-OrganizingFeatureMapsSelf-OrganizingFeatureMaps(SOFM)alsoknownasKohonenmapsortopographicmapswerefirstintroducedbyvonderMalsburg(1973)andinitspresentformbyKohonen(1982).AccordingtoKohonentheideaoffeaturemapformationcanbestatedasfollows:Thespatiallocationofanoutputneuroninthetopographicmapcorrespondstoaparticulardomain,orfeatureoftheinputdata.Morespecifically:Self-OrganizingFeaturemapsarecompetitiveneuralnetworksinwhichneuronsareorganizedinanl-dimensionallattice(grid)representingthefeaturespaceTheoutputlatticecharacterizesarelativepositionofneuronswithregardstoitsneighbours,thatistheirtopologicalpropertiesratherthanexactgeometriclocations.Inpractice,dimensionalityofthefeaturespaceisoftenrestrictedbyitsitsvisualisationaspectandtypicallyisl=1,2or3.A.P.Papli´nski10–1NNets—L.10April17,2002Consideranexampleofaself-organizingfeaturemapinwhichtheinputspaceis3-dimensional(p=3)andfeaturespaceis2-dimensional(l=2).ThestructureofsuchSOFMisillustratedinFigure10–1.2-DlatticeofneuronsMinNetDistance-measurex1xx23yy2,2yy1,11,22,1y4,4116ASELF-ORGANIZING2-DFEATUREMAPFigure10–1:A2-DSOFMwithp=3;m=[44];l=2.Eachneuron,yvintheaboveSOFMischaracterizedbyitspositioninthelatticespecifiedbya2-Dvectorv=[v1v2],andbya3-Dweightvectorwv=[w1vw2vw3v].SOFM,asacompetitiveneuralnetwork,consistsofadistance-measurelayerandacompetitivelayerwhichimplementstheMinNetalgorithmthroughthelateralinhibitiveandlocalself-excitatoryconnections.Duringthecompetitionphase(theMinNet),thewinnerisselectedfromallneuronsinthelattice.A.P.Papli´nski10–2NNets—L.10April17,2002AgeneralstructureofaSelf-OrganizingFeatureMapispresentedinFigure10–2Figure10–2:AgeneralstructureofaSOFMandcanbecharacterizedbythefollowingparameters:p—dimensionalityoftheinputspacel—dimensionalityofthefeaturespacem—thetotalnumberofneuronsW—m×pmatrixofsynapticweightsV—m×lmatrixoftopologicalpositionsofneuronsInsubsequentconsiderationsneuronswillbeidentifiedeitherbytheirindexk=1,...,m,orbytheirpositionvectorV(k,:)inthefeaturespace.10.1FeatureMapsAFeatureMapisaplotofsynapticweightsintheinputspaceinwhichweightsoftheneighbouringneuronsarejoinedbylinesorplanesegments(patches).A.P.Papli´nski10–3NNets—L.10April17,2002Example:2-Dinputspace,1-DfeaturespaceConsideraSOFMneuralnetworkwithtwoinputs(p=2)andmoutputsorganizedina1-Dfeaturespace:Figure10–3:Ageneralstructureofa(2-D,1-D)SOFMThefeaturemaphasthefollowingstructure:Figure10–4:Afeaturemapfora(2-D,1-D)SOFMNotethatinthefeaturemapthepointrepresentingtheweightvector,wj,isjoinedbylinesegmentswithpointsrepresentingweightswj−1andwj+1becauseneuronsj−1,j,andj+1arelocatedintheadjacentpositionsofthe1-Dlattice.A.P.Papli´nski10–4NNets—L.10April17,2002Example:2-Dinputspace,2-DfeaturespaceLetusconsideraSOFMwithtwoinputs(p=2)andmneuronsarrangedina2-DlatticeasinFigure10–5.Figure10–5:Ageneralstructureofa(2-D,2-D)SOFMThefeaturemapwhichmapsa2-Dinputspaceintoa2-Dfeaturespacehasthefollowingstructure:Figure10–6:Afeaturemapfora(2-D,2-D)SOFMConsideraneuronlocatedatthevertexe=(i,j)oftheoutputlatticeanditstwoneighbours,a=(i,j−1)andd=(i+1,j).Therelatedthreeweightsvectors,we=wi,j,wa=wi,j−1,andwd=wi+1,jarerepresentedbytherespectivepointsinthefeaturemapinFigure10–6.Thepointscanbejoinedbythreelinesegments,andformatriangularpatch.A.P.Papli´nski10–5NNets—L.10April17,2002Plotting(2-D,2-D)featuremaps:FM22.mscriptConsideraSOFMwithp=2inputsandm=12organizedona3×4lattice01234501234v2v1Figure10–7:Theoutputlatticeofa(2-D,2-D)SOFMThefollowingMATLABcodecanbeusedtogenerateanexampleoftheweightandpositionmatrices:m=[43];mm=prod(m);%p=2;[V1,V2]=meshgrid(1:m(1),1:m(2));V=[V2(:),V1(:)];W=V-1.4*rand(mm,2);TheresultingWandVmatricescanbeasfollows:kWV10.830.911120.722.012130.182.393142.370.061251.382.182261.412.823272.381.271382.061.772392.512.6133103.360.8514113.922.0524123.162.9034A.P.Papli´nski10–6NNets—L.10April17,2002Twoversionsofthefeaturemap,whichillustratestheweightvectorsintheinputspacetogetherwiththeneighbourhoodinformation,canbeplottedwiththefollowingMATLABcodeAcolourpatchesversionFM1=full(sparse(V(:,1),V(:,2),W(:,1)));FM2=full(sparse(V(:,1),V(:,2),W(:,2)));pcolor(FM1,FM2,32*(FM1+FM2));0.511.522.533.50.511.522.5A2−DFeatureMapusingpcolor(method1)wv11wv21wv31wv12wv22wv32wv13wv23wv33wv14wv24wv34x1x2Figure10–8:Amethod-1plotofafeaturemapfora(2-D,2-D)SOFMA.P.Papli´nski10–7NNets—L.10April17,2002AgridlinesversionFM=FM1+j*FM2;plot(FM),holdon,plot(FM.’),plot(FM,’o’),holdofftitle(’A2-DFeatureMapusinggridlines(method2)’)00.511.522.533.5400.511.522.53A2−DFeatureMapusinggridlines(method2)wv11wv21wv31wv12wv22wv32wv13wv23wv33wv14wv24wv34x1x2Figure10–9:Amethod-2plotofafeaturemapfora(2-D,2-D)SOFMA.P.Papli´nski10–8NNets—L.10April17,200210.2LearningAlgorithmforSelf-OrganizingFeatureMapsTheobjectiveofthelearningalgorithmfortheSOFMneuralnetworksisformationofthefeaturemapwhichcapturesoftheessentialcharacteristicsofthep-dimensionalinputdataandmapsthemonthetypically1-Dor2-Dfeaturespace.Thelearningalgorithmcapturestwoessentialaspectsofthemapformation,namely,competitionandcooperationbetweenneuronsoftheout