Artificial Neural Networks Technology

ARTIFICIALNEURALNETWORKSTECHNOLOGYADACSState-of-the-ArtReportContractNumberF30602-89-C-0082(Data&AnalysisCenterforSoftware)ELIN:A011August201992Preparedfor:RomeLaboratoryRL/C3CGriffissAFB,NY13441-5700Preparedby:DaveAndersonandGeorgeMcNeillKamanSciencesCorporation258GenesseStreetUtica,NewYork13502-4627iTABLEOFCONTENTS1.0IntroductionandPurpose.............................................................................12.0WhatareArtificialNeuralNetworks?......................................................22.1AnalogytotheBrain.............................................................................22.2ArtificialNeuronsandHowTheyWork.........................................32.3ElectronicImplementationofArtificialNeurons..........................52.4ArtificialNetworkOperations............................................................72.5TraininganArtificialNeuralNetwork............................................102.5.1SupervisedTraining..................................................................102.5.2Unsupervised,orAdaptiveTraining....................................112.6HowNeuralNetworksDifferfromTraditionalComputingandExpertSystems...............................................................................123.0HistoryofNeuralNetworks.........................................................................174.0DetailedDescriptionofNeuralNetworkComponentsandHowTheyWork........................................................................................................204.1MajorComponentsofanArtificialNeuron....................................224.2TeachinganArtificialNeuralNetwork............................................264.2.1SupervisedLearning.................................................................264.2.2UnsupervisedLearning............................................................274.2.3LearningRates............................................................................284.2.4LearningLaws.............................................................................295.0NetworkSelection..........................................................................................315.1NetworksforPrediction.......................................................................325.1.1Feedforward,Back-Propagation..............................................325.1.2DeltaBarDelta............................................................................355.1.3ExtendedDeltaBarDelta..........................................................365.1.4DirectedRandomSearch..........................................................375.1.5Higher-orderNeuralNetworkorFunctional-linkNetwork...................................................................................................395.1.6Self-OrganizingMapintoBack-Propagation........................405.2NetworksforClassification..................................................................415.2.1LearningVectorQuantization................................................415.2.2Counter-propagationNetwork...............................................435.2.3ProbabilisticNeuralNetwork..................................................465.3NetworksforDataAssociation...........................................................485.3.1HopfieldNetwork......................................................................485.3.2BoltzmannMachine..................................................................505.3.3HammingNetwork...................................................................515.3.4Bi-directionalAssociativeMemory.......................................535.3.5Spatio-TemporalPatternRecognition(Avalanche)...........545.4NetworksforDataConceptualization...............................................555.4.1AdaptiveResonanceNetwork................................................565.4.2Self-OrganizingMap..................................................................56ii5.5NetworksforDataFiltering.................................................................585.5.1Recirculation...............................................................................586.0HowArtificialNeuralNetworksAreBeingUsed...................................616.1LanguageProcessing..............................................................................626.2CharacterRecognition...........................................................................626.3Image(data)Compression....................................................................636.4PatternRecognition...............................................................................636.5SignalProcessing....................................................................................646.6Financial...................................................................................................656.7ServoControl..........................................................................................656.8HowtoDetermineifanApplicationisaNeuralNetworkCandidate....................................