DE102005014323A1 - Field Programmable synapse Array and method for its programming - Google Patents

Abstract

The invention is a field programmable synapse array for the integration of a neural network and a method for programming the FPSA. DOLLAR A The object of the invention is to provide a modular analog neural integrated system which contains both very flexible and very complex blocks of neural networks for recognition and classification problems. One method is to combine several network topologies in an integrated arrangement. The arrangement should be able to be controlled and structured from the outside and still be an autonomous classification system after the configuration, which learns independently. DOLLAR A The FPSA is topologically characterized by an optimized arrangement of optimized connections (synapses) as "sea of synapses" on an integrated system with a modular structure, which realizes a generic network of dendrites and axons. DOLLAR A For teaching the arrangement, a linear function, a sigmoid function, a quadratic function, and a Gaussian function are used either singly or in combination of several.

Description

The Invention is a Field Programmable Synapse Array (FPSA) intelligent processor and a chip solution for the integration of a neural Network and a method for programming the FPSA.

artificial Neural networks are models of brain function. They try in Structure and functioning of replicating brain cell complexes and thereby a sustainable Simulation of complex thinking processes too achieve. They provide a computational model for information processing and are thus an alternative to the computer computer models. These are based largely on the von Neumann architecture. In the literature is the Type of neural information processing shown in great detail et al in [Rojas, R .; Neural Networks, Springer Verlag, Berlin and others. 2002], [Zell, A .; Simulation of neural networks; Oldenbourg-Verlag, Vienna, 1994].

learning ability and parallelism counting the processing to the most important features of neural networks. Fault tolerance and generalization are often reached, hang However, the choice of network model, the network size and the Coding processing information (input and output pattern) from.

It are different learning approaches for neural Networks: supervised, unsupervised and reinforcing.

When supervised Learning (supervised learning) is the error between calculated and desired Output minimized. In the supervised Learning phase is given to an input vector (e.g., symptoms, readings, or Images) assigned to an output vector. When using a neural Netzes as a classifier, the training is done exclusively with the patterns to be classified. Not learned patterns and glitches later Use classified as unknown.

unsupervised Learning (unsupervised learning) is learning without prior knowledge Target values. It can different things are learned. Popular are the automatic Classification (clustering) or the compression of data for Dimensionality reduction. It is the optimization of weights based on criteria that the network itself has.

Of the Learning algorithm of the empowering Reinforcement learning adapts weights based on Maximizing a reward or minimizing a punishment (punishment) expressed on the basis of the calculated output becomes.

One Problem with the use of neural networks is the quite complex investigation the optimal network structure and parameters, such as output functions neurons, threshold, minimum, maximum, learning rule, learning rate and Number of learning steps. That is why many tasks are complicated conventional Methods (e.g., statistical or rule-based methods), although the use of neural networks is simpler, faster and cheaper would.

Next the actual algorithmic task is still the requirement the implementation. If for a given task has a specific neural structure and topology found, it is in the requirements of an embedded implementation important to find a suitable target hardware. Here are the ones playing Hardware implementations of neural networks play a role.

at The investigation of existing neurohardware solutions will clearly show that the digital realizations the main part of the commercial products put. This is due to the fact that digital circuit technology easier to master than analogue. Second, the design tools for digital Circuitry much more comfortable and further automated as comparable tools for analog circuit development. Due to the longer development cycles in the analogue compared to digital technology, are the development costs correspondingly higher.

analog Circuitry is special for the implementation of certain Functions of the neural network much more effective than their digital Counterpart. These include Multiplication and activation functions. It is to be expected that in addition to the digital implementations, the analog approaches will also be powerful in the future neuronal networks.

Previous Implementations have shown that with the analog implementation variant of multilayer percepts with a hidden layer of information processing speeds in the reproduction phase from the entrance to the exit of the network of below 5 μs be achieved. The non-adaptive Analog chip from INTEL, the ETANN, requires 8 μs for a reproduction. The weight accuracy is 6 bits.

In the patent US 6513023 B1 is about the implementation of an analog neural network in the form of an MLP. The back propagation algorithm is implemented with on-chip learning. For the neuron, a special circuit is presented, with which the sigmoid function and its derivative can be mapped. The weights are charged analogously as charges in capacities and must be refreshed regularly. Learning takes place under external supervision.

disadvantage The invention is that the implementation is fixed. A scaling is only over a new design possible. The implementation is limited to a network with backpropagation. There is no other algorithm usable.

In In patent application WO 2003015027 A2, learning takes place in a purely analogue manner. Weight storage is analogous during the learning phase Capacitive in a capacitor and in the working phase locally digital in registers behind MDACs in order not to have to refresh. in addition comes the global weight storage in a memory block and monitoring, which are done digitally. It is necessary to have an external controller so this is not a pure on-chip learning / self-learning implementation. The Analog-to-digital conversion is performed by 16 implemented ADCs line by line on the chip. On the chip is the structure of the multilayer perceptron freely selectable. The BPA with learning is hard-wired. There can be no other algorithms to be used.

From the patent publication DE 693 30 432 T2 is a universal neural network computer and supercomputer known. It is about the implementation of a cellular neural network (CNN), a multi-dimensional grid arrangement of a plurality of identical cells. Each cell acts directly with a given number of cells of r neighboring layers. The r-neighborhood is the "receive radius field." The application determines whether the layers of neighboring cells are 2- or 3-dimensional The basic unit is the cell with linear and nonlinear circuit elements The connection of the cells is determined by a cloning mask, depending on the application The proposed solution is a CNN which is programmable without hardware components having to be swapped.

disadvantage this publication it is that it is a 2- or 3-dimensional lattice structure, in which the processors are arranged. It is not a structure with freely selectable Network topology. The processors only work in a finite neighborhood together. Thus, the arrangement is limited to a few algorithms.

In JP 2002358503 A In general, the control of scalable analog blocks by digital switches and digital connection circuits is described. The digital circuit blocks of each cell are the so-called "programmable devices." The analog circuit blocks can be configured by programming these "devices", ie, interconnected. This can be used to structure a network. In addition, there are circuits that each exist for configuring the respective block. In addition, there is a memory block each containing the information about the switching state of the switches and the configuration of the respective circuits.

adversely Here is that there is a very general wiring of analog and digital circuit blocks to implement signal processing functions. For learners Structures as they are for a neural network is needed be, is a big one Required effort. There are no pre-assigned activation functions that correspond accordingly network topology and neuron types used.

• – Der Anwendungsingenieur die Chips bis in das kleinste Detail genau kennen muss, damit er die Schaltkreise einsetzen kann.
• – Ein sehr großer schaltungstechnischer Aufwand betrieben werden muss, um diese Chips in einer Anwendung zu integrieren.
• – Entweder nur eine Netz-Topologie umgesetzt werden kann oder die Netztopologie von außen nicht programmierbar ist.
• – Ein sehr großer schaltungstechnischer Aufwand betrieben werden muss, um die nichtlinearen Aktivierungsfunktionen zu integrieren.

The disadvantages of the aforementioned solutions are that

• - The application engineer must know the chips to the smallest detail so that he can use the circuits.
• - A very large circuit complexity must be operated to integrate these chips in one application.
• - Either only one network topology can be implemented or the network topology is not programmable from the outside.
• - A very large circuitry effort must be made to the nonlinear activity integration functions.

OBJECT OF THE INVENTION

The The object of the invention is to provide a modular analog neural integrated system, which is both very flexible, as well as very complex blocks neural networks for Contains recognition and classification problems. The neural structure on the arrangement should be designed circuitically so that one possible size Number of network topologies can be implemented.

It is still the task by a method multiple network topologies to unite on an integrated arrangement, allowing the user by programming is capable of a neural network topology to choose. The arrangement should be from the outside can be controlled and structured and still after the Configuration be a self-sufficient classification system which independently learns.

The solution The object is achieved by the field programmable synapse array according to the invention and a Method of programming the FPSA as described in the claims becomes.

The Field Programmable Synapses Array is characterized that an optimized arrangement in optimized connections (Synapses) topologielos as "sea of synapses " an integrated system with a modular structure is realized, which realizes a generic network of dendrites and axons, whereby processors (neurons) are connected in such a way that they be layered and a variety of topologies neural networks depict.

For teaching The arrangement becomes a linear function, a sigmoid function, a quadratic function and a gaussian Function used either singly or in combination of several.

advantages

The Existing developments are with the invention presented here Not comparable. The main advantage of this solution is it set that after producing a standard chip release which implements concrete topology (= application) shall be.

The Neural network on the integrated array is circuit technology designed to be as possible size Number of network topologies can be implemented.

To The synapses are designed so that they are arranged in a field are and externally about programming if possible can be connected to any neuron.

The Arrangement has, in contrast to other conventional neural hardware solutions, a modular structure. With that you can both the number of needed Synapses as well as the shape of the neuron activation functions varies become. Dependent on from the learning process and the selected network topology as well as the given patterns allow tolerance and selectivity directly work out. These properties cause a variety of embedded solutions for complex Classification tasks at all only possible becomes.

So meets the requirement for both highly flexible, and always more complex components for Recognition and classification problems are sufficient.

The simultaneous integration of a standard digital interface elevated the flexibility and the handling of the Anorgnung. In connection with the rising Acceptance of neural solutions for technical Open classification problems the planned product wide application possibilities. Even after production of a chip release can Network topologies with modern methods (e.g., evolutionary, genetic and statistical algorithms) and in the previously produced batch can be implemented. While previously errors in individual blocks of the neural network to be implemented for exploiting the whole Chip production to lead, the success of the chip production is no longer dependent on the quality of the implemented network because defective synapses or neurons are switched off can be.

• • durch das modulare Systemkonzept Modulentwickler parallel arbeiten können und der Abstimmungsaufwand reduziert wird.
• • die Entwicklung der Netztopologien parallel und unabhängig von der Chip-Entwicklung erfolgen kann.
• • die Anwendungsentwicklung nicht auf die Fertigstellung bestimmter Chips (ASICs) angewiesen ist, sondern auf standardisierte Produkt-Releases zurückgreifen kann.

With these characteristics, the development work (of dependencies) can be unbundled and paralleled be siert, there

• • be able to work in parallel with the modular system concept module developers and reduce the coordination effort.
• • the development of network topologies can be parallel and independent of the chip development.
• • application development does not rely on the completion of certain chips (ASICs), but can rely on standardized product releases.

With a variable number of processing units has the arrangement a structure that the user can choose himself.

With a processing speed in the nanosecond range requires the Circuit only a fraction of the time, which is the fastest so far Products for need the reproduction.

The modularity the network topology leads for easier handling by the application engineer. Thereby the inhibition threshold for switching to a neuronal solution is lowered.

It are novel applications in the field of sensor array evaluation in the automotive industry, in the printing industry and in the air- and space travel conceivable.

The The invention will be explained in more detail by means of examples. The figures show:

1 Synapse block with register

2 neuron functions

3 Hiddenneuroenenblock

4 Input-output neurons Hidden Block

5 Perceptron type

6 Nearest neighbor type

7 Example of network setup

8th Single-layer perceptron network and synapses with linear or nonlinear activation function, network consisting of 4 inputs, 16 synapses and 4 outputs

9 Perceptive mesh with a hidden layer, network of 4 inputs, 16 hidden synapses, 4 hidden neurons, 16 output synapses and 4 output neurons

10 Perceptual mesh with two hidden layers, network of 4 inputs, 16 hidden synapses and 4 hidden neurons in the first hidden layer, 16 hidden synapses and 4 hidden neurons in the second hidden layer, 16 output synapses and 4 output neurons

11 Tasterwire network with three hidden layers, network of 4 inputs, 16 hidden synapses and 4 hidden neurons in the first hidden layer, 16 hidden synapses and 4 hidden neurons in the second hidden layer, 16 hidden synapses and 4 hidden neurons in the third hidden layer , 16 output synapses and 4 output neurons

12 2 parallel perceptron networks with 2 inputs each, one hidden layer each consisting of 4 synapses and 2 neurons and one output layer each consisting of 4 synapses and 2 outputs

13 A first perceptron network with 2 inputs, a hidden layer consisting of 4 synapses and 2 neurons and an output layer consisting of 4 synapses and 2 outputs and a second perceptron network with 2 inputs, a first hidden layer consisting of 4 synapses and 2 neurons, a second hidden one Layer consisting of 4 synapses and 2 neurons and one output Layer consisting of 4 synapses and 2 outputs

14 Network for easy classification with 3 inputs and 4 outputs

15 RBF network with 3 inputs, a hidden layer consisting of 6 synapses and 2 neurons and an output layer consisting of 4 synapses and 2 neurons

16 Winner take all network with 3 inputs, 3 outputs and 15 synapses. If all synapse connections are not trained, this network can also be considered a Kohonen feature map.

17 Hopfield network consisting of 3 inputs, 3 outputs and 6 synapses

• – System
• – Funktionsblöcke
• – Netz
• – Topologie (bzw. Parallelprozessor-Anordnung)
• – Segmente
• – Neuron (bzw. Prozessor) • Dendrit • Axon
• – Synapse

Prerequisite for the implementation is a strictly object-oriented (modular and hierarchical) system concept with clearly defined interfaces between the hierarchy and module levels and the instances of a module level, eg

• - System
• - Function blocks
• - Network
• - topology (or parallel processor arrangement)
• - Segments
• - neuron (or processor) • dendrite • axon
• - Synapse

This System concept has the advantage that the different module levels independently be optimized and standardized from each other.

Under This requirement can be optimized on-chip in optimized connections (synapses), initially topologyless as "sea of synapses "realized become.

With This arrangement becomes a generic network of dendrites and axons realized which processors (neurons) interconnected in such a way that they can be layered and map a variety of neural network topologies.

The Arrangement takes place so that after the production of the chip a concrete application-specific topology can be "burned" into the chip.

In Analogy to FPGAs and FPAAs leaves this arrangement is called FPSA (Field Programmable Synapses Array).

The on this single technology based FPSA circuits can be used in different scaling levels for different Application areas produce and so a diversified portfolio of Produce standard products tailored for each application can be.

For teaching The chip is a process developed with a self-adjusting of the neural network possible is. The reproduction path can be fully integrated. The weights are learned externally on a computer in this variant and then completely charged.

Of the modular chip is a neural classifier that learns independently. The learning becomes supervised However, learning supervised by a coach. When unsupervised Learning will be i.a. Similarities wanted and corresponding clusters.

The Allow dimensions of this unit For the first time the use of different topologies of neural networks in a sensor-related system for mobile and autonomous applications and devices. This easier and faster Realization of a neural network based on such Chips should replace previous implementation variants and artificial neural networks, etc. For Decision tasks in connection with sensors and transducers considerably simplify.

• – allgemeine Strukturvariabilität, d. h. variabler Algorithmus und skalierbare Netzgröße (Grenze: Anzahl der Synapsen)
• – variable Anzahl von versteckten Schichten möglich (max. 4)
• – mehrere Netze parallel und unabhängig voneinander implementierbar

Properties of the network:

• General structure variability, ie variable algorithm and scalable network size (limit: number of synapses)
• - variable number of hidden layers possible (max 4)
• - Multiple networks can be implemented in parallel and independently of each other

The signal transmission between neurons, synapses, inputs and outputs is done by differential Streams.

synapses

All synapses in the synapse array have the same structure and implement the same functions ( 1 ). These are, on the one hand, the storage of a weight value in the form of a digitized voltage by storage in digital registers or RAM cells.

On the other hand, in the synapse the multiplication of the stored weight value is carried out with the input value applied to the synapse (w _i * x _i ).

Becomes The weight value stored digitally must have a DAC in the synapse be implemented. The conversion of the analog weight values in digital bit pattern is done through a network of switched:

• 1. capacities,
• 2. resistances or
• 3. Power sources.

The Synapses are addressable, so weight values are written into it or can be read out.

The functional special form Biassynapse is built exactly like the synapse and can also be used as such.

• 1. variable Programmierung von Transistorgates oder
• 2. feste Programmierung durch Fuses

auf diese Leitungen beliebig aufgeschaltet werden. Dabei dürfen die zwei differenziellen Ein- und Ausgangsleitungen einer Synapse jedoch nur jeweils auf eine Leitung verdrahtet werden. Dabei können jedoch mehrere differentielle Ein- bzw. Ausgangsleitungen von unterschiedlichen Synapsen auf den gleichen Leitungspaaren verdrahtet sein.The synapses are arranged in blocks of four each. Between the upper and lower as well as between the right and left two four four-wire cables are arranged. The inputs and outputs of the synapses can be:

• 1. variable programming of transistor gates or
• 2. Fixed programming by fuses

be switched on any of these lines. However, the two differential input and output lines of a synapse may only be wired to one line at a time. However, several differential input and output lines of different synapses on the same cable pairs can be wired.

Around programming is possible in every synapse block a register provided in which stores the switch position coded becomes.

neurons

in the Net, there are different neuron forms. Generally they have however, the task of calculating a function value from the To calculate the sum of the signals present at its input.

• 1. die lineare Funktion,
• 2. die sigmoide Funktion,
• 3. die quadratische Funktion und
• 4. die gaußsche Funktion.

There are four functions ( 2 ) for the neurons. These are:

• 1. the linear function,
• 2. the sigmoid function,
• 3. the quadratic function and
• 4. the gaussian function.

The Sigmoid function is the default activation feature in addition to the linear function. Both are considered topologically in the hidden layers and the Starting layer available posed. The sigmoid function becomes especially with the Multilayerperzeptron used.

The quadratic function can only be used in the hidden layers while the gauss The function curves can each be set via various parameters (eg sigmoid function - slope can be changed via working current).

From The four functions can either select one or a combination of several be made.

The hidden neuron functions can also assume a synaptic function task in special cases. If, as in the nearest neighbor web, through a synapse, the function for calculating the Euclidean distance (w _i - x _i), for example, be carried out ^2, this may here by the sum of (w _i ² - 2 · w _i · x _i + x ² ) are implemented. The value w _i ^{2 is} generated in a biassynapse, the value -2 · w _i - x _i in a generally usable synapse and the value x _i ² in a hidden neuron and the outputs on a node are added to (wi - xi) ² ,

The neuron functions are combined into blocks. The wiring is done as with the synapses by programming a register. For the hidden neuron blocks ( 3 ) one block each is implemented with the linear, one with the sigmoid function and one with the quadratic function together with a synapse. The synapse can be used for example as a bias or feedback synapse, which is required in fed-back networks. These neuron blocks are placed on the left, right and top edges of the mesh.

At the bottom are blocks with three inputs and two neurons each. These output-hidden neuron blocks ( 4 ) can optionally be treated as neurons of the hidden or output layer. One neuron contains the linear and the sigmoid function and the other neuron the linear and Gaussian functions. These two neurons do not contain connections to bias or feedback synapses. But a switchable driver is implemented in them, which makes it possible to control external loads without the need for additional circuitry between the chip and connected actuators. At the same time the influence of additional sources of error on the output signals of the arrangement is avoided because the signal flow between the sensor and the actuator is kept extremely short. There is no analog / digital or digital / analog conversion. In addition, no further drivers and converters are needed in the optimal case.

In 5 the perceptron type is displayed. In the perceptron type, the input signals are weighted (multiplied by a weight value). This is followed by a summation of the weighted input signals. This corresponds mathematically to the scalar product. After scalar formation, the sum is transferred by a special function (linear, sigmoid, jump-shaped).

With This type can be all possible Realize mesh topologies, which are then e.g. to the so-called Become a multilayer perceptron.

6 shows the nearest neighbor type. In the nearest neighbor type, the difference between the input signal and the weight is formed and then squared, and the sum of the squares is then formed.

attracts if you add the square root of the sum, this corresponds to the calculation the distance (Euclidean distance) between adjacent signal and stored weight vector (hence the name "nearest neighbor"). This is for the function of the network but not crucial. But it would have the advantage that this value can be interpreted geometrically with 1 to 3-dimensional vectors is, as in the 3-dimensional color space.

• – Radial Basisfunktionen Netze (RBF)
• – Generalized Regression Networks (GRNN)
• – Probabilistic Neural Networks (PNN)
• – Self-Organizing Maps (SOM) (auch Kohonen Feature Maps)
• – Learning Vector Quantization Networks (LVQ)

Es muss betont werden, dass sich hinter den verschiedenen Netzvarianten nicht immer eine andere Architektur versteckt, sondern dass vielmehr eine Trainingsmethode oder Nutzungsweise für die Bezeichnung verantwortlich ist.With the "Nearest Neighbor type" (through some additions) opens up a whole series of network variants, which are known as:

• - Radial Basis Functions Networks (RBF)
• - Generalized Regression Networks (GRNN)
• - Probabilistic Neural Networks (PNN)
• - Self-Organizing Maps (SOM) (also Kohonen Feature Maps)
• - Learning Vector Quantization Networks (LVQ)

It must be emphasized that behind the different network variants does not always hide a different architecture, but that rather a training method or usage for the designation responsible is.

Perceptronnets - one layer and linear transfer function

This is the simplest case of a perceptronnetzes. The interesting thing This is because the weights can be calculated analytically. It Exactly one global minimum exists. You can use this network form e.g. For linear color space transformations such as for sensor correction or for conversions between different color monitor types.

Perceptronnets - one layer and non-linear transfer function

The Weights can be calculated analytically. With a simple one iterative minimization techniques provide a global minimum.

Perceptronnets - multiple layers

The Determination of weights for the to be solved Task can only be done with a statistical iterative minimization method done (learning). There are generally a variety of local ones Minima. Therefore, the success of weight determination is unpredictable but by chance. It is also not possible indicate the appropriate topology of networking for a particular task, there are no analytical connections exist between task type and topology properties. Of the Minimization process may require very many iterations and therefore (depending on the computing power) require a lot of time.

in principle can with multi-layered perceptron networks classification and function approximation tasks solved become. It should be noted, however, that a real classification only possible with an additional layer becomes, from the fact from the output vector of the network by e.g. a maximum determination the class emerges.

The quality a function approximation is difficult to verify because not the whole (multidimensional) Signal range can be driven through. It is therefore possible that the reaction of the network to a specific input signal unpredictable of the desired Behavior deviates.

Nearest Neighbor type - easier classifier

At By way of example, only two simple representatives will be listed here. where the essential characteristics become clear.

This Network has only one layer and no special transfer function. It can directly the sum at the output can be evaluated. In the weights The so-called prototypes of a class are stored. By the calculation of the network creates a vector at the output of the network, the (square) distances represents the input signal to the stored class prototypes. By determining the minimum output signal is the appropriate one Class found. Ideally, the minimum is zero because input signal and prototype are identical.

This Net type is very good for Classification tasks that specify class prototypes can and where no further re-interpretation (mapping) of the output signals necessary is. A well-known example is the classification of colors.

Advantageous is that through the topology (number of inputs and outputs) exactly It is clear what the chip does and what does not. So it can be be told in advance whether a task is solvable or not.

Nearest neighbor type RBF network

The RBF network has two layers. The first layer is with the entrance fully wired, as with the simple multilayer perceptron. Indeed the weighted sums are calculated in a special function, which has a bell shape. This provides the layer for test patterns the outside The training patterns are only minor activities, as only areas within The respective bells are leading to activities of the neurons. The second layer of the RBF network is of the perceptron type with linear Output function. Therefore, the weights of this layer can be calculate analytically, so that no statistical iterative minimization process is required.

The second layer can realize different tasks (everything a single-layered network can). One function is the "mapping" First layer output vector a particular vector at the output associated with the second layer by e.g. Define classes.

Farther can be made possible by the second layer that the RBF network for general function approximations can be used.

Advantageous Here is the property that inherently no unpredictable Reactions of the network can arise on unknown input signals, since the network is always interpolated only between the stored nodes.

The hidden neuron functions can also assume a synaptic function task in special cases. If, as in the nearest neighbor web, through a synapse, the function for calculating the Euclidean distance (w _i - x _i), for example, be carried out ^2, this may be incorporated by the sum of (w _i ² - 2 · w _i · x _i + x ² ) are implemented. In this case, the value w _i ^{2 is} generated in a biassynapse, the value -2 · w _i · x _i in a generally usable synapse, and the value x _i ² in a hidden neuron, and the outputs on a node are added to (wi - xi) ² ,

The synapse-neuron-array is strictly modular in order to create an almost arbitrary shape and size of a neural network. A proposal to build the FPSA shows 7 , The structure is designed in such a way that as many conceivable as possible realistic network topologies can be implemented in hardware.

The Neurons are in blocks summarized. The wiring is done as synapses through Programming a register. For the hidden neuron blocks are one block each with the linear, one with the sigmoiden and one one implemented with the quadratic function along with a synapse. The Synapse can be e.g. used as a bias or feedback synapse what is fed back at Nets is required. These neuron blocks are on the left, right and the top of the net. At the bottom are blocks with four neurons that perform the sigmoid function, arranged. One can this e.g. each split into three input and one output neuron.

In Fig. 7 designates

in the Example is a network with 64 synapses, 12 biassynaps, 16 hidden neurons, 12 input neurons, 4 output neurons and 3 bias neurons.

One A network that could be implemented in practice could be 1088 (1024 synapses + 64 biassynaps) synapses and 131 (48 input neurons + 64 hidden neurons + 16 Output neurons + 3 bias neurons) neurons.

Perceptron Netre

at Perceptron networks are the input signals with a weight value multiplied. Subsequently there is a summation of the weighted input signals. This matches with mathematically the scalar product. After the scalar formation, the transmission takes place the sum by a special function (linear, sigmoid, jump).

With This type can be all possible Realize mesh topologies, which are then e.g. to the so-called Become a multilayer perceptron.

in the Below are examples of simple representatives of perceptron networks listed become.

Perceptron network with one Weight layer and linear transfer function

This is the simplest case of a perceptronnetzes. The interesting thing This is because the weights can be calculated analytically. It Exactly one global minimum exists. This network form can be used e.g. For linear color space transformations such as for sensor correction or for conversions between different color monitor types.

A practical perceptron network with a weight layer and linear transfer function for the cited FPSA example is in 8th shown. The network has 4 inputs (INA1, INA2, INA3, INA4), a weight layer consisting of 16 synapses (SA1 ... 4,1 ... 4) and 4 outputs (OHNA1, OHNA2, OHNA3, OHNA4). The network can be operated with or without bias neurons and synapses. The bias neurons can also be used as additional input neurons. In the neuron blocks HNR1, HNR2, HNR3 and HNR4, the linear activation is performed. The synapses SD1,1, SD2,2, SD3,3 and SD4,4 are for signal redirection only. There is no weighting in them.

Single-layer perceptron network with non-linear transfer function

The Weights can be calculated analytically. With a simple one iterative minimization techniques provide a global minimum.

A practical perceptron network with an output layer and non-linear transfer function for the cited FPSA example is shown in FIG 8th shown. The network has 4 inputs (INA1, INA2, INA3, INA4), a weight layer consisting of 16 synapses (SA1 ... 4,1 ... 4) and 4 outputs (OHNA1, OHNA2, OHNA3, OHNA4). The network can be operated with or without bias neurons and synapses. The bias neurons can also be used as additional input neurons. In the neuron blocks HNR1, HNR2, HNR3 and HNR4, nonlinear activation is performed. The synapses SD1,1, SD2,2, SD3,3 and SD4,4 are for signal redirection only. There is no weighting in them.

Perceptron networks with several Hidden layers and linear or nonlinear transfer function

The Determination of weights for the to be solved Task can only be done with a statistical iterative minimization method done (learning). There are generally a variety of local ones Minima. Therefore, the success of weight determination is unpredictable but by chance. It is also not possible indicate the appropriate topology of networking for a particular task, there are no analytical connections exist between task type and topology properties. Of the Minimization process may require very many iterations and therefore (depending on the computing power) require a lot of time.

in principle can with multi-layered perceptron networks classification and function approximation tasks solved become.

One Perceptron network for the cited FPSA example with one, two, or even three hidden layers and with also in the individual layers optionally usable linear or non-linear transfer function is thus feasible. The networks can, depending on the requirement the topology, with or without bias neurons and synapses. The bias neurons can also as additional Input neurons are used.

A network with 4 inputs (INA1, INA2, INA3, INA4), a hidden layer consisting of 16 synapses (SA1 ... 4,1 ... 4) and 4 neurons (HNR1, HNR2, HNR3, HNR4) and an output layer consisting of 16 synapses (SB1 ... 4,1 ... 4) and 4 outlets (OHNA1, OHNA2, OHNA3, OHNA4) is in 9 shown.

4 inputs (INA1, INA2, INA3, INA4), a first hidden layer consisting of 16 synapses (SA1 ... 4,1 ... 4) and 4 neurons (HNR1, HNR2, HNR3, HNR4), a second hidden layer consisting of 16 synapses (SB1 ... 4,1 ... 4) and 4 neurons (HNO1, HNO2, HNO3, HNO4) and an output layer consisting of 16 synapses (SC1 ... 4,1 ... 4) and 4 outputs (OHNA1, OHNA2, OHNA3, OHNA4) has the network, which 10 shows. The activation is carried out in the neuron blocks HNL1, HNL2, HNL3 and HNL4. The synapses SD1,1, SD2,2, SD3,3 and SD4,4 are for signal redirection only. There is no weighting in them.

An example of a 4-input network (INA1, INA2, INA3, INA4), a first hidden layer consisting of 16 synapses (SA1 ... 4,1 ... 4) and 4 neurons (HNR1, HNR2, HNR3, HNR4 ), a second hidden layer consisting of 16 synapses (SB1 ... 4,1 ... 4) and 4 neurons (HNO1, HNO2, HNO3, HNO4), a third hidden layer consisting of 16 synapses (SC1 ... 4 , 1 ... 4) and 4 neurons (HNL1, HNL2, HNL3, HNL4) and an output layer consisting of 16 synapses (SD1 ... 4,1 ... 4) and 4 outputs (OHNA1, OHNA2, OHNA3, OHNA4 ) gives 11 again.

The three examples already show how big the possibilities of variation alone for Are perceptron networks that are possible with the cited FPSA example.

Parallel arrangement of multiple perceptron networks with one or more weight layers and linear or non-linear transfer function

Around several of each other completely independent To process sensor signals, but do not operate the same hardware effort several times to have to, With the FPSA, it is possible to parallel, completely independent networks to implement. It can the topologies used may also be completely different.

12 shows an implementation example for the cited FPSA example, if you have two topologically identical networks with 2 inputs (INA1, INA2 or INA3, INA4), each of a hidden layer consisting of 4 synapses (SA1..2,1..2 resp SA3 ... 4,3 ... 4) and 2 neurons (HNR1, HNR2 or HNR3, HNR4) per one output layer consisting of 4 synapses (SB1 ... 2,1 ... 2 or SB3 .. .4,3 ... 4) and 2 outputs (OHNA1, OHNA2 or OHNA3, OHNA4) would like to operate. The networks can be operated with or without bias neurons and synapses, or linear or nonlinear activation function. The bias neurons can also be used as additional input neurons.

If you want to operate two networks with different topology in the cited FPSA example, shows 13 a possible implementation variant. There is a first network with 2 inputs (INA1, INA2), a hidden layer consisting of 4 synapses (SA1 ... 2,1 ... 2) and 2 neurons (HNR1, HNR2) of an output layer consisting of 4 synapses (SB1 ... 2,1 ... 2) and 2 outputs (OHNA1, OHNA2) and a second network with 2 inputs (INA3, INA4), a first hidden layer consisting of 4 synapses (SA3 ... 4,3 .. .4) and 2 neurons (HNR3, HNR4), a second hidden layer consisting of 4 synapses (SB3 ... 4,3 ... 4) and 2 neurons (HNO3, HNO4) and an output layer consisting of 4 synapses (SC3 ... 4,3 ... 4) and 2 outputs (OHNA3, OHNA4). In the second network, the activation is performed in the neuron blocks HNL3 and HNL4. The synapses SD3,3 and SD4,4 are only for signal redirection. There is no weighting in them. Both networks can be operated either with or without bias neurons and synapses or linear or non-linear activation function. The bias neurons can also be used as additional input neurons.

Nearest neighbor type

At the "Nearest Neighbor type "will the difference between input signal and weight is formed and this subsequently squared. The sum of the squares is then formed.

attracts if you add the square root of the sum, this corresponds to the calculation the distance (Euclidean distance) between adjacent signal and stored weight vector (hence the name "Nearest Neighbor "). This is for the function of the network but not crucial. It would have the advantage of having this value in 1 to 3-dimensional vectors is geometrically interpretable, such as in 3-dimensional color space.

With Networks from the "Nearest Neighbor "open a whole series of network variants.

At By way of example, only two simple representatives of Next neighbor type networks listed where the essential characteristics become clear.

Easier classifier

This Network has only one layer and no special transfer function. It can evaluated directly the sum of the Euclidean distances at the exit become. In the weights, the so-called prototypes of a Class stored. Due to the calculation of the network arises on Output of the network a vector representing the square intervals of the Represents input to the stored class prototypes. By determining the minimum output signal is the appropriate one Class found. Ideally, the minimum is zero because input signal and prototype are identical.

This Net type is very good for Classification tasks that specify class prototypes can and where no further re-interpretation (mapping) of the output signals necessary is. A well-known example is the classification of colors.

A practical network for the cited FPSA example for simple classification is in 14 shown. The classifier has 3 inputs (INA1, INA2, INA3) and 4 outputs (OHNA1, OHNA2, OHNA3, OHNA4). In the simple classification, first the difference between the input signals and the weights is formed. This happens in the hidden neuron blocks (HNR1 ... 4, HNL1 ... 4, HNO1 ... 4). The bias value (BNR, BNL, BNO) is set to -1. This results in the weight value (wi) in the synapses (SR1 ... 4, SL1 ... 4, SO1 ... 4), which are stored in the hidden neuron blocks (HNR1 ... 4, HNL1 ... 4, HNO1 .. .4) are multiplied by -1 (-wi). The input signal (xi) is added to this value (xi - wi). Then this sum is squared ((xi - wi) 2). The sum of the squares is then formed at the downstream input hidden output neuron blocks (OHNA1, OHNA2, OHNA3, OHNA4) (sum ((xi - wi) 2)). The synapses SB1,1, SB2,2, SB3,3, SB4,4, SC1,1, SC2,2, SC3,3, SC4,4, SD1,1 SD2,2 SD3,3 and SD4,4 are only for signal routing. There is no weighting in them.

RBF network

The RBF network has two layers. The first layer is a hidden layer like the simple multilayer perceptron. However, the weighted ones Calculates sums in a special function that takes the form of a Bell curve has. Thus, the layer for test patterns provides the outside The training patterns are only minor activities, as only areas within The respective bells are leading to activities of the neurons. The second layer of the RBF network is of the perceptron type with linear Output function. Therefore, the weights of this stage can be analytically so that no statistical iterative minimization process is required.

The second layer can realize different tasks. A function is the "folder" First layer output vector a particular vector at the output associated with the second layer by e.g. Define classes. Farther can be made possible by the second layer that the RBF network for use general function approximations.

Advantageous Here is the property that inherently no unpredictable Reactions of the network can arise on unknown input signals, since the network is always interpolated only between the stored nodes.

An implementation example for the cited FPSA could be a 3-input RBF network (INA1, INA2, INA3), a hidden layer consisting of 6 synapses (SB1,1, SB2,2, SC1,1, SC2,2, SD1,1, SD2,2) and 2 neurons (OHNB1, OHNB2) and an initial layer consist of 4 synapses (SB1,4, SB2,4, SC1,4, SC2,4) and 2 neurons (OHNA1 OHNA2) 15 The network works like a simple classifier with downstream output layer. The two hidden neurons (OHNB1, OHNB2) work with the Gaussian function as activation. The bias neurons (BNL, BNO, BNR) are used to reference the weight values stored in the synapses SL1, SL2, SO1, SO2, SR1 and SR2. The synapses SA1,1, SA1,2, SA2,1, SA2,2, SA3,1, SA3,2, SA1,4 and SA2,4 as well as the neuron blocks HNL4 and HNR4 are for signal steering. In these blocks, no weighting or activation is made.

Feedback networks

So far became exclusive Network topologies without feedback considered. Propagation processing is done from the input layer to the output layer. When fed back Networks, on the other hand, have to the conditions the neurons are recalculated until the net into one Hibernation is converged in which no change in the activation states more results. A stable state is therefore due to the input, the weight matrix and determines the thresholds of the neurons.

in the Below are three typical representatives of feedback networks briefly presented become.

Winner Take All (WTA)

At the Competitive learning (learning through competition, competition learning) The units of the network compete with each other for the "right" to enter to produce an output. Only one is allowed to deliver an issue and At the same time, stop everyone else from doing it.

At the unsupervised Learning the n-dimensional inputs with just as many arithmetic units processes how clusters are to be determined. For three clusters a network of three competitive units is needed.

A practical to implement WTA network for the cited FPSA, which 16 shows, has 3 inputs (INA1, INA2, INA3) and 3 outputs (OHNA1, OHNA2, OHNA3). The network is single-layered and fully networked with 9 synapses (SA1,1 ... 3, SA2,1 ... 3, SA3,1 ... 3). The outputs of each neuron (HNR1, HNR2, HNR3) are additionally switched to the inputs of all other neurons with 6 synapses (SB1,2, SB1,3, SB2,1, SB2,3, SB3,1, SB3,2). The synapse blocks SD1,4, SD2,4 and SD3,4 are not used as synapses, but are merely for switching the outputs of the neurons to the FPSA outputs.

Kohonen feature Maps

Kohonen Feature maps work similarly like WTA networks. The difference, however, is that the output is not limited to one unit and the number of clusters is not known before learning. The So clustering will be during of learning.

An example of how to implement a Kohonen network with 3 inputs and 3 outputs in the cited FPSA example is in 16 shown. The network is exactly the same as in the WTA example. However, the illustrated feedback connections through the synapses SB1,2, SB1,3, SB2,1, SB2,3, SB3,1 and SB3,2 may not be formed during the teaching.

Hopfield nets

• – Das Netz besteht aus einer einzigen Schicht von n Neuronen.
• – Die n Neuronen sind untereinander total vernetzt, d.h. jedes Neuron besitzt eine Verbindung zu jedem anderen Neuron (Rückkopplung, Rekursion).
• – Kein Neuron ist direkt mit sich selbst verbunden (keine unmittelbare Rückkopplung).
• – Das Netz ist symmetrisch gewichtet, d.h. das Gewicht der Bindung zwischen Neuron i und Neuron j ist gleich dem Gewicht der Verbindung zwischen Neuron j und Neuron i.
• – Den einzelnen Neuronen ist jeweils eine lineare Schwellenwertfunktion als Aktivierungsfunktion zugeordnet.
• – Eingabe ist die übliche gewichtete Summe.

Hopfield networks are based on the following assumptions:

• - The network consists of a single layer of n neurons.
• - The n neurons are totally interconnected, ie each neuron has a connection to every other neuron (feedback, recursion).
• - No neuron is directly connected to itself (no immediate feedback).
• The network is symmetrically weighted, ie the weight of the bond between neuron i and neuron j is equal to the weight of the connection between neuron j and neuron i.
• - Each neuron is assigned a linear threshold function as an activation function.
• - Input is the usual weighted sum.

Hopfield networks are therefore single-layer neural networks that are exclusively indirect feedback between each two different nodes i, j (i ≠ j) of the network, but no direct feedback to the same node. All connections between two neurons are symmetric, i.e. wij = wij. This can also be interpreted as being between two neurons only one bidirectional line exists.

The Hopfield network converges - under certain conditions - after a finite time to a resting state. If part of the output lines is attributed to the entrance, the corresponding part of the output pattern y can be the input pattern contribute x. y = x, i. Input of the neurons = output of the neurons (Autoassociation).

provides one fed back this one System in which such pattern pairs are stored, an incomplete input pattern At first, this results in a correspondingly incomplete output pattern. Whose right areas can suffice to return to partially to supplement the input the missing shares. The improved output patterns are again presented to the network as input, so the system will respond with a further improved output becomes. Each output of a neuron acts through the feedback on the entrances all neurons back.

The Hopfield network in 17 with 3 inputs (INA1, INA2, INA3) and 3 outputs (OHNA1, OHNA2, OHNA3) represents another implementation possibility of a topology in FPSA. The network is single-layered and has 6 synapses (SB1,2, SB1,3, SB2,1 , SB2,3, SB3,1, SB3,2) are completely fed back. Each neuron (HNR1, HNR2, HNR3) has a connection to every other neuron. No neuron, however, is directly connected to itself. The synapse blocks SA1,1, SA2,2, SA3,3, SB1,1, SB2,2 and SB3,3 are not used as synapses, but serve only to switch the inputs or outputs of the neurons to the FPSA input. or - outputs.

application

With the integration of multiple neural network topologies, other learning algorithms can be used. The combination of modularity, speed and a large number of processing units makes that Chip is a novelty that enables many applications in sensor processing. The applications for the invention are primarily in the field of sensor technology (sensor signal processing, adaptation), in industrial automation, as well as other areas in which very fast control functions are involved. This mainly affects systems with changing environmental conditions. Here, a learning ability for event states is required (teach-in functionality). The training takes place via a host computer with a special training software.

On Reason of the technical parameters of the product are, as many inputs to disposal stand, parallel evaluations of multi-sensor arrays in one component allows.

Of the Sensor market is one of the strongest growing segments within the entire microelectronics market. Based on standardized interfaces, it becomes possible sensors without problems to link with networks.

• – zunehmende Miniaturisierung
• – wachsende Chip-Integration
• – intelligente Verfahren
• – dezentrale Verarbeitung
• – steigende Echtzeit-Anforderungen
• – zunehmende Komplexität zu erfassender Zustände/Situationen

This trend arises from qualitatively new requirements for the sensor signal processing, eg:

• - increasing miniaturization
• - growing chip integration
• - intelligent procedures
• - decentralized processing
• - increasing real-time requirements
• - Increasing complexity of detected states / situations

The predominant value added lies next to the sensor elements themselves, especially in the following sensor-related signal processing. The sensor elements are always smaller, better, more integrated, more robust and less expensive.

conceivable Applications can be found in the main areas of wireless communication, Automotive industry and industrial process automation. There are for example, in the application fields to solve tasks such as Channel equalization, classification, and adaptive control, these being enumeration not final.

Claims (43)

Field programmable synapse array is characterized in that an optimized arrangement of optimized connections (synapses) is realized topologielos as "sea of synapses" on an integrated system with modular structure, which realizes a generic network of dendrites and axons, whereby processors (neurons) are interconnected so that they are layered and map a variety of topologies neural networks. Field Programmable synapse array according to claim 1 characterized in that the synapses are shaped that they are arranged modularly in a field and from the outside via programming with be connected to each neuron, so that both the number of synapses needed with variable algorithm and scalable network size as well the form of the neuron activation functions vary, creating a variable number of hidden layers allows and multiple networks of different or similar topology parallel and independent can be implemented from each other. Field Programmable synapse array after one of the claims 1 or 2, characterized in that the synapses in blocks to four each and between the top and bottom and between the right and left two synapses four pairs each are arranged, wherein the inputs and outputs of the synapse by variable Programming of transistor gates or fixed programming by Fuses be switched on any of these lines. Field Programmable synapse array after one of the claims 1 to 3, characterized in that the synapses in the synapse array are the same structure and addressable and the same functions realize. Field programmable synapse array according to claim 4, characterized in that in the synapses, the storage of a weight value in the form of a voltage by storage in the C-unit (poly-poly-cap or gate-cap) or in a digital register and the multiplication of stored weight value is performed with the input to the synapse input value (w _i · x _i ). Field Programmable synapse array after one of the claims 1 to 5, characterized in that for programming in each Synapse block is provided a register in which encodes the switch position is stored. Field Programmable synapse array after one of the claims 1 to 6, characterized in that for the neurons the linear, sigmoide, gaussian and quadratic functions are provided, these being individual selected or a combination of several is made. Field Programmable synapse array according to claim 7 characterized in that the linear and the sigmoidal function the standard functions are and in the input, hidden and output layers to disposal be put. Field Programmable synapse array according to claim 7 characterized in that the quadratic function only in the hidden layer is used. Field Programmable synapse array according to claim 7 characterized in that the Gaussian function only in the starting layer is used. Field Programmable synapse array after one the claims 7 to 9 characterized in that for the hidden neuron blocks respectively a block with linear, one with the sigmoiden and one with the square Function are implemented together with a synapse. Field Programmable synapse array after one the claims 7 to 11, characterized in that the neuron blocks on the left, the right and top edges of the net are placed, being at the bottom blocks arranged with input neurons and two output hidden neurons are. Field Programmable synapse array according to claim 12 characterized in that special output hidden neuron blocks arranged that are a combination of either linear and sigmoider Function or linear and gaussian Function included. Field Programmable synapse array after one the claims 12 to 13, characterized in that special output hidden neuron blocks arranged which contain a switchable function, the external loads can drive. Field Programmable synapse array after one the claims 1 to 14, characterized in that the neurons are combined into blocks and the wiring is done by programming a register. Field Programmable synapse array after one the claims 1 to 15, characterized in that several networks in parallel and independently can be implemented from each other. Field Programmable synapse array after one the claims 1 to 16 characterized in that the neurons and the synapses each pre-structured and these prestructured circuits via switches be interconnected. Field Programmable synapse array after one the claims 1 to 17, characterized in that the signal transmission between neurons, Synapses, inputs and outputs done by differential currents. Field Programmable synapse array after one the claims 1 to 18, characterized in that they have a switchable driver contains which makes it possible to control external loads without additional circuitry Expense between the arrangement and connected actuators operate. Field Programmable synapse array according to claim 19, characterized in that the influence of additional sources of error the output signals of the arrangement is avoided, since the signal flow between sensor and actuator is kept extremely short. It takes place no analog / digital or digital / analog conversion. In addition, the optimal Case no further drivers and converters needed. Method of programming a field Programmable synapse arrays thereby characterized net, that an arrangement of self-optimized connections (synapses) is realized topologielos as "sea of synapses" on an integrated system with modular structure, which realizes a generic network of dendrites and axons, whereby processors (neurons) are connected in such a way that they layered and map a variety of topologies neural networks and the teaching of the system is carried out by an automatic mechanism with which an independent adjustment of the neural network is possible. A method according to claim 21, characterized that the synapses are arranged in a modular array and by outside via programming be associated with each neuron, so that both the number of required Synapses with variable algorithm and scalable network size as well the shape of the neuron activation functions vary, thereby a variable number of hidden layers is enabled and multiple networks of different or similar topology parallel and independent can be implemented from each other. A method according to claim 21 or 22, characterized that for teaching the array a linear function, a sigmoid function, a quadratic function and a Gaussian function either individually or used in combination of several. Method according to claim 23, characterized that the sigmoid function and the linear function as standard activation functions be made available in the hidden and outgoing layers. Method according to claim 24, characterized that the standard activation functions in the multilayer perceptron be used. Method according to claim 23, characterized that the quadratic function is used only in the hidden layer becomes, whereby the function courses each over different parameters are set. Method according to one of claims 23 or 26, characterized that the gaussian Function is used only in the output layer, wherein the function curves each have different Parameters are set. A method according to one of claims 21 to 27 characterized in that, the hidden neurons functions play a synaptic function task by through a synapse, the function for calculating the Euclidean distance (w _i - x _i) is carried out ² by the sum of (w _i ² - 2 · w _i * x _i + x ² ), where the value w _i ^{2 is} generated in a biassynapse, the value - 2 · w _i · x _i in a general purpose synapse, and the value x ² in a hidden neuron and the outputs on a node to (wi-xi) ^{2 are} added. A method according to claim 21 or 22, characterized that input signals are multiplied by a weight value, subsequently a summation of the weighted input signals (scalar product) and after the scalar formation a transmission the sum by a special function, in particular by a linear, a sigmoide or a jump-shaped function, takes place. A method according to claim 29, characterized that with a weight layer and linear transfer function, the weights calculated analytically and exactly one global minimum exists. A method according to claim 29, characterized that with a weight layer and nonlinear transfer function the weights will conditionally be calculated analytically and with a simple iterative minimization method a global minimum is found. A method according to claim 31, characterized that the networks depending on the request to the topology with or without Biasneuronen and synapses are operated and / or the Biasneuronen as an additional Input neurons are used. A method according to claim 29, characterized that with one or more weight layers and linear or non-linear transfer function the weights for the to be solved Task with a statistical iterative minimization method be determined, where a plurality of local minima exists. A method according to claim 29, characterized in that with a parallel arrangement of meh Rere perceptron networks with one or more weight layers and linear or non-linear transfer function will process several completely independent sensor signals, the topologies used may be completely different. A method according to claim 21 or 22, characterized that formed a difference between input signal and weight, this afterwards squared and the sum of the squares is formed, with the distance (Euclidean distance) by dragging the square root of the sum is calculated and this is the calculation of the intervening Signal and stored weight vector corresponds. A method according to claim 35, characterized that only one layer and no special transfer function used in which is directly the sum of the Euclidean distances at the exit evaluated and in the weights the so-called prototypes of a Class are stored, creating at the output of the network a vector arises, which adjusts the square intervals of the input signal represents the stored class prototypes and by a determination of the minimum output signal, the matching class is found. A method according to claim 35, characterized that two layers are used, wherein the first layer is a simple weight layer, whose sums are subsequently in a special function in the form of a bell curve and the second layer of the perceptron type with linear output function is whose weights are calculated analytically, so that no statistical iterative minimization process is required. A method according to claim 21 or 22, characterized that fed back Networks are used in which the states of the neurons are new as long as calculated until the network converges to a sleep state is in which there is no change the activation states more results. A method according to claim 38, characterized that a stable state through the parameters input, weight matrix and thresholds of the neurons is determined. A method according to claim 38 or 39, characterized that the n-dimensional inputs with just as many arithmetic units how clusters should be determined and only one Unit of the network delivers an issue and at the same time all preventing others from doing so. A method according to claim 38 or 39, characterized that the n-dimensional inputs with just as many arithmetic units are processed, how clusters should be determined and the output is not is delivered by only one unit and the clustering during the Learning is generated. A method according to claim 38 or 39, characterized that the network consists of a single layer of n neurons, which are totally networked with each other, i. every neuron possesses a connection to every other neuron, no neuron directly with itself connected, the network is symmetrically weighted, the individual neurons each have a linear threshold function as Activation function is assigned and as input the usual weighted Total is done. A method according to claim 38 or 39, characterized that an incomplete Input pattern in the feedback System is entered and when returning to the input the missing shares are partially replenished and the improved Output pattern is presented to the network again as input, so that the System responded with a further improved output.