Method for supervised teaching of a recurrent artificial neural network

• US 7,321,882 B2
• Filed: 10/05/2001
• Issued: 01/22/2008
• Est. Priority Date: 10/13/2000
• Status: Expired due to Term

• Current Assignee: Fraunhofer Gesellschaft Zur Foerderung Der Angewandten Forsching E.V.
• Original Assignee: Fraunhofer Gesellschaft Zur Foerderung Der Angewandten Forsching E.V.
• Inventors: Jaeger, Herbert
• Primary Examiner(s): Holmes, Michael B.
• Application Number:
  US10/398,914
• Publication Number:
  US 20040015459A1
• Time in Patent Office: 2,300 Days
• Field of Search: 706/30
• US Class Current: 706/30
• CPC Class Codes: G06N 3/08 Learning methods

Abstract

A method for the supervised teaching of a recurrent neutral network (RNN) is disclosed. A typical embodiment of the method utilizes a large (50 units or more), randomly initialized RNN with a globally stable dynamics. During the training period, the output units of this RNN are teacher-forced to follow the desired output signal. During this period, activations from all hidden units are recorded. At the end of the teaching period, these recorded data are used as input for a method which computes new weights of those connections that feed into the output units. The method is distinguished from existing training methods for RNNs through the following characteristics: (1) Only the weights of connections to output units are changed by learning—existing methods for teaching recurrent networks adjust all network weights. (2) The internal dynamics of large networks are used as a “reservoir” of dynamical components which are not changed, but only newly combined by the learning procedure—existing methods use small networks, whose internal dynamics are themselves completely re-shaped through learning.

Claims

1. A method for constructing a discrete-time recurrent neural network and training it in order to minimize its output error, comprising;

constructing a recurrent neural network as a reservoir for excitable dynamics (dynamical reservoir network;

providing means of feeding input to the dynamical reservoir network;

attaching output units to the dynamical reservoir network through weighted connections; and

training the weights of the connections only from the dynamical reservoir network to the output units in a supervised training scheme.

2. The method of claim 1, wherein the dynamical reservoir network has a number of units greater than 50.

3. The method of claim 1 or 2, wherein the dynamical reservoir network is sparsely connected.

4. The method of claim 1, wherein the connections within the dynamical reservoir network have randomly assigned weights.

5. The method of claim 1, wherein different update rules or differently parameterized update rules are used for different dynamical reservoir units.

6. The method of claim 1, wherein a spatial structure is imprinted on the dynamical reservoir network through the connectivity pattern.

7. The method of claim 6, wherein the spatial structure is a regular grid.

8. The method of claim 6, wherein the spatial structure is a local neighborhood structure induced by banded or subbanded structure of the connectivity matrix.

9. The method of claim 6, wherein the spatial structure is modular or organized in levels.

10. The method of claim 1, wherein the weights within the dynamical reservoir are globally scaled such that the resulting dynamics of the isolated dynamical reservoir network is globally stable.

11. The method of claim 1, wherein the weights within the dynamical reservoir are globally scaled such that the resulting dynamics of the isolated dynamical reservoir network is marginally globally stable, in order to achieve long duration of memory effects in the final network after training.

12. The method of claim 10 or 11, wherein input is fed to the dynamical reservoir by means of extra input units.

13. The method of claim 12, wherein the connections from the input units to the dynamical reservoir are sparse.

14. The method of claim 12, wherein the weights of connections from the input units to the dynamical reservoir are randomly fixed and have negative and positive signs.

15. The method of claim 12, wherein the weights of connections from the input units to the dynamical reservoir are globally scaled to small absolute values in order to achieve a long duration of memory effects in the final network I/O characteristics, or in order to achieve slow or low-pass time characteristics in the final network I/O characteristics, or in order to achieve nearly linear I/O characteristics.

16. The method of claim 12, wherein the weights of connections from the input units to the dynamical reservoir are globally scaled to absolute large values in order to achieve short duration of memory effects, or in order to achieve fast I/O behavior, or in order to achieve highly nonlinear or “

switching”

characteristics in the final trained network.

17. The method of claim 10 or 11, wherein input is fed to the dynamical reservoir by means other than by extra input units.

18. The method of claim 1, wherein extra output units are attached to the dynamical reservoir without feedback connections from the output units to the dynamical reservoir, in order to obtain a passive signal processing network after training.

19. The method of claim 1, wherein extra output units are attached to the dynamical reservoir with feedback connections from the output units to the dynamical reservoir, in order to obtain an active signal processing or signal generation network after training.

20. The method of claim 19, wherein the feedback connections are sparse.

21. The method of claim 19 or 20, wherein the weights of feedback connections are randomly fixed and have negative and positive signs.

22. The method of claim 19, wherein the weights of feedback connections are globally scaled to small absolute values in order to achieve a long duration of memory effects in the final network I/O characteristics, or in order to achieve slow or low-pass time characteristics in the final network I/O characteristics, or in order to achieve linear I/O characteristics.

23. The method of claim 19, wherein the weights of connections from the input units to the dynamical reservoir are globally scaled to absolute large values in order to achieve short duration of memory effects, or in order to achieve fast I/O behavior, or in order to achieve highly nonlinear or “

switching”

characteristics in the final trained network.

24. The method of claim 1, wherein the network is trained in an offline version of supervised teaching.

25. The method of claim 24, wherein the task to be learnt is a signal generation task, no input exists, and the teacher signal consists only of a sample of the desired output signal.

26. The method of claim 24, wherein the task to be learnt is a signal processing task, where input exists, and where the teacher signal consists of a sample of the desired input/output pairing.

27. The methods of any one of claims 24 to 26, wherein output-error-minimizing weights of the connections to the output nodes are computed, comprising;

presenting the teacher signals to the network and running the network in teacher-forced mode for the duration of the teaching period,

saving into a memory the network states and the signals f.sub.j.sup.−

1({tilde over (y)}.sub.j(t)) obtained by mapping the inverse of the output unit'"'"'s transfer function on the teacher output,

optionally discarding initial state/output pairs in order to accommodate initial transient effects, and

computing the weights of the connections to the output nodes by a standard linear regression method.

28. The method of claim 24, wherein during the training period noise is inserted into the network dynamics, by utilizing a noisy update rule and/or by adding noise on the input and/or by adding a noise component to the teacher output before it is fed back into the dynamical reservoir if output to the dynamical reservoir feedback connections exist.

29. The methods of claim 24, wherein weights of connection from only a subset of the networks units (i.e., a subset of the input, dynamical reservoir, output units) to the output units are trained, and the other ones are set to zero.

30. The methods of claim 1, wherein the network is trained in an online version of supervised teaching.

31. The method of claim 30, wherein the task to be learnt is a signal generation task, no input exists, and the teacher signal consists only of a sample of the desired output signal.

32. The method of claim 30, wherein the task to be learnt is a signal processing task, where input exists, and where the teacher signal consists of a sample of the desired input/output pairing.

33. The method of any one of claims 30 to 32, wherein output-error-minimizing weights of the connections to the output nodes are updated at every time step, the update comprising;

feeding the input to the network and updating the network,

for every output unit, computing an error as the difference between the desired teacher output and the actual network output;

or, alternatively, as the difference between the value f.sub.j.sup. −

1({tilde over (y)}.sub.j(t)) obtained by mapping the inverse of the output unit'"'"'s transfer function on the teacher output, and the value obtained by mapping the inverse of the output unit'"'"'s transfer function on the actual output (output state error),

updating the weights of the connections to the output nodes by a standard method for minimizing the error computed in the previous substep, and

in cases of signal generation tasks or active signal processing tasks, forcing the teacher output into the output units.

34. The method of claim 30, wherein noise is inserted into the network dynamics, by utilizing a noisy update rule or by adding a noise component to the teacher output before it is fed back into the dynamical reservoir if feedback connections exist.

35. The method of claim 30, wherein weights of connection from only a subset of the networks units (i.e., a subset of the input, dynamical reservoir, output units) to the output units are trained, and the other ones set to zero.

36. The method of claim 1, wherein the network is trained on two or more output units with feedback connections to the dynamical reservoir, which in the exploitation phase are utilized in any chosen “

direction”

, by treating any some of the trained units as input units and the remaining ones as output units to realize the learning of dynamical relationships between signals.

37. The method of claim 36 applied to tasks of reconstructive memory of multidimensional dynamical patterns, comprising;

training the network with teaching signals consisting of complete-dimensional samples of the patterns, and

in the exploitation phase, presenting cue patterns which are incompletely given in only some of the dimensions as input in those dimensions, and reading out the completed dynamical patterns on the remaining units.

38. The method of claim 1, applied to tasks of closed-loop state or observation feedback tracking control of a plant, comprising

using training samples consisting of two kinds of input signals to the network, namely, (i) a future version of the variables that will serve as a reference signal in the exploitation phase, and (ii) plant output or plant state observation;

and consisting further of a desired network output signal, namely, (iii) plant control input,

training a network using the teacher input and output signal from a., in order to obtain a network which computes as network output a plant control input depending on the current plant output observation and a future version of reference variables,

exploiting the network as an closed-loop controller by feeding it with the inputs (i) future reference signals, (ii) current plant output or plant state observation; and

letting the network generate the current plant control input.

39. A neural network for constructing a discrete-time recurrent neural network and training it in order to minimize its output error comprising;

a recurrent neural network as a reservoir for excitable dynamics dynamical reservoir network;

means for feeding input to the dynamical reservoir network;

output units attached to the dynamical reservoir network through weighted connections; and

wherein the weights of the connections are trained only from the dynamical reservoir network to the output units in a supervised training scheme.

40. A neural network according to claim 39, wherein it is implemented as a microcircuit.

41. A neural network according to claim 39, wherein it is implemented by a suitably programmed computer.