Method for supervised teaching of a recurrent artificial neural network

• US 20040015459A1
• Filed: 05/29/2003
• Published: 01/22/2004
• Est. Priority Date: 10/13/2000
• Status: Active Grant

• Current Assignee: Fraunhofer Gesellschaft Zur Foerderung Der Angewandten Forsching E.V.
• Original Assignee: Fraunhofer Gesellschaft Zur Foerderung Der Angewandten Forsching E.V.
• Inventors: Jaeger, Herbert
• Granted Patent: US 7,321,882 B2
• Time in Patent Office: Days
• US Class Current: 706/15
• CPC Class Codes: G06N 3/08 Learning methods

Abstract

A method for the supervised teaching of a recurrent neutral network (RNN) is disclosed. A typical embodiment of the method utilizes a large (50 units or more), randomly initialized RNN with a globally stable dynamics. During the training period, the output units of this RNN are teacher-forced to follow the desired output signal. During this period, activations from all hidden units are recorded. At the end of the teaching period, these recorded data are used as input for a method which computes new weights of those connections that feed into the output units. The method is distinguished from existing training methods for RNNs through the following characteristics: (1) Only the weights of connections to output units are changed by learning—existing methods for teaching recurrent networks adjust all network weights. (2) The internal dynamics of large networks are used as a “reservoir” of dynamical components which are not changed, but only newly combined by the learning procedure—existing methods use small networks, whose internal dynamics are themselves completely re-shaped through learning.

Claims

1. A method for constructing a discrete-time recurrent neural network and training it in order to minimize its output error, comprising the steps

a. constructing a recurrent neural network as a reservoir for excitable dynamics (DR network);

b. providing means of feeding input to the DR network;

c. attaching output units to the DR network through weighted connections;

d. training the weights of the connections from the DR network to the output units in a supervised training scheme.

2. The method of claim 1, wherein the DR network has a large number of units (greater than 50).

3. The method of claim 1 or 2, wherein the DR network is sparsely connected.

4. The method of any one of claims 1 to 3, wherein the connections within the DR network have randomly assigned weights.

5. The method of any one of claims 1 to 4, wherein different update rules or differently parameterized update rules are used for different DR units.

6. The method of any one of claims 1 to 5, wherein a spatial structure is imprinted on the DR network through the connectivity pattern.

7. The method of claim 6, wherein the spatial structure is a regular grid.

8. The method of claim 6, wherein the spatial structure is a local neighborhood structure (induced by banded or subbanded structure of the connectivity matrix).

9. The method of claim 6, wherein the spatial structure is modular or organized in levels.

10. The method of any one of claims 1 to 9, wherein the weights within the DR are globally scaled such that the resulting dynamics of the isolated DR network is globally stable.

11. The method of any one of claims 1 to 9, wherein the weights within the DR are globally scaled such that the resulting dynamics of the isolated DR network is marginally globally stable, in order to achieve long duration of memory effects in the final network after training.

12. The method of claim 10 or 11, wherein input is fed to the DR by means of extra input units.

13. The method of claim 12, wherein the connections from the input units to the DR are sparse.

14. The method of claim 12 or 13, wherein the weights of connections from the input units to the DR are randomly fixed and have negative and positive signs.

15. The method of any one of claims 12 to 14, wherein the weights of connections from the input units to the DR are globally scaled to small absolute values in order to achieve a long duration of memory effects in the final network I/O characteristics, or in order to achieve slow or low-pass time characteristics in the final network I/O characteristics, or in order to achieve nearly linear I/O characteristics.

16. The method of any one of claims 12 to 14, wherein the weights of connections from the input units to the DR are globally scaled to absolute large values in order to achieve short duration of memory effects, or in order to achieve fast I/O behavior, or in order to achieve highly nonlinear or “

switching”

characteristics in the final trained network.

17. The method of claim 10 or 11, wherein input is fed to the DR by means other than by extra input units.

18. The method of any one of claims 1 to 17, wherein extra output units are attached to the DR without feedback connections from the output units to the DR, in order to obtain a passive signal processing network after training.

19. The method of any one of claims 1 to 17, wherein extra output units are attached to the DR with feedback connections from the output units to the DR, in order to obtain an active signal processing or signal generation network after training.

20. The method of claim 19, wherein the feedback connections are sparse.

21. The method of claim 19 or 20, wherein the weights of feedback connections are randomly fixed and have negative and positive signs.

22. The method of any one of claims 19 to 21, wherein the weights of feedback connections are globally scaled to small absolute values in order to achieve a long duration of memory effects in the final network I/O characteristics, or in order to achieve slow or low-pass time characteristics in the final network I/O characteristics, or in order to achieve linear I/O characteristics.

23. The method of any one of claims 19 to 21, wherein the weights of connections from the input units to the DR are globally scaled to absolute large values in order to achieve short duration of memory effects, or in order to achieve fast I/O behavior, or in order to achieve highly nonlinear or “

switching”

characteristics in the final trained network.

24. The method of any one of claims 1 to 23, wherein the network is trained in an offline version of supervised teaching.

25. The method of claim 24, wherein the task to be learnt is a signal generation task, no input exists, and the teacher signal consists only of a sample of the desired output signal.

26. The method of claim 24, wherein the task to be learnt is a signal processing task, where input exists, and where the teacher signal consists of a sample of the desired input/output pairing.

27. The methods of any one of claims 24 to 26, wherein output-error-minimizing weights of the connections to the output nodes are computed, comprising the steps

a. presenting the teacher signals to the network and running the network in teacher-forced mode for the duration of the teaching period,

b. saving into a memory the network states and the signals f_j⁻

1({tilde over (y)}_j(t)) obtained by mapping the inverse of the output unit'"'"'s transfer function on the teacher output,

c. optionally discarding initial state/output pairs in order to accommodate initial transient effects,

d. computing the weights of the connections to the output nodes by a standard linear regression method.

28. The method of any one of claims 24 to 27, wherein during the training period noise is inserted into the network dynamics, by utilizing a noisy update rule and/or by adding noise on the input and/or (if output-to-DR feedback connections exist) by adding a noise component to the teacher output before it is fed back into the DR.

29. The methods of any one of claims 24 to 28, wherein weights of connection from only a subset of the networks units (i.e., a subset of the input, DR, output units) to the output units are trained, and the other ones are set to zero.

30. The methods of any one of claims 1 to 23, wherein the network is trained in an online version of supervised teaching.

31. The method of claim 30, wherein the task to be learnt is a signal generation task, no input exists, and the teacher signal consists only of a sample of the desired output signal.

32. The method of claim 30, wherein the task to be learnt is a signal processing task, where input exists, and where the teacher signal consists of a sample of the desired input/output pairing.

33. The method of any one of claims 30 to 32, wherein output-error-minimizing weights of the connections to the output nodes are updated at every time step, the update comprising the substeps

a. feeding the input to the network and updating the network,

b. for every output unit, computing an error as the difference between the desired teacher output and the actual network output (output value error);

or, alternatively, as the difference between the value f_j<sup>−

1</sup>({tilde over (y)}_j(t)) obtained by mapping the inverse of the output unit'"'"'s transfer function on the teacher output, and the value obtained by mapping the inverse of the output unit'"'"'s transfer function on the actual output (output state error),

c. updating the weights of the connections to the output nodes by a standard method for minimizing the error computed in the previous substep b.,

d. in cases of signal generation tasks or active signal processing tasks, forcing the teacher output into the output units.

34. The method of any one of claims 30 to 33, wherein noise is inserted into the network dynamics, by utilizing a noisy update rule or (optionally, if feedback connections exist) by adding a noise component to the teacher output before it is fed back into the DR.

35. The method of any one of claims 30 to 34, wherein weights of connection from only a subset of the networks units (i.e., a subset of the input, DR, output units) to the output units are trained, and the other ones set to zero.

36. The method of any one of claims 1 to 35, wherein the network is trained on two or more output units with feedback connections to the DR, which in the exploitation phase are utilized in any chosen “

direction”

, by treating any some of the trained units as input units and the remaining ones as output units. (This realizes the learning of dynamical relationships between signals.)

37. The method of claim 36 applied to tasks of reconstructive memory of multidimensional dynamical patterns, comprising

a. training the network with teaching signals consisting of complete-dimensional samples of the patterns,

b. in the exploitation phase, presenting cue patterns which are incompletely given in only some of the dimensions as input in those dimensions, and reading out the completed dynamical patterns on the remaining units.

38. The method of any one of claims 1 to 35, applied to tasks of closed-loop (state or observation feedback) tracking control of a plant, comprising

a. using training samples consisting of two kinds of input signals to the network, namely, (i) a future version of the variables that will serve as a reference signal in the exploitation phase, and (ii) plant output observation (or plant state observation);

and consisting further of a desired network output signal, namely, (iii) plant control input,

b. training a network using the teacher input and output signal from a., in order to obtain a network which computes as network output a plant control input (i.e., (iii)), depending on the current plant output observation (i.e., (ii)) and a future version of reference variables (i.e., (i)),

c. exploiting the network as an closed-loop controller by feeding it with the inputs (i) future reference signals, (ii) current plant output observation (or plant state observation); and

letting the network generate the current plant control input.

39. A neural network constructed and trained according to any one of the preceeding claims.

40. A neural network according to claim 39, wherein it is implemented as a microcircuit.

41. A neural network according to claim 39, wherein it is implemented by a suitably programmed computer.