Fixed-weight recurrent neural network controller with fixed long-term and adaptive short-term memory

• US 7,647,284 B2
• Filed: 01/12/2007
• Issued: 01/12/2010
• Est. Priority Date: 01/12/2007
• Status: Expired due to Fees

• Current Assignee: Toyota Motor Corporation
• Original Assignee: Toyota Motor Engineering & Manufacturing North America Incorporated (Toyota Motor Corporation)
• Inventors: Prokhorov, Danil V.
• Primary Examiner(s): Vincent; David R
• Assistant Examiner(s): Tran; Mai T
• Application Number:
  US11/622,766
• Publication Number:
  US 20080172349A1
• Time in Patent Office: 1,096 Days
• Field of Search: 706/15, 700/48
• US Class Current: 706/15
• CPC Class Codes: G05B 13/027 using neural networks only
• CPC Class Codes: G06N 3/044 Recurrent networks, e.g. Hopfield networks
• CPC Class Codes: G06N 3/045 Combinations of networks

Abstract

A controller for a plant having a fixed-weight recurrent neural network with at least one external input signal representative of a desired condition of the plant and actual condition of the plant, and an output connected as a control signal to the plant. The fixed recurrent neural network includes a set of nodes with fixed weight interconnections between the nodes and at least one feedback input interconnecting an output from at least one of the nodes to an input of at least one node. These nodes collectively determine the value of the output from the neural network as a function of the input signal and the feedback input. The controller also includes an adaptive neural network having a plurality of nodes with variable weight interconnections between the nodes. A cost input from the plant is connected to the adaptive neural network while an output from the adaptive neural network is coupled as a processed feedback signal to nodes of the fixed-weight recurrent neural network.

Claims

1. Apparatus for controlling a plant comprising:

a fixed-weight recurrent neural network having a processor with memory and at least one external input signal representative of a desired condition of the plant, an output from the recurrent neural network connected as a control signal to the plant, a set of nodes with fixed weight interconnections between said nodes and at least one feedback input interconnecting an output from at least one of said nodes to an input of at least one node, said nodes collectively determining the value of an output of the fixed-weight recurrent neural network as a function of the value(s) of said at least one external input signal and said at least one feedback input,

an adaptive neural system having a cost input corresponding to a difference between a target value for the plant and an actual value for the plant, an output and a plurality of nodes with variable weight interconnections between said nodes, said adaptive neural system output being coupled to at least one feedback input of said fixed-weight recurrent neural network to thereby vary a short-term memory of the fixed-weight recurrent neural network without changing a long-term memory of the fixed-weight recurrent neural network to optimize the cost input.

2. The apparatus as defined in claim 1 wherein said adaptive neural system comprises a recurrent neural network.

3. The apparatus as defined in claim 1 wherein said adaptive neural system comprises an adaptive critic having at least one input connected to said cost input and an output connected to a finite difference algorithm, an output of said finite difference algorithm forming said output from said adaptive neural system.

4. The apparatus as defined in claim 3 wherein said finite difference processor utilizes a simultaneous perturbation stochastic approximation.

5. The apparatus as defined in claim 3 wherein said adaptive critic produces a prediction of the cost parameter for use by the finite difference algorithm.

6. The apparatus as defined in claim 1 wherein said adaptive neural system receives at least one node output signal from said fixed-weight neural network as an input signal.

7. The apparatus as defined in claim 1 and comprising means for adjusting the weights between the adaptive critic nodes in real-time operation.

8. The apparatus as defined in claim 7 wherein said adjusting means comprises means for performing stochastic meta-descent optimization on the node weights.

9. Apparatus for controlling a plant comprising:

a fixed-weight recurrent neural network having a processor with memory, at least one external input signal representative of a desired condition of the plant, an output from The recurrent neural network connected as a control signal to The plant, a set of nodes with fixed weight interconnections between said nodes, said nodes comprising short-term memory and said weights comprising long-term memory, and both said nodes and said weights defining a fixed-weight recurrent neural network,

an adaptive neural system having a cost input corresponding to a difference between a target value for the plant and an actual value for the plant, an output and a plurality of nodes with variable weight interconnections between said nodes, said adaptive neural system output being coupled to at least one feedback input of said fixed-weight recurrent neural network to thereby vary the state of said fixed-weight recurrent neural network without changing a long-term memory of the fixed-weight recurrent neural network.

10. The apparatus as defined in claim 9 wherein said adaptive neural system comprises a recurrent neural network.

11. The apparatus as defined in claim 9 wherein said adaptive neural system comprises an adaptive critic having at least one input connected to said cost signal and an output connected to a finite difference algorithm, an output of said finite difference algorithm forming said output from said adaptive neural system.

12. The apparatus as defined in claim 11 wherein said finite difference processor utilizes a simultaneous perturbation stochastic approximation.

13. The apparatus as defined in claim 11 wherein said adaptive critic produces a prediction of the cost parameter for use by the finite difference algorithm.

14. The apparatus as defined in claim 9 wherein said adaptive neural system receives at least one node output signal from said fixed-weight neural network as an input signal.

15. The apparatus as defined in claim 9 and comprising means for adjusting the weights between the nodes in real-time operation.

16. The apparatus as defined in claim 15 wherein said adjusting means comprises means for performing stochastic meta-descent optimization on the node weights.