Neural networks for prediction and control

• US 7,395,251 B2
• Filed: 07/01/2005
• Issued: 07/01/2008
• Est. Priority Date: 07/01/2005
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: International Business Machines Corporation
• Inventors: Linsker, Ralph
• Primary Examiner(s): Holmes; Michael B
• Application Number:
  US11/171,447
• Publication Number:
  US 20070022068A1
• Time in Patent Office: 1,096 Days
• Field of Search: 706/23, 607/60
• US Class Current: 706/23
• CPC Class Codes: G05B 13/026 using a predictor
• CPC Class Codes: G05B 13/027 using neural networks only
• CPC Class Codes: G06N 3/08 Learning methods

Abstract

Neural networks for optimal estimation (including prediction) and/or control involve an execution step and a learning step, and are characterized by the learning step being performed by neural computations. The set of learning rules cause the circuit'"'"'s connection strengths to learn to approximate the optimal estimation and/or control function that minimizes estimation error and/or a measure of control cost. The classical Kalman filter and the classical Kalman optimal controller are important examples of such an optimal estimation and/or control function. The circuit uses only a stream of noisy measurements to infer relevant properties of the external dynamical system, learn the optimal estimation and/or control function, and apply its learning of this optimal function to input data streams in an online manner. In this way, the circuit simultaneously learns and generates estimates and/or control output signals that are optimal, given the network'"'"'s current state of learning.

Claims

1. A method for estimation, comprising the steps of:

(a) specifying an estimation error criterion, a class of allowed estimation functions, an initial estimation function selected from said class, and an initial measurement estimate,

(b) inputting at least one measurement vector,

(c) determining an updated estimation function using said estimation error criterion, a previously determined estimation function, a previous measurement estimate, and said at least one measurement vector,

(d) determining an updated measurement estimate using an estimation function and said measurement vector,

(e) outputting said updated measurement estimate, and

(f) iterating steps (b) through (e) a plurality of times, wherein the step of determining an updated estimation function is performed using a neural network equivalent system (NNES).

2. The method of claim 1, wherein the estimation error computed using said estimation error criterion approximates its minimum achievable value as the number of times said steps are repeated is increased.

3. The method of claim 1, wherein the result of said step of determining an updated estimation function does not depend directly on a plant state.

4. The method of claim 1, wherein said class of allowed estimation functions comprises a set of functions that compute a linear combination of said measurement vector and said previous measurement estimate, to produce said updated measurement estimate.

5. The method of claim 1, wherein said updated estimation function is jointly implemented by two or more sets of connection weights.

6. The method of claim 1, wherein said updated estimation function approximates a Kalman filter or extended Kalman filter estimation function.

7. The method of claim 6, further comprising the step of specifying a tolerance within which said updated estimation function is required to approximate said Kalman filter or extended Kalman filter estimation function, wherein said tolerance is met by performing a plurality of said iterations.

8. The method of claim 6, wherein said updating of the estimation function is performed by an incremental learning process at each time step.

9. The method of claim 1, wherein a plurality of measurement vectors is inputted, and wherein said step of determining an updated estimation function comprises the substeps of:

(a) determining a plurality of update quantities, one said update quantity corresponding to each of said plurality of measurement vectors, and

(b) performing said updating of the estimation function by combining said plurality of update quantities.

10. The method of claim 9, wherein said updated estimation function approximates a Kalman filter or extended Kalman filter estimation function.

11. The method of claim 10, wherein the closeness with which said estimation function approximates that produced by classical Kalman estimation or extended Kalman estimation, is improved as the number of said measurement vectors and said update quantities is increased.

12. The method of claim 1, further comprising the step of determining at least one parameter that describes the dynamics of a plant state, by means of a neural network equivalent system (NNES).

13. The method of claim 1, wherein said steps of determining an updated estimation function and of determining an updated measurement estimate are performed by means of a single neural network equivalent system (NNES).

14. The method of claim 13, wherein said measurement vector is input to the NNES at a plurality of distinct locations.

15. The method of claim 1 wherein said step of determining an updated estimation function comprises the substeps of:

(a) computing a plurality of measurement residual vectors, and

(b) computing an approximation of the covariance matrix and/or the covariance matrix inverse of said plurality of measurement residual vectors.

16. A method for control, comprising the steps of:

(a) specifying a control cost criterion, a class of allowed control functions, and an initial control function selected from said class,

(b) specifying a sequence of time values,

(c) for each time value in said sequence of time values, determining an updated control function corresponding to said time value using said control cost criterion and a previously determined control function,

(d) inputting state data comprising at least one of a plant state vector, a measurement vector, or a measurement estimate,

(e) determining a control vector using one of said updated control functions and said state data,

(f) outputting said control vector,

(g) optionally iterating steps (d) through (f) one or more times,

(h) iterating steps (b) through (g) one or more times,

wherein step (c) is performed using a neural network equivalent system (NNES).

17. The method of claim 16, wherein:

(a) step (c) comprises storing at least one said updated control function using storage means, and

(b) step (e) comprises obtaining said one of said updated control functions from said storage means.

18. The method of claim 16, wherein said step of specifying a sequence of time values comprises specifying a target time, and wherein each of said time values is less than said target time.

19. The method of claim 16, wherein said updated control function approximates a Kalman control function.

20. The method of claim 19, further comprising the step of specifying a tolerance within which said updated control function is required to approximate said Kalman control function, wherein said tolerance is met by performing a plurality of said iterations.

21. The method of claim 16, wherein the control cost computed using said control cost criterion approximates its minimum achievable value.

22. The method of claim 16, wherein the result of said step of determining an updated control function does not depend directly on a plant state.

23. The method of claim 16, wherein said steps of determining an updated control function and of determining a control vector are performed by means of a single neural network equivalent system (NNES).

24. The method of claim 16, wherein said step of determining an updated control function comprises the substeps of:

(a) computing a plurality of activity vectors, and

(b) computing an approximation of the covariance matrix and/or the covariance matrix inverse of said plurality of activity vectors.

25. A method for combined estimation and control, comprising the steps of:

an estimation step comprising the steps of;

(a) specifying an estimation error criterion, a class of allowed estimation functions, an initial estimation function selected from said class, and an initial measurement estimate,

(b) inputting at least one measurement vector,

(c) determining an updated estimation function using said estimation error criterion, a previously determined estimation function, a previous measurement estimate, and said at least one measurement vector,

(d) determining an updated measurement estimate using an estimation function and said measurement vector,

(e) outputting said updated measurement estimate, and

(f) iteration of steps (b) through (e) a plurality of times,

wherein the step of determining an updated estimation function is performed using a neural network equivalent system, and

a control step comprising the steps of;

(g) specifying a control cost criterion, a class of allowed control functions, and an initial control function selected from said class,

(h) specifying a sequence of time values,

(i) for each time value in said sequence of time values, determining an updated control function corresponding to said time value using said control cost criterion and a previously determined control function,

(j) inputting state data comprising at least one of a plant state vector, a measurement vector, or a measurement estimate,

(k) determining a control vector using one of said updated control functions and said state data,

(l) outputting said control vector,

(m) optionally iterating steps (j) through (l) one or more times,

(n) iterating steps (h) through (m) one or more times,

wherein step (i) is performed using a neural network equivalent system (NNES),

wherein said estimation and control steps are performed by a single neural network equivalent system (NNES).

26. In a method for estimation, control, system identification, reinforcement learning, supervised learning, unsupervised learning, and/or classification, comprising a step of iteratively transforming a first matrix into a second matrix, the improvement comprising the steps of:

(a) specifying a functional relationship between said first matrix and a first set of vectors,

(b) specifying a transformation of each vector in said first set of vectors into a vector of a second set of vectors,

(c) implementing said first set of vectors as a first set of activity vectors in a neural network equivalent system (NNES),

(d) implementing an approximation of said first matrix as a first set of connection strength values in said NNES,

(e) determining, by means of neural computations, a second set of connection strength values as a function of said first set of activity vectors, and

(f) determining, by means of neural computations, a second set of activity vectors as a function of said first set of activity vectors and of said first set of connection strength values, wherein said second set of connection strength values approximates said second matrix.

27. The method of claim 26, wherein said step of iteratively transforming a first matrix into a second matrix comprises solving a Riccati equation.

28. The method of claim 26, wherein said functional relationship between said first matrix and a first set of vectors is selected from the group consisting of (a) the functional relationship wherein said first matrix is equal or approximately equal to the covariance matrix of said first set of vectors and (b) the functional relationship wherein said first matrix is equal or approximately equal to the inverse of the covariance matrix of said first set of vectors.

29. The method of claim 26, wherein said method is a method selected from the group consisting of Kalman estimation, extended Kalman filtering, and Kalman control.

30. A neural network equivalent system (NNES), having a processor, comprising:

a plurality of activity vectors and at least one set of connection strength values, wherein said set of connection strength values is updated using a learning rule that comprises the application of a ganging operation to said plurality of activity vectors.

31. The system of claim 30, wherein said NNES is an NNES for estimation and/or control.

32. The system of claim 30, wherein said ganging operation performs a function selected from the group consisting of (a) approximating the covariance matrix of said plurality of activity vectors and (b) approximating the inverse of the covariance matrix of said plurality of activity vectors.

33. The system of claim 30, wherein said NNES is a hardware artificial neural network (ANN) or circuit and said ganging operation comprises the determination of which sets of activity vectors are to be ganged together at each of a plurality of time steps.

34. The system of claim 33, wherein said determination makes use of the relative timing and/or synchronization of the activities at a plurality of nodes.

35. A neural network equivalent system (NNES) for estimation and/or control comprising:

(a) a plurality of nodes connected to perform a sequence of steps in a prescribed order,

(b) said plurality of nodes comprising processors programmed or constructed to perform an execution step and/or a learning step using neural computations,

(c) input means for providing measurement values to said plurality of nodes,

(d) neural computation means for determining a plurality of measurement estimates, plant state estimates, and/or control vector signals using said measurement signals, and

(e) means for outputting said plurality of measurement estimates, plant state estimates, and/or control vector signals.

36. The NNES of claim 35, wherein said execution step combines at least one measurement and a first set of at least one measurement estimate using an estimation function to determine a second set of at least one measurement estimate.

37. The NNES of claim 35, wherein said learning step determines a second estimation function, using the first estimation function, a set of at least one measurement, and a set of at least one measurement estimate.

38. The NNES of claim 35, wherein:

(a) said processors compute a matrix of connection weights by learning within the NNES,

(b) the computation by said processors of said matrix of connection weights is influenced by past observations, and

(c) said matrix of connection weights cause a quantitative blending of an a priori estimated measurement vector with an actual measurement to obtain a measurement estimate that is closer than the actual measurements to what the actual measurement would have been in the absence of measurement noise.

39. The NNES of claim 38, wherein the learned weights satisfy or tend to approximate a Kalman optimal filter.

40. The NNES of claim 35, wherein the NNES is a recurrent NNES.

41. The NNES of claim 35, wherein the NNES comprises multiple physical layers of connected nodes in which selected nodes in separate layers are interconnected.

42. A recurrent neural network equivalent system (NNES), having a processor, for estimation wherein a measurement vector is input to the network at a plurality of distinct locations and/or distinct times.

43. A recurrent neural network equivalent system (NNES), having a processor, for estimation and/or control, comprising time-sequencing means whereby:

(a) the data flow is organized in time using major and minor time steps,

(b) each of said major time steps corresponds to the input of at least one measurement vector,

(c) each of said major time steps is divided into a plurality of minor time steps, and

(d) the data flow is organized so that specified network functions are carried out at specified locations and at specified minor time steps.

44. A recurrent neural network equivalent system (NNES) for estimation and/or control comprising:

(a) means for inputting a plurality of measurement vectors,

(b) means for learning an estimation and/or control function using said measurement vectors, and

(c) means for performing system identification using said measurement vectors.

45. A recurrent neural network equivalent system (NNES) for estimation and/or control comprising:

(a) means for storing, as a set of connection strengths, the values of a matrix,

(b) means for updating said values of said matrix, and

(c) means for computing the result of multiplying an activity vector by a member selected from the group consisting of said matrix and the inverse of said matrix.

46. The recurrent NNES of claim 45 wherein said means for updating said values of said matrix comprises:

(a) means for generating a plurality of matrix update instances, and

(b) means for combining said plurality of matrix update instances to determine said updated values of said matrix.