Predictive network with learned preprocessing parameters

• US 6,144,952 A
• Filed: 06/11/1999
• Issued: 11/07/2000
• Est. Priority Date: 09/20/1995
• Status: Expired due to Term

• Current Assignee: Rockwell Automation Technologies Incorporated (Allen-Bradley Co., Inc.)
• Original Assignee: Devendra B. Godbole, Eric J. Hartman, James D. Keeler, Jill L. Kempf, Steven A O'Hara
• Inventors: Keeler, James D., Hartman, Eric J., Godbole, Devendra B., Kempf, Jill L., O'Hara, Steven A.
• Primary Examiner(s): Dorvil, Richemond
• Application Number:
  US09/330,326
• Time in Patent Office: 515 Days
• Field of Search: 706/15, 706/25, 706/26, 706/27, 706/20, 706/22, 706/23, 706/21
• US Class Current: 706/21
• CPC Class Codes: G06N 3/047 Probabilistic or stochastic networks
• CPC Class Codes: G06N 3/049 Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs

Abstract

A predictive network is disclosed for operating in a runtime mode and in a training mode. The network includes a preprocessor (34'"'"') for preprocessing input data in accordance with parameters stored in a storage device (14'"'"') for output as preprocessed data to a delay device (36'"'"'). The delay device (36'"'"') provides a predetermined amount of delay as defined by predetermined delay settings in a storage device (18). The delayed data is input to a system model (26'"'"') which is operable in a training mode or a runtime mode. In the training mode, training data is stored in a data file (10) and retrieved therefrom for preprocessing and delay and then input to the system model (26'"'"'). Model parameters are learned and then stored in the storage device (22). During the training mode, the preprocess parameters are defined and stored in a storage device (14) in a particular sequence and delay settings are determined in the storage device (18). During the runtime mode, runtime data is derived from a distributed control system (24) and then preprocessed in accordance with predetermined process parameters and delayed in accordance with the predetermined delay settings. The preprocessed data is then input to the system model (26'"'"') to provide a predicted output, which is a control output to the distributed control system (24).

Claims

1. A predictive system for predicting the operation of a system, comprising:

a data storage device for storing historical data obtained during the operation of the system;

a data preprocessor for preprocessing received data from the system during operation thereof in accordance with predetermined preprocessing parameters to output preprocessed data;

a system model of the system having an input for receiving said preprocessed data and mapping it to an output through a stored representation of the system in accordance with associated model parameters that define said stored representation;

a control device for controlling said data preprocessor in a training mode to preprocess said stored historical data and output preprocessed historical data and, in a runtime mode, to receive and preprocess runtime data received from the system during operation thereof to output preprocessed runtime data;

a training device operating in said training mode to train said system model with said stored historical data in accordance with a predetermined training algorithm to define said model parameters; and

said system model operating in said runtime mode to generate a predicted output for the received output preprocessor runtime data from said data preprocessor.

2. The predictive system of claim 1, and further comprising:

an input device for determining said predetermined preprocessing parameters in accordance with predetermined criteria;

a parameter storage device for storing said determining preprocessing parameters after determination by said input device; and

said data preprocessor controlled by said control device to select said determined preprocessing parameters from said parameter storage device in said runtime mode and to operate under the control of said input device during said training mode.

3. The predictive system of claim 1, wherein said data preprocessor comprises:

an input buffer for receiving and storing data to be preprocessed, the data to be preprocessed being on different time scales;

a time merge device for selecting a predetermined time scale and reconciling the data stored in said input buffer such that all of the data is on the same time scale; and

an output device for outputting the data reconciled by said time merge device as said preprocessed data to said system model.

4. The network of claim 3, and further comprising a pre-time merge processor for applying a predetermined algorithm to the data to be preprocessed received by said input buffer prior to input to said time merge device.

5. The predictive system of claim 3, wherein said output device further comprises a post-time merge processor for applying a predetermining algorithm to the data reconciled by said time merge device prior to output as said preprocessed data.

6. The predictive system of claim 1 wherein said data preprocessor comprises:

an input buffer for receiving and storing said data to be preprocessed;

a delay device for receiving select portions of said data to be preprocessed from said input buffer and introducing a predetermined amount of delay therein to output delayed data; and

an output device for outputting the undelayed and delayed portions of said data to be preprocessed as said preprocessed data.

7. The predictive system of claim 6, wherein said received data comprises a plurality of variables, each of the variables comprising an input variable with an associated set of data, wherein said delay device is operable to receive at least a select one of said input variables and introduce said predetermined amount of delay therein to output a delayed input variable and an associated set of output delayed data having the associated delay.

8. The predictive system of claim 7, and further comprising means for determining said delay.

9. The predictive system of claim 1, wherein said system model is a non-linear neural network with an input layer for receiving said runtime data and providing a predicted output on an output layer in the runtime mode, and a hidden layer for mapping said input layer to said output layer through said stored representation of the system during operation thereof, said neural network operable in said training mode to receive said stored historical data on said input and output layers and define said model parameters in accordance with said predetermined training algorithm.

10. The predictive system of claim 1, wherein the system is a distributed control system and the output of said network provides control inputs to said system.

11. A predictive system for predicting the operation of a system, comprising:

a data storage device for storing historical data for a the system during operation thereof;

a training preprocessor for preprocessing said historical data in accordance with predetermined preprocessing parameters to output preprocessed training data;

a first memory for storing said preprocessing parameters;

a training network having model parameters associated therewith for receiving said preprocessed historical data and adjusting said model parameters in accordance with a predetermined training algorithm to generate a representation of the system during operation thereof;

a second memory for storing said adjusted model parameters associated with said generated system representation;

a runtime preprocessor substantially similar to said training preprocessor for receiving runtime data from the system during operation thereof and preprocessing said runtime data in accordance with said stored preprocessing parameters in said first memory to output said preprocessed runtime data; and

a runtime system model substantially similar to said training network for generating a representation of the system during operation thereof in accordance with said stored model parameters in said second memory and for receiving said preprocessed runtime data and generating a predicted output.

12. The predictive system of claim 11, wherein said runtime preprocessor operates in real time.

13. The predictive system of claim 11, wherein each of said training and runtime data preprocessors comprise:

an input buffer for receiving and storing data to be preprocessed, the received data being on different time scales;

a time merge device for selecting a predetermined time scale and reconciling the received data stored in said input buffer such that all of the received data is on the same time scale; and

an output device for outputting the data reconciled by said time merge device as said preprocessed data to the respective one of said training network or said runtime model.

14. The predictive system of claim 11 wherein each of said training and runtime data preprocessors comprise:

an input buffer for receiving and storing data to be preprocessed;

a delay device for receiving select portions of said received data from said input buffer and introducing a predetermined amount of delay therein to output delayed data; and

an output device for outputting the undelayed and delayed portions of said received data as said preprocessed data.

15. A method for generating a prediction in a predictive system of the operation of a system, comprising the steps of:

storing historical data received from the system during the operation thereof in a data storage device;

providing a data preprocessor that is operable to preprocess received data in accordance with predetermined preprocessing parameters to output preprocessed data;

providing a system model that is operable to map input data from an input layer to an output layer through a stored representation of the system during the operation thereof in accordance with associated model parameters that define the stored representation;

operating the data preprocessor in a training mode to receive the historical data from the data storage device and output preprocessed historical data;

training the system model on the preprocessed historical data to define the model parameters;

storing the trained model parameters generated in the step of training;

operating the data preprocessor in a runtime mode to receive runtime data and generate preprocessed runtime data; and

operating the system model with the trained system model parameters to receive on the input thereof the preprocessed runtime data and generate a predicted output on the output thereof.

16. The method of claim 15, and further comprising the steps of:

determining the predetermined preprocessing parameters in accordance with predetermined criteria;

storing the determined preprocessing parameters after determination thereof; and

selecting the stored determined preprocessing parameters in the runtime mode for the operation of the data preprocessor and determining the predetermined preprocessing parameters during the training mode.

17. The method of claim 15, wherein the step of operating the data preprocessor in both the runtime mode and the training mode comprises:

receiving and storing the data to be preprocessed, the data to be preprocessed being on different time scales;

selecting a predetermined time scale and time merging the data stored in the input buffer such that all of the time merged data is on the same time scale; and

outputting the time merged data as the preprocessed data.

18. The method of claim 15, wherein the step of operating the data preprocessor in both the runtime mode and the training mode comprises:

receiving and storing data to be preprocessed;

selecting portions of the stored data to be preprocessed and introducing a predetermined amount of delay therein to output delay data; and

outputting the undelayed and delayed portions of the data to be preprocessed as the preprocessed data.