Predictive network with learned preprocessing parameters

• US 5,479,573 A
• Filed: 01/25/1993
• Issued: 12/26/1995
• Est. Priority Date: 11/24/1992
• Status: Expired due to Term

• Current Assignee: Rockwell Automation Technologies Incorporated (Allen-Bradley Co., Inc.)
• Original Assignee: Pavilion Technologies Incorporated (Rockwell Automation, Inc.)
• Inventors: Godbole, Devandra B., Keeler, James D., Hartman, Eric J., Kempf, Jill L., O'Hara, Steven A.
• Primary Examiner(s): MacDonald, Allen R.
• Assistant Examiner(s): Dorvil, Richemond
• Application Number:
  US08/008,218
• Time in Patent Office: 1,065 Days
• Field of Search: 395/22, 395/24, 395/23, 395/11, 364/569, 364/703.04
• US Class Current: 706/21
• CPC Class Codes: G06F 17/17 Function evaluation by approximation methods, e.g. inter- or extrapolation, smoothing, least mean square method G06F17/18 takes precedence ; interpolation for numerical control G05B19/18
• CPC Class Codes: G06N 3/047 Probabilistic or stochastic networks
• CPC Class Codes: G06N 3/049 Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs

Abstract

A predictive network is disclosed for operating in a runtime mode and in a training mode. The network includes a preprocessor (34'"'"') for preprocessing input data in accordance with parameters stored in a storage device (14'"'"') for output as preprocessed data to a delay device (36'"'"'). The delay device (36'"'"') provides a predetermined amount of delay as defined by predetermined delay settings in a storage device (18). The delayed data is input to a system model (26'"'"') which is operable in a training mode or a runtime mode. In the training mode, training data is stored in a data file (10) and retrieved therefrom for preprocessing and delay and then input to the system model (26'"'"'). Model parameters are learned and then stored in the storage device (22). During the training mode, the preprocess parameters are defined and stored in a storage device (14) in a particular sequence and delay settings are determined in the storage device (18). During the runtime mode, runtime data is derived from a distributed control system (24) and then preprocessed in accordance with predetermined process parameters and delayed in accordance with the predetermined delay settings. The preprocessed data is then input to the system model (26'"'"') to provide a predicted output, which is a control output to the distributed control system (24).

Claims

1. A predictive network, comprising:

a data storage device for storing non-time synchronous training data from a runtime system;

a data preprocessor for preprocessing received non-time synchronous data in accordance with predetermined preprocessing parameters to time synchronous output preprocessed data;

a system model having an input for receiving said time synchronous preprocessed data and mapping it to an output through a stored representation of said runtime system in accordance with associated model parameters that define said stored representation;

a control device for controlling said data preprocessor in a training mode to preprocess said stored non-time synchronous training data and output time synchronous preprocessed training data and, in a runtime mode, to receive and preprocess non-time synchronous runtime data received from said runtime system to output preprocessed time synchronous runtime data;

a training device operating in said training mode to train said system model with said stored time synchronous training data in accordance with a predetermined training algorithm to define said model parameters; and

said system model operating in said runtime mode to generate a predicted output for the received non-time synchronous runtime data from said data preprocessor.

2. The network of claim 1, and further comprising:

an input device for determining said predetermined preprocessing parameters in accordance with predetermined criteria;

a parameter storage device for storing said determined preprocessing parameters after determination by said input device; and

said data preprocessor controlled by said control device to select said determined preprocessing parameters from said parameter storage device in said runtime mode and to operate under the control of said input device during said training mode.

3. The network of claim 1, wherein said data preprocessor comprises:

an input buffer for receiving and storing non-time synchronous data to be preprocessed, non-time synchronous the data to be preprocessed being on different time scales;

a time merge device for selecting a predetermined time scale and reconciling the non-time synchronous data stored in said input buffer such that all of the data is on the same time scale; and

an output device for outputting the data reconciled by said time merge device as said preprocessed data to said system model.

4. The network of claim 3, and further comprising a pre-time merge processor for applying a predetermined algorithm to the data to be preprocessed received by said input buffer prior to input to said time merge device.

5. The network of claim 3, wherein said output device further comprises a post-time merge processor for applying a predetermining algorithm to the data reconciled by said merge device prior to output as said time synchronous preprocessed data.

6. The network of claim 1 wherein said data preprocessor includes:

an input buffer for receiving and storing said non-time synchronous data to be preprocessed;

a delay device for receiving select portions of said non-time synchronous data to be preprocessed from said input buffer and introducing a predetermined amount of delay therein to output delayed data; and

an output device for outputting the undelayed and delayed portions of said non-time synchronous data to be preprocessed as said time synchronous preprocessed data.

7. The data preprocessor of claim 6, wherein said received noon-time synchronous data comprises a plurality of variables, each of the variables comprising an input variable with an associated set of data, wherein said delay device is operable to receive at least a select one of said input variables and introduce said predetermined amount of delay therein to output a delayed input variable and an associated set of output delayed data having the associated delay.

8. The data preprocessor of claim 7, and further comprising means for determining said delay.

9. The network of claim 1, wherein said system model is a non-linear neural network with an input layer for receiving said runtime time synchronous data and providing a predicted output on an output layer in the runtime mode, and a hidden layer for mapping said input layer to said output layer through said stored representation of said routine system, said neural network operable in said training mode to receive said stored preprocessed time synchronous training data on said input and output layers and define said model parameters in accordance with said predetermined training alogorithm.

10. The network of claim 1, wherein said runtime system is a distributed control system and the output of said network provides control inputs to said system.

11. A predictive network, comprising:

a data storage device for storing training data for a runtime system;

a training preprocessor for preprocessing said training data in accordance with predetermined preprocessing parameters to output preprocessed training data;

a first memory for storing said preprocessing parameters;

a training network having model parameters associated therewith for receiving said preprocessed training data and adjusting said model parameters in accordance with a predetermined training algorithm to generate a representation of said runtime system;

a second memory for storing said adjusted model parameters associated with said generated system representation;

a runtime preprocessor substantially similar to said training preprocessor for receiving runtime data from said runtime system and preprocessing said runtime data in accordance with said stored preprocessing parameters in said first memory to output said preprocessed runtime data; and

a runtime network substantially similar to said training network for generating a representation of said runtime system in accordance with said stored model parameters in said second memory and for receiving said preprocessed runtime data and generating a predicted output.

12. The network of claim 11, wherein said runtime preprocessor operates in real time.

13. The network of claim 11, wherein each of said training and runtime data preprocessors comprise:

an input buffer for receiving and storing data to be preprocessed, the received data being on different time scales;

a time merge device for selecting a predetermined time scale and reconciling the received data stored in said input buffer such that all of the received data is on the same time scale; and

an output device for outputting the data reconciled by said time merge device as said preprocessed data to the respective one of said training network or said runtime network.

14. The network of claim 11 wherein each of said training and runtime data preprocessors comprise:

an input buffer for receiving and storing data to be preprocessed;

a delay device for receiving select portions of said received data from said input buffer and introducing a predetermined amount of delay therein to output delayed data; and

an output device for outputting the undelayed and delayed portions of said received data as said preprocessed data.

15. A method for generating a prediction in a predictive network, comprising the steps of:

storing training data received from a runtime system in a data storage device;

preprocessing received non-time synchronous data using a data preprocessor in accordance with predetermined preprocessing parameters to time synchronous and output time synchronous preprocessed data;

mapping input data from an input layer of a system model to an output layer of the system model through a stored representation of the runtime system in accordance with associated model parameters that define the stored representation;

operating the data preprocessor in a training mode to receive the non-time synchronous training data from the data storage device and output preprocessed time synchronous training data;

training the system model on the preprocessed non-time synchronous training data to define the model parameters;

storing the trained model parameters generated in the step of training;

operating the data preprocessor in a runtime mode to receive non-time synchronous runtime data and generate time synchronous preprocessed runtime data; and

operating the system model with the trained system model parameters to receive on the input thereof the time synchronous preprocessed runtime data and generate a predicted output on the output thereof.

16. The method of claim 15, and further comprising the steps of:

determining the predetermined preprocessing parameters in accordance with predetermined criteria;

storing the determined preprocessing parameters after determination thereof; and

selecting the stored determined preprocessing parameters in the runtime mode for the operation of the data preprocessor and determining the predetermined preprocessing parameters during the training mode.

17. The method of claim 15 wherein, the step of operating the data preprocessor in both the runtime mode and the training mode comprises:

receiving and storing the non-time synchronous data to be preprocessed, the non-time synchronous data to be preprocessed being on different time scales;

selecting a predetermined time scale and time merging the non-time synchronous data stored in the input buffer such that all of the time merged data is on the same time scale; and

outputting the time merged data as the time synchronous preprocessed data.

18. The method of claim 15, wherein the step of operating the data preprocessor in both the runtime mode and the training mode includes the steps of:

receiving and storing non-time synchronous data to be preprocessed;

selecting portions of the stored non-time synchronous data to be preprocessed and introducing a predetermined amount of delay therein to output delayed data; and

outputting the undelayed and delayed portions of the non-time synchronous data to be preprocessed as the time synchronous preprocessed data.