Method and apparatus for preprocessing input data to a neural network

• US 5,729,661 A
• Filed: 01/25/1993
• Issued: 03/17/1998
• Est. Priority Date: 11/24/1992
• Status: Expired due to Term

• Current Assignee: Rockwell Automation Technologies Incorporated (Allen-Bradley Co., Inc.)
• Original Assignee: Pavilion Technologies Incorporated (Rockwell Automation, Inc.)
• Inventors: Keeler, James D., Hartman, Eric J., Godbole, Devendra B., Kempf, Jill L., O'Hara, Steven A.
• Primary Examiner(s): MacDonald, Allen R.
• Assistant Examiner(s): Dorvil, Richemond
• Application Number:
  US08/008,170
• Time in Patent Office: 1,877 Days
• Field of Search: 395/22, 395/24, 395/11, 395/23, 364/569, 364/703.09
• US Class Current: 706/16
• CPC Class Codes: G06F 17/17 Function evaluation by approximation methods, e.g. inter- or extrapolation, smoothing, least mean square method G06F17/18 takes precedence ; interpolation for numerical control G05B19/18
• CPC Class Codes: G06N 3/047 Probabilistic or stochastic networks
• CPC Class Codes: G06N 3/049 Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs

Abstract

A preprocessing system for preprocessing input data to a neural network includes a training system for training a model (20) on data from a data file (10). The data is first preprocessed in a preprocessor (12) to fill in bad or missing data and merge all the time values on a common time scale. The preprocess operation utilizes preprocessing algorithms and time merging algorithms which are stored in a storage area (14). The output of the preprocessor (12) is then delayed in a delay block (16) in accordance with delay settings in storage area (18). These delayed outputs are then utilized to train the model (20), the model parameter is then stored in a storage area (22) during run time, a distributed control system (24) outputs the data to a preprocess block (34) and then preprocesses data in accordance with the algorithms in storage area (14). These outputs are then delayed in accordance with a delay block (36) with the delay settings (18). The output of the delay block (36) comprises inputs to a run time system model (26) which is built to provide a representation of the system in accordance with the model parameters in the storage area (22). A predicted control output or predicted control inputs are then generated. The control input is input back to the DCS (24).

Claims

1. A data preprocessor for preprocessing input data prior to input to a system model having multiple inputs, each of the inputs associated with a portion of the input data, comprising:

an input buffer for receiving and storing the input data, the input data associated with at least two of the inputs being on different time scales relative to each other;

a time merge device for selecting a predetermined time scale and reconciling the input data stored in the input buffer such that all of the input data for all of the inputs is on the same time scale; and

an output device for outputting the data reconciled by the time merge device as reconciled data, said reconciled data comprising the input data to the system model.

2. The data preprocessor of claim 1, and further comprising a pretime merge processor for applying a predetermined algorithm to the input data received by said input buffer prior to input to said time merge device.

3. The data preprocessor of claim 2, wherein each piece of data has associated therewith a time value corresponding to the time the input data was generated.

4. The data preprocessor of claim 3, wherein the input data processed by said pre-time merge processor has associated therewith the time value of the input data prior to processing by said pre-time merge processor.

5. The data preprocessor of claim 2, and further comprising an input device for selecting said predetermined algorithm from a group of available algorithms.

6. The data preprocessor of claim 1, wherein said output device further comprises a post-time merge processor for applying a predetermined algorithm to the data reconciled by said time merge device prior to output as said reconciled data.

7. The data preprocessor of claim 6, and further comprising an input device for selecting said predetermined algorithm from a group of available algorithms.

8. The data preprocessor of claim 1, wherein the system model is a non-linear network having a set of model parameters defining a representation of a system, said model parameters capable of being trained, the input data comprised of target input data and target output data, said reconciled data comprised of target reconciled input data and target reconciled output data, and further comprising a training device for training said non-linear network according to a predetermined training algorithm applied to said reconciled target input data and said reconciled target output data to develop new model parameters such that said non-linear network has stored therein a representation of the system that generated the target input data and the target output data.

9. The data preprocessor of claim 1, wherein said input buffer is controllable to arrange the input data in a predetermined format.

10. The data preprocessor of claim 9, wherein the input data, prior to being arranged in said predetermined format, has a predetermined time reference for all data, such that each piece of input data has associated therewith a time value relative to said predetermined time reference.

11. The data preprocessor of claim 1, wherein each piece of data has associated therewith a time value corresponding to the time the input data was generated.

12. The data preprocessor of claim 1, wherein the input data is comprised of a plurality of variables, each of the variables comprising an input variable with an associated set of data wherein each of said variables comprises an input to said input buffer and a potential one of the inputs to the system model.

13. The data preprocessor of claim 12, wherein select ones of said input variables and the associated set of data are on different time scales.

14. The data preprocessor of claim 12, and further comprising a delay device for receiving reconciled data associated with a select one of said input variables and introducing a predetermined mount of delay to said reconciled data to output a delayed input variable and associated set of delayed input reconciled data.

15. The data preprocessor of claim 14, wherein said predetermined amount of delay is a function of an external variable and further comprising means for varying said predetermined amount of delay as the function of said external variable.

16. The data preprocessor of claim 14, and further comprising means for learning said predetermined delay as a function of training parameters generated by a system corresponding to the system model.

17. The data preprocessor of claim 14, and further comprising means for determining said predetermined amount of delay.

18. The data preprocessor of claim 14, wherein said delay device comprises a plurality of buffers, each having a predetermined delay length at least as long as said predetermined delay, each of said input buffers having associated therewith one of said buffers with said delayed input variable determined by tapping said buffer associated with the select one of said input variables at a predetermined point along said delay length.

19. The data preprocessor of claim 1, wherein the input data associated with at least one of the inputs has missing or bad data in the associated time sequence and said time merge device is operable to reconcile said missing or bad data.

23. The method of claim 2, wherein the input data, after having the predetermined algorithm applied thereto, has the time value of the input associated therewith that existed prior to the step of applying the predetermined algorithm thereto.

20. A method for preprocessing input data prior to input to a system model having multiple inputs, each of the inputs associated with a portion of the input data, comprising the steps of:

receiving and storing the input data in an input buffer, the input data associated with at least two of the inputs being on different time scales relative to each other;

selecting a predetermined time scale and time merging the input data for all of the inputs such that all of the input data is reconciled on the same time scale; and

outputting the reconciled time merged data as reconciled data, the reconciled data comprising the input data to the system model.

21. The method of claim 20 and further comprising applying the predetermined algorithm to the input data received by the input buffer prior to the step of time merging.

22. The method of claim 21, wherein each piece of data has associated therewith a time value corresponding to the time the input data was generated.

24. The method of claim 21, and further comprising the step of selecting the predetermined algorithm from a group of available algorithms.

25. The method of claim 20, wherein the step of outputting comprises applying a predetermined algorithm to the data after the time merging step and prior to output as the reconciled data.

26. The method of claim 25, and further comprising selecting the predetermined algorithm from a group of available algorithms.

27. The method of claim 20, wherein the system model is a non-linear network having a set of model parameters defining a representation of a system, the model parameters capable of being trained, the input data comprised of target input data and target output data, the reconciled data comprised of target reconciled input data and target reconciled output data, and further comprising the step of training the non-linear network according to a non-linear training algorithm applied to the reconciled target input data and the reconciled target output data to develop new model parameters such that the non-linear network has stored therein a representation of the system that generated the target input data and the target output data.

28. The method of claim 20, and further comprising the step of arranging the input data in a predetermined format prior to input to the input buffer.

29. The method of claim 28, wherein the input data prior to being arranged in the predetermined format, has a predetermined time reference for all data, such that each piece of input data has associated therewith a time value relative to the predetermined time reference.

30. The method of claim 20, wherein each piece of data has associated therewith a time value corresponding to the time the input data was generated.

31. The method of claim 20, wherein the input data is comprised of a plurality of variables, each of the variables comprising an input variable with an associated set of data wherein each of the variables comprises an input to the input buffer and a potential input to the system model, and further comprising the steps of:

receiving the reconciled data associated with a select one of the input variables; and

introducing a predetermined amount of delay to the reconciled data to output a delayed input variable and an associated set of delayed input reconciled data.

32. The method of claim 20, wherein the input data associated with at least one of the inputs has missing or bad data in the associated time sequence and the time merging step further reconciles the missing or bad data.