Historical database training method for neural networks

• US 5,408,586 A
• Filed: 04/02/1993
• Issued: 04/18/1995
• Est. Priority Date: 08/03/1990
• Status: Expired due to Term

• Current Assignee: Rockwell Automation Technologies Incorporated (Allen-Bradley Co., Inc.)
• Original Assignee: E. I. du Pont de Nemours and Company (Corteva, Inc.)
• Inventors: Skeirik, Richard D.
• Primary Examiner(s): MacDonald, Allen R.
• Assistant Examiner(s): SHAPIRO, STUART
• Application Number:
  US08/042,500
• Time in Patent Office: 746 Days
• Field of Search: 395/22, 395/23, 395/24, 395/27
• US Class Current: 706/25
• CPC Class Codes: G05B 13/027 using neural networks only
• CPC Class Codes: Y10S 706/906 Process plant

Abstract

An on-line training neural network for process control system and method trains by retrieving training sets from the stream of process data. The neural network detects the availability of new training data, and constructs a training set by retrieving the corresponding input data. The neural network is trained using the training set. Over time, many training sets are presented to the neural network.

When multiple presentations are needed to effectively train, a buffer of training sets is filled and updated as new training data becomes available. The size of the buffer is selected in accordance with the training needs of the neural network. Once the buffer is full, a new training set bumps the oldest training set off the top of the buffer stack. The training sets in the buffer stack can be presented one or more times each time a new training set is constructed.

An historical database of timestamped data can be used to construct training sets when training input data has a time delay from sample time to availability for the neural network. The timestamps of the training input data are used to select the appropriate timestamp at which input data is retrieved for use in the training set. Using the historical database, the neural network can be trained retrospectively by searching the historical database and constructing training sets based on past data.

Claims

1. A method for training a neural network, comprising the steps of:

(1) training a neural network using a first training set based on first lab data;

(2) training or retraining said neural network using a second training set based on second lab data, and using said first training set; and

(3) training or retraining said neural network using a third training set based on third lab data, and using said second training set, without using said first training set;

wherein at least one of steps (1), (2), (3) comprises the steps of,

(a) retrieving, from an historical database, training input data having a timestamp(s);

(b) selecting a training input data time based on said timestamp(s); and

(c) retrieving an input data indicated by said training input data time.

2. The method of claim 1, wherein at least one said steps (1), (2), or (3) operates in real-time.

3. The method of claim 1, wherein:

step (1) is preceded by a step of (a) analyzing a physical specimen from the process; and

wherein step (1) further comprises a step of using data representative of said analyzing step (a) as said lab data.

4. A method for training a neural network using real-time data, comprising the steps of:

(1) detecting first lab data;

(2) training or retraining a neural network, when said first lab data is detected by step (1), by using a first training set based on said first lab data;

(3) detecting second lab data;

(4) training or retraining said neural network, when said second lab data is detected by step (3), by using a second training set based on said second lab data, and by using said first training set;

(5) detecting third lab data;

(6) training or retraining said neural network, when said third lab data is detected by step (5), by using a third training set based on said third lab data, and using said second training set, without using said first training set;

wherein at least one of steps (2), (4), and (6) comprises the steps of,

(a) retrieving, from an historical database, training input data having a timestamp(s);

(b) selecting a training input data time based on said timestamp(s); and

(c) retrieving an input data indicated by said training input data time.

5. The method of claim 4, further comprising between steps (4) and (5) the step of discarding said first training set.

6. The method of claim 4, further comprising after step (6) the step of discarding said second training step.

7. A method for training a neural network, comprising the steps of:

(1) constructing a stack containing at least two training sets;

(2) training or retraining the neural network using said at least two training sets in said stack;

(3) constructing a new training set and replacing an oldest training set in said stack with said new training set; and

(4) repeating steps (2) and (3) at least once;

wherein at least one of steps (1) and (3) comprises the steps of,

(a) retrieving, form an historical database, training input data having a timestamp(s);

(b) selecting a training input data time based on said timestamp(s); and

(c) retrieving an input data indicated by said training input data time.

8. The method of claim 7, wherein step (3) comprises the steps of:

(a) monitoring in real time for the presence of new training input data; and

(b) retrieving input data indicated by said new training input data to construct said new training set.

9. The method of claim 7, wherein step (2) of training uses only once said at least two training sets of said stack.

10. The method of claim 7, wherein step (2) of training uses at least twice said at least two training sets of said stack.

11. A method for training a neural network using data from a physical process, comprising the steps of:

(1) operating the physical process and measuring the same to produce a first lab data, a second lab data, and a third lab data;

(2) training a neural network using a first training set based on first lab data;

(3) training or retraining said neural network using a second training set based on second lab data, and using said first training set; and

(4) training or retraining;

said neural network using a third training set based on third lab data, and using said second training set, without using said first training set;

wherein at least one of steps (2), (3), and (4) comprises the steps of,

(a) retrieving, from an historical database, training input data having a timestamp(s);

(b) selecting a training input data time based on said timestamp(s); and

(c) retrieving an input data indicated by said training input data time.

12. A method for training a neural network for process control, comprising the steps of:

(1) training a neural network using a first training set based on first lab data;

(2) training or retraining said neural network using a second training set based on second lab data, and using said first training set;

(3) training or retraining said neural network using a third training set based on third lab data, and using said second training set, without using said first training set;

(4) predicting, using said neural network, a first output data using a first input data; and

(5) changing a physical state of an actuator in accordance with said first output data;

wherein at least one of steps (1), (2), and (3) comprises the steps of,

(a) retrieving, front an historical database, training input data having a timestamp(s);

(b) selecting a training input data time based on said timestamp(s); and

(c) retrieving an input data indicated by said training input data time.

13. A method for training a neural network for process control using real-time data, comprising the steps of:

((1) detecting first lab data;

(2) training or retraining a neural network, when said first lab data is detected by step (1), by using a first training set based on said first lab data;

(3) detecting second lab data;

(4) training or retraining said neural network, when said second lab data is detected by step (3), by using a second training set based on said second lab data, and by using a second training set based on second lab data, and by using said first training set;

(5) detecting third lab data;

(6) training or retraining said neural network, when said third lab data is detected in step (5), by using a third training set based on said second lab data, and by using a third training set based on third lab data, and using said second training set, without using said first training set;

(7) predicting, using said neural network, a first output data using a first input data; and

(8) changing a physical state of an actuator in accordance with said first output data;

wherein at least one of steps (2), (4), and (6) comprises the steps of,

(a) retrieving, from an historical database, training input data having a timestamp(s);

(b) selecting a training output data time based on said timestamp(s); and

(c) retrieving an input data indicated by said training input data time.

14. A method for training a neural network using real-time data from a physical process, comprising the steps of:

(1) operating the physical process and measuring the same to produce a first lab data, a second lab data, and a third lab data;

(2) detecting first lab data;

(3) training or retraining a neural network, when said first lab data is detected by step (2), by using a first training set based on said first lab data;

(4) detecting second lab data;

(5) training or retraining said neural network, when said second lab data is detected by step (4), by using a second training set based on said second lab data, and by using a second training set based on second lab data, and by using said first training set;

(6) detecting third lab data; and

(7) training or retraining said neural network, when said third lab data is detected in step (6), by using a third training set based on said second lab data, and by using a third training set based on third lab data, and using said second training set, without using said first training set;

wherein at least one of steps (3), (5), and (7) comprises the steps of,

(a) retrieving, from an historical database, training input data having a timestamp(s);

(b) selecting a training input data time based on said timestamp(s); and

(c) retrieving an input data indicated by said training input data time.