System and method for historical database training of support vector machines

• US 20030101161A1
• Filed: 11/28/2001
• Published: 05/29/2003
• Est. Priority Date: 11/28/2001
• Status: Active Grant

• Current Assignee: Rockwell Automation Technologies Incorporated (Rockwell Automation, Inc.)
• Original Assignee: Pavilion Technologies Incorporated (Rockwell Automation, Inc.)
• Inventors: Ferguson, Bruce, Hartman, Eric, Johnson, Doug, Hurley, Eric
• Granted Patent: US 6,944,616 B2
• Time in Patent Office: Days
• US Class Current: 707/1
• CPC Class Codes: G05B 13/0265 the criterion being a learning criterion
• CPC Class Codes: G05B 15/02 electric
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06F 18/2411 based on the proximity to a decision surface, e.g. support vector machines
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching

Abstract

A system and method for historical database training of a support vector machine (SVM). The SVM is trained with training sets from a stream of process data. The system detects availability of new training data, and constructs a training set from the corresponding input data. Over time, many training sets are presented to the SVM. When multiple presentations are needed to effectively train the SVM, a buffer of training sets is filled and updated as new training data becomes available. Once the buffer is full, a new training set bumps the oldest training set from the buffer. The training sets are presented one or more times each time a new training set is constructed. A historical database of time-stamped data may be used to construct training sets for the SVM. The SVM may be trained retrospectively by searching the historical database and constructing training sets based on the time-stamped data.

Claims

1. A method for training a support vector machine used to control a process, the method comprising:

(1) training a support vector machine using a first training set, wherein said first training set is based on first data;

(2) training said support vector machine using said first training set and a second training set, wherein said second training set is based on second data; and

(3) training said support vector machine using said second training set and a third training set, without using said first training set, wherein said third training set is based on third data;

wherein at least one of (1), (2), and (3) comprises;

(a) retrieving training input data from a historical database, wherein said training input data has one or more timestamps;

(b) selecting a training input data time period based on said one or more timestamps; and

(c) retrieving input data indicated by said training input data time period.

2. The method of claim 1, wherein at least one of (1), (2), and (3) operates substantially in real-time.

3. The method of claim 1,

wherein (1) is preceded by analyzing a physical specimen from the process;

and

wherein (1) further comprises using data representative of said analyzing as said first data.

4. A method for training a support vector machine using real-time data, the method comprising:

(1) detecting first data;

(2) training a support vector machine in response to said detecting first data, using a first training set based on said first data;

(3) detecting second data;

(4) training said support vector machine in response to said detecting second data, using said first training set and a second training set, wherein said second training set is based on said second data;

(5) detecting third data;

(6) training said support vector machine in response to said detecting third data, using said second training set and a third training set, without using said first training set, wherein said third training set is based on said third data;

wherein at least one of (2), (4), and (6) comprises;

(a) retrieving training input data from a historical database, wherein said training input data has one or more timestamps;

(b) selecting a training input data time period based on said one or more timestamps; and

(c) retrieving an input data indicated by said training input data time period.

5. The method of claim 4, further comprising discarding said first training set between (4) and (5).

6. The method of claim 4, further comprising discarding said second training set after (6).

7. A method for training a support vector machine, the method comprising:

(1) constructing a list containing at least two training sets;

(2) training the support vector machine using said at least two training sets in said list;

(3) constructing a new training set and replacing an oldest training set in said list with said new training set; and

(4) repeating (2) and (3) at least once;

wherein at least one of (1) and (3) comprises;

(a) retrieving training input data from a historical database, wherein said training input data has one or more timestamps;

(b) selecting a training input data time period based on said one or more timestamps; and

(c) retrieving an input data indicated by said training input data time period.

8. The method of claim 7, wherein (3) comprises:

(a) monitoring substantially in real-time for new training input data; and

(b) retrieving input data indicated by said new training input data to construct said new training set.

9. The method of claim 7, wherein (2) uses said at least two training sets once.

10. The method of claim 7, wherein (2) uses said at least two training sets at least twice.

11. A method for training a support vector machine using data from a physical process, the method comprising:

(1) operating the physical process and measuring the physical process to produce first data, second data, and third data;

(2) training a support vector machine using a first training set;

wherein said first training set is based on said first data;

(3) training said support vector machine using said first training set and a second training set, wherein said second training set is based on said second data; and

(4) training said support vector machine using said second training set and a third training set, without using said first training set, wherein said third training set is based on said third data;

wherein at least one of (2), (3), and (4) comprises;

(a) retrieving training input data from a historical database, wherein said training input data has one or more timestamps;

(b) selecting a training input data time period based on said one or more timestamps; and

(c) retrieving an input data indicated by said training input data time period.

12. A method for training a support vector machine for process control, the method comprising:

(1) training a support vector machine using a first training set, wherein said first training set is based on first data;

(2) training said support vector machine using said first training set and a second training set, wherein said second training set is based on second data;

(3) training said support vector machine using said second training set and a third training set, without using said first training set, wherein said third training set is based on third data;

(4) using said support vector machine to predict a first output data using first input data; and

(5) changing a physical state of an actuator in accordance with said first output data;

wherein at least one of (1), (2), and (3) comprises;

(a) retrieving training input data from a historical database, wherein said training input data has one or more timestamps;

(b) selecting a training input data time period based on said one or more timestamps; and

(c) retrieving an input data indicated by said training input data time period.

13. A method for training a support vector machine for process control using real-time data, the method comprising:

(1) detecting first data;

(2) training a support vector machine in response to said detecting first data, using a first training set, wherein said first training set is based on said first data;

(3) detecting second data;

(4) training said support vector machine in response to said detecting said second data, using said first training set and a second training set, wherein said second training set is based on said second data;

(5) detecting third data;

(6) training said support vector machine in response to said detecting third data, using said second training set and a third training set, without using said first training set, wherein said third training set is based on said third data;

(7) using said support vector machine to predict first output data using first input data; and

(8) changing a physical state of an actuator in accordance with said first output data;

wherein at least one of (2), (4), and (6) comprises;

(a) retrieving training input data from a historical database, wherein said training input data has one or more timestamps;

(b) selecting a training input data time period based on said one or more timestamps; and

(c) retrieving an input data indicated by said training input data time period.

14. A method for training a support vector machine using real-time data from a physical process, the method comprising:

(1) operating the physical process and measuring the physical process to produce first data, second data, and third data;

(2) detecting said first data;

(3) training a support vector machine in response to said detecting first data, using a first training set, wherein said first training set is based on said first data;

(4) detecting said second data;

(5) training said support vector machine in response to said detecting second data, using said first training set and a second training set;

wherein said second training set is based on said second data;

(6) detecting said third data; and

(7) training said support vector machine in response to said detecting third data, using said second training set and a third training set, without using said first training set, wherein said third training set is based on said third data;

wherein at least one of (3), (5), and (7) comprises;

(a) retrieving training input data from a historical database, wherein said training input data has one or more timestamps;

(b) selecting a training input data time period based on said one or more timestamps; and

(c) retrieving an input data indicated by said training input data time period.

15. A method for constructing training sets for a support vector machine, the method comprising:

(1) developing a first training set for a support vector machine by;

(a) retrieving first training input data from a historical database, wherein said first training input data has a first one or more timestamps;

(b) selecting a first training input data time period based on said first one or more timestamps; and

(c) retrieving first input data indicated by said first training input data time period; and

(2) developing a second training set for said support vector machine by;

(a) retrieving second training input data from said historical database, wherein said second training input data has a second one or more timestamps;

(b) selecting a second training input data time period based on said second one or more timestamps; and

(c) retrieving second input data indicated by said second training input data time period.

16. The method of claim 15, further comprising:

(3) searching said historical database in either a forward time direction or a backward time direction so that said second training input data is the next training input data in time to said first training input data in said forward time direction or said backward time direction, whichever is used.

17. The method of claim 15, further comprising:

(3) training said support vector machine using said first training set and/or said second training set.

18. A computer support vector machine process control method adapted for predicting output data provided to a controller used to control a process for producing a product having at least one product property, the computer support vector machine process control method comprising:

a processor;

a memory medium coupled to the processor, wherein the memory medium stores a support vector machine software program, wherein the support vector machine software program comprises;

(1) monitoring for the availability of new training input data by monitoring for a change in an associated timestamp of said training input data;

(2) constructing a training set by retrieving first input data corresponding to said training input data;

(3) training the support vector machine using said training set; and

(4) predicting the output data from second input data using the support vector machine.

19. The computer support vector machine process control method of claim 18, wherein (2) further comprises using data pointers to indicate said training input data and said first input data.

20. The computer support vector machine process control method of claim 18,

wherein (1) is preceded by:

(i) presenting to a user a template for a partially specified support vector machine; and

(ii) entering data into said template to create a complete support vector machine specification; and

wherein (3) further comprises using a support vector machine representative of said complete support vector machine specification.

21. The computer support vector machine process control method of claim 18,

wherein (1) is preceded by:

(i) presenting to a user an interface for accepting a limited set of substantially natural language format specifications; and

(ii) entering into said interface sufficient specifications in said substantially natural language format to completely define a support vector machine; and

wherein (3) further comprises using a support vector machine representative of said completely defined support vector machine.

22. The computer support vector machine process control method of claim 18, wherein (1), (2), and (3) operate substantially in real-time.

23. A method for constructing training sets for a support vector machine, the method comprising:

(a) retrieving training input data from a historical database, wherein said training input data has one or more timestamps;

(b) selecting a training input data time period based on said one or more timestamps; and

(c) retrieving input data indicated by said training input data time period.

24. A computer support vector machine process control method adapted for predicting output data provided to a controller used to control a process for producing a product having at least one product property, the computer support vector machine process control method comprising:

(1) monitoring for the availability of new training input data;

(2) constructing a training set by retrieving first input data corresponding to said training input data comprising;

(a) selecting a training input data time using a one or more timestamps associated with said training input data; and

(b) retrieving input data representing measurement(s) at said training input data time, said input data comprising said first input data;

(3) training the support vector machine using said training set; and

(4) predicting the output data from second input data using the support vector machine.

25. The computer support vector machine process control method of claim 24, wherein (1) comprises monitoring for a change between two successive training input data values.

26. The computer support vector machine process control method of claim 24,

wherein (1) comprises computing a difference between a most recent training input data value and a next most recent training input value;

and

wherein (3) further comprises using said difference with said first input data for said training.

27. The computer support vector machine process control method of claim 24, wherein (2) further comprises using data pointers to indicate said training input data and said first input data.

28. The computer support vector machine process control method of claim 24, wherein (1), (2), and (3) operate substantially in real-time.

29. A computer support vector machine process control method adapted for predicting output data provided to a controller used to control a process for producing a product having at least one product property, the computer support vector machine process control method comprising:

(1) presenting to a user a template for a partially specified support vector machine;

(2) entering data into said template to create a complete support vector machine specification;

(3) monitoring for the availability of new training input data;

(4) constructing a training set by retrieving first input data corresponding to said training input data;

(5) training the support vector machine using said training set, said training further comprising using a support vector machine representative of said complete support vector machine specification; and

(6) predicting the output data from second input data using the support vector machine.

30. The computer support vector machine process control method of claim 29, wherein (3) comprises monitoring for a change between two successive training input data values.

31. The computer support vector machine process control method of claim 29,

wherein (3) comprises computing a difference between a most recent training input data value and a next most recent training input value;

and

wherein (5) further comprises using said difference with said first input data for said training.

32. The computer support vector machine process control method of claim 29, wherein (4) further comprises using data pointers to indicate said training input data and said first input data.

33. The computer support vector machine process control method of claim 29, wherein (3), (4), and (5) operate substantially in real-time.

34. A computer support vector machine process control method adapted for predicting output data provided to a controller used to control a process for producing a product having at least one product property, the computer support vector machine process control method comprising:

(1) presenting to a user an interface for accepting a limited set of substantially natural language format specifications;

(2) entering into said interface sufficient specifications in said substantially natural language format to completely define a support vector machine;

(3) monitoring for the availability of new training input data;

(4) constructing a training set by retrieving first input data corresponding to said training input data;

(5) training the support vector machine using said training set, wherein said training comprises using a support vector machine representative of said completely defined support vector machine; and

(6) predicting the output data from second input data using the support vector machine.

35. The computer support vector machine process control method of claim 34, wherein (3) comprises monitoring for a change between two successive training input data values.

36. The computer support vector machine process control method of claim 34,

wherein (3) comprises computing a difference between a most recent training input data value and a next most recent training input value;

and

wherein (5) further comprises using said difference with said first input data for said training.

37. The computer support vector machine process control method of claim 34, wherein (4) further comprises using data pointers to indicate said training input data and said first input data.

38. The computer support vector machine process control method of claim 34, wherein (3), (4), and (5) operate substantially in real-time.

39. A method for training a support vector machine used to control a process, the method comprising:

building a first training set using training data, wherein said training data includes one or more timestamps indicating a chronology of said training data and one or more process parameter values corresponding to each timestamp, and wherein said first training set comprises process parameter values corresponding to a first time period in said chronology;

training a support vector machine using said first training set.

40. The method of claim 39, wherein said building a first training set comprises:

retrieving said training data from a historical database;

selecting a training data time period based on said one or more timestamps; and

retrieving said process parameter values from said training data indicated by said training data time period, wherein said first training set comprises said retrieved process parameter values in chronological order over said selected training data time period.

41. The method of claim 40, further comprising:

generating a second training set by;

removing at least a subset of the parameter values of said first training set, wherein said at least a subset of the parameter values comprises oldest parameter values of said training set; and

adding new parameter values from said training data based on said timestamps to generate a second training set;

wherein said second training set corresponds to a second time period in said chronology; and

training a support vector machine using said second training set.