System and method for historical database training of support vector machines

• US 6,944,616 B2
• Filed: 11/28/2001
• Issued: 09/13/2005
• Est. Priority Date: 11/28/2001
• Status: Expired due to Term

• Current Assignee: Rockwell Automation Technologies Incorporated (Allen-Bradley Co., Inc.)
• Original Assignee: Pavilion Technologies Incorporated (Rockwell Automation, Inc.)
• Inventors: Hurley, Eric S., Johnson, William Douglas, Ferguson, Ralph Bruce, Hartman, Eric J.
• Primary Examiner(s): Alam, Shahid Al
• Application Number:
  US09/997,627
• Publication Number:
  US 20030101161A1
• Time in Patent Office: 1,385 Days
• Field of Search: 707/10, 707/104.1, 707/2, 707/3, 707/5, 707/100, 706/16, 706/25, 706/7, 706/12, 706/20, 706/21, 706/23, 709/223, 709/206, 715/513, 715/514, 700/44, 716/5, 705/9
• US Class Current: 1/1
• CPC Class Codes: G05B 13/0265 the criterion being a learning criterion
• CPC Class Codes: G05B 15/02 electric
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06F 18/2411 based on the proximity to a decision surface, e.g. support vector machines
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching

Abstract

A system and method for historical database training of a support vector machine (SVM). The SVM is trained with training sets from a stream of process data. The system detects availability of new training data, and constructs a training set from the corresponding input data. Over time, many training sets are presented to the SVM. When multiple presentations are needed to effectively train the SVM, a buffer of training sets is filled and updated as new training data becomes available. Once the buffer is full, a new training set bumps the oldest training set from the buffer. The training sets are presented one or more times each time a new training set is constructed. A historical database of time-stamped data may be used to construct training sets for the SVM. The SVM may be trained retrospectively by searching the historical database and constructing training sets based on the time-stamped data.

Claims

1. A computer implemented method for training a support vector machine, the method comprising:

(1) constructing a list containing at least two training sets;

(2) training the support vector machine using said at least two training sets in said list;

(3) constructing a new training set and replacing an oldest training set in said list with said new training set; and

(4) repeating (2) and (3) at least once;

wherein at least one of (1) and (3) comprises;

(a) retrieving training input data from a historical database, wherein said training input data has one or more timestamps;

(b) selecting a training input data time period based on said one or more timestamps; and

(c) retrieving an input data indicated by said training input data time period.

2. The method of claim 1, wherein (3) comprises:

(a) monitoring substantially in real-time for new training input data; and

(b) retrieving input data indicated by said new training input data to construct said new training set.

3. The method of claim 1, wherein (2) uses said at least two training sets once.

4. The method of claim 1, wherein (2) uses said at least two training sets at least twice.

5. A computer implemented method for constructing training sets for a support vector machine, the method comprising:

(1) developing a first training set for a support vector machine by;

(a) retrieving first training input data from a historical database, wherein said first training input data has a first one or more timestamps;

(b) selecting a first training input data time period based on said first one or more timestamps; and

(c) retrieving first input data indicated by said first training input data time period; and

(2) developing a second training set for said support vector machine by;

(a) retrieving second training input data from said historical database, wherein said second training input data has a second one or more timestamps;

(b) selecting a second training input data time period based on said second one or more timestamps; and

(c) retrieving second input data indicated by said second training input data time period.

6. The method of claim 5, further comprising:

(3) searching said historical database in either a forward time direction or a backward time direction so that said second training input data is the next training input data in time to said first training input data in said forward time direction or said backward time direction, whichever is used.

7. The method of claim 5, further comprising:

(3) training said support vector machine using said first training set and/or said second training set.

8. A computer support vector machine process control method adapted for predicting output data provided to a controller used to control a process for producing a product having at least one product property, the computer support vector machine process control method comprising:

a processor;

a memory medium coupled to the processor, wherein the memory medium stores a support vector machine software program, wherein the support vector machine software program comprises;

(1) monitoring for the availability of new training input data by monitoring for a change in an associated timestamp of said training input data;

(2) constructing a training set by retrieving first input data corresponding to said training input data;

(3) training the support vector machine using said training set; and

(4) predicting the output data from second input data using the support vector machine.

9. The computer support vector machine process control method of claim 8, wherein (2) further comprises using data pointers to indicate said training input data and said first input data.

10. The computer support vector machine process control method of claim 8, wherein (1) is preceded by:

(i) presenting to a user a template for a partially specified support vector machine; and

(ii) entering data into said template to create a complete support vector machine specification; and

wherein (3) further comprises using a support vector machine representative of said complete support vector machine specification.

12. The computer support vector machine process control method of claim 8,

wherein (1), (2), and (3) operate substantially in real-time.

13. A computer support vector machine process control method adapted for predicting output data provided to a controller used to control a process for producing a product having at least one product property, the computer support vector machine process control method comprising:

(1) monitoring for the availability of new training input data;

(2) constructing a training set by retrieving first input data corresponding to said training input data comprising;

(a) selecting a training input data time using a one or more timestamps associated with said training input data; and

(b) retrieving input data representing measurement(s) at said training input data time, said input data comprising said first input data;

(3) training the support vector machine using said training set; and

(4) predicting the output data from second input data using the support vector machine.

14. The computer support vector machine process control method of claim 13, wherein (1) comprises monitoring for a change between two successive training input data values.

15. The computer support vector machine process control method of claim 13,

wherein (1) comprises computing a difference between a most recent training input data value and a next most recent training input value;

and

wherein (3) further comprises using said difference with said first input data for said training.

16. The computer support vector machine process control method of claim 13, wherein (2) further comprises using data pointers to indicate said training input data and said first input data.

17. The computer support vector machine process control method of claim 13, wherein (1), (2), and (3) operate substantially in real-time.

18. A computer implemented method for training a support vector machine used to control a process, the method comprising:

building a first training set using training data, wherein said training data includes one or more timestamps indicating a chronology of said training data and one or more process parameter values corresponding to each timestamp, and wherein said first training set comprises process parameter values corresponding to a first time period in said chronology;

training a support vector machine using said first training set.

11. The computer support vector machine process control method of claim 18, wherein (1) is preceded by:

(i) presenting to a user an interface for accepting a limited set of substantially natural language format specifications; and

(ii) entering into said interface sufficient specifications in said substantially natural language format to completely define a support vector machine; and

wherein (3) further comprises using a support vector machine representative of said completely defined support vector machine.

19. The method of claim 18, wherein said building a first training set comprises:

retrieving said training data from a historical database;

selecting a training data time period based on said one or more timestamps; and

retrieving said process parameter values from said training data indicated by said training data time period, wherein said first training set comprises said retrieved process parameter values in chronological order over said selected training data time period.

20. The method of claim 19, further comprising:

generating a second training set by;

removing at least a subset of the parameter values of said first training set, wherein said at least a subset of the parameter values comprises oldest parameter values of said training set; and

adding new parameter values from said training data based on said timestamps to generate a second training set;

wherein said second training set corresponds to a second time period in said chronology; and

training a support vector machine using said second training set.

21. A computer redable carrier medium which stores program instructions for training a support vector machine, wherein the program instructions are executable to perform:

(1) constructing a list containing at least two training sets;

(2) training the support vector machine using said at least two training sets in said list;

(3) constructing a new training set and replacing an oldest training set in said list with said new training set; and

(4) repeating (2) and (3) at least once;

wherein at least one of (1) and (3) comprises;

(a) retrieving training input data from a historical database, wherein said training input data has one or more timestamps;

(b) selecting a training input data time period based on said one or more timestamps; and

(c) retrieving an input data indicated by said training input data time period.

22. The carrier medium of claim 21, wherein (3) comprises:

(a) monitoring substantially in real-time for new training input data; and

(b) retrieving input data indicated by said new training input data to construct said new training set.

23. The carrier medium of claim 21, wherein (2) uses said at least two training sets once.

24. The carrier medium of claim 21, wherein (2) uses said at least two training sets at least twice.

25. A computer redable carrier medium which stores program instructions for constructing training sets for a support vector machine, wherein the program instructions are executable to perform:

(1) developing a first training set for a support vector machine by;

(a) retrieving first training input data from a historical database, wherein said first training input data has a first one or more timestamps;

(b) selecting a first training input data time period based on said first one or more timestamps; and

(c) retrieving first input data indicated by said first training input data time period; and

(2) developing a second training set for said support vector machine by;

(a) retrieving second training input data from said historical database, wherein said second training input data has a second one or more timestamps;

(b) selecting a second training input data time period based on said second one or more timestamps; and

(c) retrieving second input data indicated by said second training input data time period.

26. The carrier medium of claim 25, wherein the program instructions are further executable to perform:

(3) searching said historical database in either a forward time direction or a backward time direction so that said second training input data is the next training input data in time to said first training input data in said forward time direction or said backward time direction, whichever is used.

27. The carrier medium of claim 25, wherein the program instructions are further executable to perform:

(3) training said support vector machine using said first training set and/or said second training set.

28. A computer redable carrier medium which stores program instructions for predicting out put data provided to a controller used to control a process for producing a product having at least one product property, wherein the program instructions are executable to perform:

(1) monitoring for the availability of new training input data;

(2) constructing a training set by retrieving first input data corresponding to said training input data comprising;

(a) selecting a training input data time using a one or more timestamps associated with said training input data; and

(b) retrieving input data representing measurement(s) at said training input data time, said input data comprising said first input data;

(3) training the support vector machine using said training set; and

(4) predicting the output data from second input data using the support vector machine.

29. The carrier medium of claim 28, wherein (1) comprises monitoring for a change between two successive training input data values.

30. The carrier medium of claim 28,

wherein (1) comprises computing a difference between a most recent training input data value and a next most recent training input value;

and

wherein (3) further comprises using said difference with said first input data for said training.

31. The carrier medium of claim 28, wherein (2) further comprises using data pointers to indicate said training input data and said first input data.

32. The carrier medium of claim 28 wherein (1), (2), and (3) operate substantially in real-time.

33. A computer readable carrier medium which stores program instructions for training a support vector machine used to control a process, wherein the program instructions are executable to perform:

building a first training set using training data, wherein said training data includes one or more timestamps indicating a chronology of said training data and one or more process parameter values corresponding to each timestamp, and wherein said first training set comprises process parameter values corresponding to a first time period in said chronology;

training a support vector machine using said first training set.

34. The carrier medium of claim 33, wherein said building a first training set comprises:

retrieving said training data from a historical database;

selecting a training data time period based on said one or more timestamps; and

retrieving said process parameter values from said training data indicated by said training data time period, wherein said first training set comprises said retrieved process parameter values in chronological order over said selected training data time period.

35. The carrier medium of claim 34, wherein the program instructions are further executable to perform:

generating a second training set by;

removing at least a subset of the parameter values of said first training set, wherein said at least a subset of the parameter values comprises oldest parameter values of said first training set; and

adding new parameter values from said training data based on said timestamps to generate said second training set, wherein said second training set corresponds to a second time period in said chronology; and

training a support vector machine using said second training set.