Pre-processing input data with outlier values for a support vector machine

• US 6,941,301 B2
• Filed: 01/18/2002
• Issued: 09/06/2005
• Est. Priority Date: 01/18/2002
• Status: Expired due to Term

• Current Assignee: IP3 2017, Series 200 of Allied Security Trust I (Allied Security Trust)
• Original Assignee: Pavilion Technologies Incorporated (Rockwell Automation, Inc.)
• Inventors: Ferguson, Bruce, Hartman, Eric
• Primary Examiner(s): Rones, Charles
• Assistant Examiner(s): ABEL JALIL, NEVEEN
• Application Number:
  US10/051,266
• Publication Number:
  US 20030140039A1
• Time in Patent Office: 1,327 Days
• Field of Search: 707 1-100, 707/102, 707/104.1, 706/16, 706 19- 22, 706 25- 26, 703/2
• US Class Current: 1/1
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06F 2218/02 Preprocessing
• CPC Class Codes: Y10S 707/99934 Query formulation, input preparation, or translation
• CPC Class Codes: Y10S 707/99936 Pattern matching access
• CPC Class Codes: Y10S 707/99943 Generating database or data structure, e.g. via user interface

Abstract

A system and method for preprocessing input data to a support vector machine (SVM). The SVM is a system model having parameters that define the representation of the system being modeled, and operates in two modes: run-time and training. A data preprocessor preprocesses received data in accordance with predetermined preprocessing parameters, and outputs preprocessed data. The data preprocessor includes an input buffer for receiving and storing the input data. The input data may include one or more outlier values. A data filter detects and removes any outlier values in the input data, generating corrected input data. The filter may optionally replace the outlier values in the input data. An output device outputs the corrected data from the data filter as preprocessed data. The corrected data may be input to the SVM in training mode to train the SVM, and/or in run-time mode to generate control parameters and/or predictive output information.

Claims

1. A data preprocessor for preprocessing input data for a support vector machine, wherein the input data include one or more outlier values, comprising:

an input buffer which is operable to receive and store the input data wherein the input data comprise run-time data;

a data filter which is operable to detect and remove said one or more outlier values, thereby generating corrected input data, wherein said corrected input data comprise corrected run-time data; and

an output device for outputting the corrected input data, said corrected input data comprising the input data to the support vector machine;

wherein the support vector machine comprises a non-linear model having a set of model parameters defining a representation of a system, wherein said model parameters of said support vector machine have been trained to represent said system; and

wherein the support vector machine is operable to receive said corrected run-time data and generate run-time output data, wherein said run-time output data comprise one or both of control parameters for said system and predictive output information for said system.

2. The data preprocessor of claim 1, wherein the support vector machine comprises a non-linear model having a set of model parameters defining a representation of a system, said model parameters capable of being trained;

wherein the input data comprise training data, wherein said corrected data comprise corrected training data including corrected target input data and corrected target output data; and

wherein the support vector machine is operable to be trained according to a predetermined training algorithm applied to said corrected target input data and said corrected target output data to develop model parameter values such that said support vector machine has stored therein a representation of the system that generated the target output data in response to the corrected target input data.

3. The data preprocessor of claim 1, wherein said control parameters are usable to determine control inputs to said system for run-time operation of said system.

4. The data preprocessor of claim 1, wherein the data filter is further operable to replace said one or more outlier values with replacement values, wherein said corrected input includes said replacement values.

5. The data preprocessor of claim 4, wherein the data filter is operable to replace said one or more outlier values using one or more of clipping, interpolation, extrapolation, spline fit, and sample/hold of a last prior value.

6. The data preprocessor of claim 1, further comprising:

a graphical user interface (GUI) which is operable to receive user input specifying one or more data filtering operations to be performed on said input data, wherein said one or more data filtering operations operate to remove and/or replace said one or more outlier values.

7. The data preprocessor of claim 6, wherein said GUI is further operable to display said input data prior to and after performing said filtering operations on said input data.

8. The data preprocessor of claim 6, wherein said GUI is further operable to receive user input specifying a portion of said input data for said data filtering operations.

9. The data preprocessor of claim 1, wherein the input data comprise a plurality of variables, each of the variables comprising an input variable with an associated set of data wherein each of said variables comprises an input to said input buffer;

and

wherein each of at least a subset of said variables comprises a corresponding one of the inputs to the support vector machine.

10. A system for preprocessing input data for a support vector machine having multiple inputs, each of the inputs associated with a portion of the input data, wherein the input data include one or more outlier values, comprising:

means for receiving and storing the input data, wherein the input data comprise run-time data;

means for analyzing said input data to determine said one or more outliers values;

means for removing said one or more outlier values, thereby generating corrected input data, wherein said corrected input data comprise corrected run-time data; and

means for outputting the corrected input data, said corrected input data comprising the input data to the support vector machine;

wherein the support vector machine comprises a non-linear model having a set of model parameters defining a representation of a system, wherein said model parameters of said support vector machine have been trained to represent said system, the system further comprising;

means for inputting said corrected run-time data into the support vector machine to generate run-time output data, wherein said run-time output data comprise one or both of control parameters for said system and predictive output information for said system.

11. The system of claim 10, wherein the support vector machine comprises a non-linear model having a set of model parameters defining a representation of a system, said model parameters capable of being trained;

and

wherein the input data comprise training data including target input data and target output data, wherein said corrected data comprise corrected training data including corrected target input data and corrected target output data;

the system further comprising;

means for training the support vector machine according to a predetermined training algorithm applied to said corrected target input data and said corrected target output data to develop model parameter values such that said support vector machine has stored therein a representation of the system that generated the target output data in response to the target input data.

12. The system of claim 10, wherein said control parameters are usable to determine control inputs to said system for run-time operation of said system.

13. The system of claim 10, further comprising:

means for replacing said one or more outlier values with replacement values, wherein said corrected input includes said replacement values.

14. The system of claim 13, wherein said replacing said one or more outlier values is performed using one or more of clipping, interpolation, extrapolation, spline fit, and sample/hold of a last prior value.

15. The system of claim 10, further comprising:

means for receiving user input specifying one or more data filtering operations to be performed on said input data, wherein said analyzing and said removing said one or more outlier values comprises performing said one or more data filtering operations on the input data.

16. The system of claim 10, further comprising:

means for displaying said input data prior to and after performing said filtering operations on said input data.

17. The system of claim 10, further comprising:

means for receiving user input specifying a portion of said input data for said data filtering operations.

18. The system of claim 10, wherein the input data comprise a plurality of variables, each of the variables comprising an input variable with an associated set of data wherein each of said variables comprises an input to said input buffer;

and

wherein each of at least a subset of said variables comprises a corresponding one of the inputs to the support vector machine.

19. A carrier medium which stores program instructions for preprocessing input data prior to input to a support vector machine having multiple inputs, each of the inputs associated with a portion of the input data, wherein the input data include one or more outlier values, wherein said program instructions are executable to:

receive and store the input data, wherein the input data comprise run-time data;

analyze said input data to determine said one or more outliers values;

remove said one or more outlier values, thereby generating corrected input data2 wherein said corrected input data comprise corrected run-time data; and

output the corrected input data, said corrected input data comprising the input data to the support vector machine, wherein the support vector machine comprises a non-linear model having a set of model parameters defining a representation of a system, wherein said model parameters of said support vector machine have been trained to represent said system, wherein said program instructions are further executable to;

input said corrected run-time data into the support vector machine to generate run-time output data, wherein said run-time output data comprise one or both of control parameters for said system and predictive output information for said system.

20. The carrier medium of claim 19, wherein the support vector machine comprises a non-linear model having a set of model parameters defining a representation of a system, said model parameters capable of being trained;

and

wherein the input data comprise training data including target input data and target output data, wherein said corrected data comprise corrected training data including corrected target input data and corrected target output data;

wherein said program instructions are further executable to;

train the support vector machine according to a predetermined training algorithm applied to said corrected target input data and said corrected target output data to develop model parameter values such that said support vector machine has stored therein a representation of the system that generated the target output data in response to the target input data.

21. The carrier medium of claim 19, wherein said control parameters are usable to determine control inputs to said system for run-time operation of said system.

22. The carrier medium of claim 19, wherein said program instructions are further executable to:

replace said one or more outlier values with replacement values, wherein said corrected input includes said replacement values.

23. The carrier medium of claim 19, wherein said replacing said one or more outlier values is performed using one or more of clipping, interpolation, extrapolation, spline fit, and sample/hold of a last prior value.

24. The carrier medium of claim 19, wherein said program instructions are further executable to:

receive user input specifying one or more data filtering operations to be performed on said input data, wherein said analyzing and said removing said one or more outlier values comprises performing said one or more data filtering operations on the input data.

25. The carrier medium of claim 19, wherein said program instructions are further executable to:

display said input data prior to and after performing said filtering operations on said input data.

26. The carrier medium of claim 19, wherein said program instructions are further executable to:

receive user input specifying a portion of said input data for said data filtering operations.

27. The carrier medium of claim 26, wherein the input data comprise a plurality of variables, each of the variables comprising an input variable with an associated set of data wherein each of said variables comprises an input to said input buffer;

and

wherein each of at least a subset of said variables comprises a corresponding one of the inputs to the support vector machine.

28. A method for preprocessing input data prior to input to a support vector machine having multiple inputs, each of the inputs associated with a portion of the input data, wherein the input data include one or more outlier values, the method comprising:

receiving and storing the input data wherein the input data comprise run-time data;

analyzing said input data to determine said one or more outliers values;

removing said one or more outlier values, thereby generating corrected input data1 wherein said corrected input data comprise corrected run-time data; and

outputting the corrected input data, said corrected input data comprising the input data to the support vector machine;

wherein the support vector machine comprises a non-linear model having a set of model parameters defining a representation of a system, wherein said model parameters of said support vector machine have been trained to represent said system, the method further comprising;

inputting said corrected run-time data into the support vector machine to generate run-time output data, wherein said run-time output data comprise one or both of control parameters for said system and predictive output information for said system.

29. The method of claim 28, wherein the support vector machine comprises a non-linear model having a set of model parameters defining a representation of a system, said model parameters capable of being trained;

and

wherein the input data comprise training data including target input data and target output data, wherein said corrected data comprise corrected training data including corrected target input data and corrected target output data;

the method further comprising;

training the support vector machine according to a predetermined training algorithm applied to said corrected target input data and said corrected target output data to develop model parameter values such that said support vector machine has stored therein a representation of the system that generated the target output data in response to the target input data.

30. The method of claim 28, wherein said control parameters are usable to determine control inputs to said system for run-time operation of said system.

31. The method of claim 28, further comprising:

replacing said one or more outlier values with replacement values, wherein said corrected input includes said replacement values.

32. The method of claim 31, wherein said replacing said one or more outlier values is performed using one or more of clipping, interpolation, extrapolation, spine fit, and sample/hold of a last prior value.

33. The method of claim 28, further comprising:

receiving user input specifying one or more data filtering operations to be performed on said input data, wherein said analyzing and said removing said one or more outlier values comprises performing said one or more data filtering operations on the input data.

34. The method of claim 28, further comprising:

displaying said input data prior to and after performing said filtering operations on said input data.

35. The method of claim 28, further comprising:

receiving user input specifying a portion of said input data for said data filtering operations.

36. The method of claim 28, wherein the input data comprise a plurality of variables, each of the variables comprising an input variable with an associated set of data wherein each of said variables comprises an input to said input buffer;

and

wherein each of at least a subset of said variables comprises a corresponding one of the inputs to the support vector machine.