Enhancing knowledge discovery using support vector machines in a distributed network environment

• US 6,157,921 A
• Filed: 05/01/1999
• Issued: 12/05/2000
• Est. Priority Date: 05/01/1998
• Status: Expired due to Term

• Current Assignee: Curtis Anderson, Health Discovery Corporation, James Roberts, Joe Mckenzie, Jules B. Paderewski, Julian N. Stern, Memorial Health Systems Incorporated, Timothy P. O'Hayer
• Original Assignee: Barnhill Technologies, LLC
• Inventors: Barnhill, Stephen D.
• Primary Examiner(s): Hafiz, Tariq R.
• Assistant Examiner(s): DAVIS, GEORGE B
• Application Number:
  US09/305,345
• Time in Patent Office: 584 Days
• Field of Search: 706/45, 706/20, 706/25, 706/19, 706/16, 705/26, 702/153
• US Class Current: 706/16
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06F 18/2411 based on the proximity to a decision surface, e.g. support vector machines
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 20/10 using kernel methods, e.g. support vector machines [SVM]

Abstract

A system and method for enhancing knowledge discovery from data using a learning machine in general and a support vector machine in particular in a distributed network environment. A customer may transmit training data, test data and live data to a vendor'"'"'s server from a remote source, via a distributed network. The customer may also transmit to the server identification information such as a user name, a password and a financial account identifier. The training data, test data and live data may be stored in a storage device. Training data may then be pre-processed in order to add meaning thereto. Pre-processing data may involve transforming the data points and/or expanding the data points. By adding meaning to the data, the learning machine is provided with a greater amount of information for processing. With regard to support vector machines in particular, the greater the amount of information that is processed, the better generalizations about the data that may be derived. The learning machine is therefore trained with the pre-processed training data and is tested with test data that is pre-processed in the same manner. The test output from the learning machine is post-processed in order to determine if the knowledge discovered from the test data is desirable. Post-processing involves interpreting the test output into a format that may be compared with the test data. Live data is pre-processed and input into the trained and tested learning machine. The live output from the learning machine may then be post-processed into a computationally derived alphanumerical classifier for interpretation by a human or computer automated process. Prior to transmitting the alpha numerical classifier to the customer via the distributed network, the server is operable to communicate with a financial institution for the purpose of receiving funds from a financial account of the customer identified by the financial account identifier.

Claims

1. A system for enhancing knowledge discovery using a support vector machine comprising:

a server in communication with a distributed network for receiving a training data set, a test data set, a live data set and a financial account identifier from a remote source, the remote source also in communication with the distributed network;

one or more storage devices in communication with the server for storing the training data set and the test data set;

a processor for executing a support vector machine;

the processor further operable for;

collecting the training data set from the one or more storage devices,

pre-processing the training data set to add dimensionality to each of a plurality of training data points,

inputting the pre-processed training data set into the support vector machine so as to train the support vector machine,

in response to training of the support vector machine, collecting the test data set from the one or more storage devices,

pre-processing the test data set to add dimensionality to each of a plurality of test data points,

inputting the test data set into the trained support vector machine in order to test the support vector machine,

in response to receiving a test output from the trained support vector machine, collecting the live data set,

inputting the live data set into the tested and trained support vector machine in order to process the live data,

in response to receiving a live output from the support vector machine, post-processing the live output to derive a computationally based alphanumerical classifier, and

transmitting the alphanumerical classifier to the server;

wherein the server is further operable for;

communicating with a financial institution in order to receive funds from a financial account identified by the financial account identifier, and

in response to receiving the funds, transmitting the alphanumerical classifier to the remote source or another remote source.

2. The system of claim 1,

wherein pre-processing the training data set further comprises:

determining that at least one of the training data points is dirty; and

in response to determining that one of the training data points is dirty, cleaning the dirty training data point.

3. The system of claim 2, wherein each training data point comprises a vector having one or more coordinates;

and

wherein cleaning the training data point comprises deleting, repairing or replacing one or more of the coordinates of the data point.

4. The system of claim 1, wherein each training data point comprises a vector having one or more original coordinates;

and

wherein pre-processing the training data set comprises adding one or more new coordinates to the vector.

5. The system of claim 4, wherein the one or more new coordinates added to the vector are derived by applying a transformation to one or more of the original coordinates.

6. The system of claim 5, wherein the transformation is based on expert knowledge.

7. The system of claim 5, wherein the transformation is computationally derived.

8. The system of claim 5, wherein the training data set comprises a continuous variable;

and

wherein the transformation comprises optimally categorizing the continuous variable of the training data set.

9. The system of claim 1, wherein the knowledge to be discovered from the data relates to a regression or density estimation;

wherein the support vector machine produces a training output comprising a continuous variable; and

wherein the processor is further operable for post-processing the training output by optimally categorizing the training output to derive cutoff points in the continuous variable.

10. The system of claim 1, further comprising a plurality of additional processors for executing a plurality of additional support vector machines, each support vector machine comprising a different kernel;

and

wherein one of the processors is further operable for;

receiving test output from each of the support vector machines,

comparing each of the test outputs with each other to determine that none of the test outputs is the optimal solution;

adjusting the different kernels of one or more of the plurality of support vector machines; and

in response to adjusting the different kernels, retraining and retesting each of the plurality of support vector machines.

14. The system of claim 1, wherein each training data point comprises a vector having one or more original coordinates;

and

wherein pre-processing the training data set comprises adding one or more new coordinates to the vector.

15. The system of claim 14, wherein the one or more new coordinates added to the vector are derived by applying a transformation to one or more of the original coordinates.

16. The system of claim 15, wherein the transformation is based on expert knowledge.

17. The system of claim 15, wherein the transformation is computationally derived.

11. A system for enhancing the discovery of knowledge relating to a regression or density estimation using a support vector machine comprising:

a server in communication with a distributed network for receiving a training data set, a test data set, a live data set and a financial account identifier from a remote source, the remote source also in communication with the distributed network;

one or more storage devices in communication with the server for storing the training data set and the test data set;

a processor for executing a support vector machine;

the processor further operable for;

collecting the training data set from the one or more storage devices,

pre-processing the training data set to alter dimensionality to each of a plurality of training data points,

inputting the pre-processed training data set into the support vector machine so as to train the support vector machine and to produces a training output comprising a continuous variable,

post-processing the training output by optimally categorizing the training output to derive cutoff points in the continuous variable,

in response to post-processing the training output, collecting the test data set from the one or more storage devices,

pre-processing the test data set to alter dimensionality to each of a plurality of test data points,

inputting the test data set into the trained support vector machine in order to test the support vector machine,

in response to receiving a test output from the trained support vector machine, collecting the live data set,

inputting the live data set into the tested and trained support vector machine in order to process the live data,

in response to receiving a live output from the support vector machine, post-processing the live output to derive a computationally based alphanumerical classifier, and

transmitting the alphanumerical classifier to the server;

wherein the server is further operable for;

communicating with a financial institution in order to receive funds from a financial account identified by the financial account identifier, and

in response to receiving the funds, transmitting the alphanumerical classifier to the remote source or another remote source.

12. The system of claim 11, wherein pre-processing the training data set comprises:

determining that at least one of the training data points is dirty; and

in response to determining that one of the training data points is dirty, cleaning the dirty training data point.

13. The system of claim 12, wherein each training data point comprises a vector having one or more coordinates;

and

wherein cleaning the training data point comprises deleting, repairing or replacing one or more of the coordinates of the data point.

18. The system of claim 11, further comprising a plurality of additional processors for executing a plurality of additional support vector machines, each support vector machine comprising a different kernel;

and

wherein one of the processors is further operable for;

receiving test output from each of the support vector machines,

comparing each of the test outputs with each other to determine that none of the test outputs is the optimal solution;

adjusting the different kernels of one or more of the plurality of support vector machines; and

in response to adjusting the different kernels, retraining and retesting each of the plurality of support vector machines.

19. A system for enhancing knowledge discovery using multiple support vector machine comprising:

a server in communication with a distributed network for receiving a training data set, a test data set, a live data set and a financial account identifier from a remote source, the remote source also in communication with the distributed network;

one or more storage devices in communication with the server for storing the training data set and the test data set;

a plurality of processors for executing a plurality of support vector machines, each support vector machine comprising a different kernel;

each processor further operable for;

collecting the training data set from the one or more storage devices,

pre-processing the training data set to alter dimensionality to each of a plurality of training data points,

inputting the pre-processed training data set into the support vector machine so as to train the support vector machine,

in response to training of the support vector machine, collecting the test data set from the one or more storage devices,

pre-processing the test data set to alter dimensionality to each of a plurality of test data points, and

inputting the test data set into the trained support vector machine in order to test the support vector machine;

wherein one of the processors is further operable for;

receiving test output from each of the support vector machines,

comparing each of the test outputs with each other to determine that none of the test outputs is the optimal solution,

adjusting the different kernels of one or more of the plurality of support vector machines,

in response to adjusting the different kernels, retraining and retesting each of the plurality of support vector machines until the optimal solution is reached, the optimal solution indicating that one of the support vector machines is an optimal support vector machine comprising an optimal kernel;

wherein the processor of the optimal support vector machine is further operable for;

collecting and processing the live data set in order to produce a live output,

post-processing the live output to derive a computationally based alphanumerical classifier, and

transmitting the alphanumerical classifier to the server; and

wherein the server is further operable for;

communicating with a financial institution in order to receive funds from a financial account identified by the financial account identifier, and

in response to receiving the funds, transmitting the alphanumerical classifier to the remote source or another remote source.

20. The system of claim 19, wherein pre-processing the training data set comprises:

determining that at least one of the training data points is dirty; and

in response to determining that one of the training data points is dirty, cleaning the dirty training data point.

21. The system of claim 20, wherein each training data point comprises a vector having one or more coordinates;

and

wherein cleaning the training data point comprises deleting, repairing or replacing one or more of the coordinates of the data point.

22. The system of claim 19, wherein each training data point comprises a vector having one or more original coordinates;

and

wherein pre-processing the training data set comprises adding one or more new coordinates to the vector.

23. The system of claim 22, wherein the one or more new coordinates added to the vector are derived by applying a transformation to one or more of the original coordinates.

24. The system of claim 23, wherein the transformation is based on expert knowledge.

25. The system of claim 23, wherein the transformation is computationally derived.

26. The system of claim 23, wherein the training data set comprises a continuous variable;

and

wherein the transformation comprises optimally categorizing the continuous variable of the training data set.

27. The system of claim 19, wherein the knowledge to be discovered from the data relates to a regression or density estimation;

wherein each of the support vector machines produces a training output comprising a continuous variable; and

wherein each of the processors is further operable for post-processing the training output by optimally categorizing the training output to derive cutoff points in the continuous variable.