Machine learning systems and methods

• US 7,389,277 B2
• Filed: 07/08/2005
• Issued: 06/17/2008
• Est. Priority Date: 06/15/2001
• Status: Expired due to Fees

• Current Assignee: Staywell Health Management LLC (Merck & Co., Inc.)
• Original Assignee: Medical Scientists, Inc. (Merck & Co., Inc.)
• Inventors: Snow, Kristin Kendall, Poteat, Harry Towsley, Chen, Hung-Han, Hunter, Lawrence
• Primary Examiner(s): Holmes; Michael B.
• Application Number:
  US11/177,200
• Publication Number:
  US 20050267850A1
• Time in Patent Office: 1,075 Days
• Field of Search: 706/12, 706/21, 706 15- 16
• US Class Current: 706/21
• CPC Class Codes: G06F 18/211 Selection of the most significant subset of features
• CPC Class Codes: G06N 20/00 Machine learning

Abstract

A method for using machine learning to solve problems having either a “positive” result (the event occurred) or a “negative” result (the event did not occur), in which the probability of a positive result is very low and the consequences of the positive result are significant. Training data is obtained and a subset of that data is distilled for application to a machine learning system. The training data includes some records corresponding to the positive result, some nearest neighbors from the records corresponding to the negative result, and some other records corresponding to the negative result. The machine learning system uses a co-evolution approach to obtain a rule set for predicting results after a number of cycles. The machine system uses a fitness function derived for use with the type of problem, such as a fitness function based on the sensitivity and positive predictive value of the rules. The rules are validated using the entire set of training data.

Claims

1. A computer-executable method for using machine learning to predict an outcome associated with a medical condition, the method comprising:

receiving training data including a plurality of records associating feature variables with outcome variables corresponding to at least one medical condition, wherein the training data comprises a first data set associated with a first outcome and comprises a second data set associated with a second outcome substantially less likely than the first outcome;

identifying within the first data set a third data set that consists essentially of nearby neighbors to the second data set;

using a plurality of software-based, computer-executable machine learners to develop from the first, second and third data sets at least one set of computer-executable rules usable to predict the first outcome or the second outcome;

processing a portion of the training data with a machine learning system, wherein the training data portion is recorded in a computer-readable medium, and wherein said processing includes;

selecting a first subset of the first data set,

selecting a second subset of the second data set, and

selecting a third subset of the third data set; and

wherein said using the plurality of software-based, computer-executable machine learners comprises developing from the first, second and third subsets the at least one set of computer-executable rules.

2. The method of claim 1, wherein selecting the second subset comprises selecting substantially all the data of the second set.

3. The method of claim 1, wherein the first subset, second subset and third subset are substantially the same size.

4. A computer-executable method for using machine learning to predict an outcome associated with a medical condition, the method comprising:

receiving training data including a plurality of records associating feature variables with outcome variables corresponding to at least one medical condition, wherein the training data comprises a first data set associated with a first outcome and comprises a second data set associated with a second outcome substantially less likely than the first outcome;

identifying within the first data set a third data set that consists essentially of nearby neighbors to the second data set;

using a plurality of software-based, computer-executable machine learners to develop from the first, second and third data sets at least one set of computer-executable rules usable to predict the first outcome or the second outcome; and

identifying the nearby neighbors in the first data set based on a proximity of data in the first data set to the second outcome.

5. A computer-executable method for using machine learning to predict an outcome associated with a medical condition, the method comprising:

receiving training data including a plurality of records associating feature variables with outcome variables corresponding to at least one medical condition, wherein the training data comprises a first data set associated with a first outcome and comprises a second data set associated with a second outcome substantially less likely than the first outcome;

identifying within the first data set a third data set that consists essentially of nearby neighbors to the second data set;

using a plurality of software-based, computer-executable machine learners to develop from the first, second and third data sets at least one set of computer-executable rules usable to predict the first outcome or the second outcome; and

identifying the nearby neighbors in the first data set based on a proximity of data in the first data set to the feature variables of data in the second data set.

6. A computer-executable method for using machine learning to predict an outcome associated with a medical condition, the method comprising:

receiving training data including a plurality of records associating feature variables with outcome variables corresponding to at least one medical condition, wherein the training data comprises a first data set associated with a first outcome and comprises a second data set associated with a second outcome substantially less likely than the first outcome;

identifying within the first data set a third data set that consists essentially of nearby neighbors to the second data set;

using a plurality of software-based, computer-executable machine learners to develop from the first, second and third data sets at least one set of computer-executable rules usable to predict the first outcome or the second outcome; and

validating the at least one set of rules using substantially all the training data.

7. The method of claim 6, wherein said validating includes obtaining at least one accuracy measure for the at least one set of rules.

8. The method of claim 7, wherein obtaining the at least one accuracy measure includes obtaining at least one of a positive predictive value and a sensitivity of the at least one set of rules.

9. A computer-executable method for using machine learning to predict an outcome associated with a medical condition, the method comprising:

receiving training data including a plurality of records associating feature variables with outcome variables corresponding to at least one medical condition, wherein the training data comprises a first data set associated with a first outcome and comprises a second data set associated with a second outcome substantially less likely than the first outcome;

identifying within the first data set a third data set that consists essentially of nearby neighbors to the second data set; and

using a plurality of software-based, computer-executable machine learners to develop from the first, second and third data sets at least one set of computer-executable rules usable to predict the first outcome or the second outcome, wherein using the plurality of software-based, computer-executable machine learners further includes developing a set of interim rules using the plurality of software-based, computer-executable machine learners, evaluating the set of interim rules, and developing a revised set of interim rules using the results of the evaluating act.

10. The method of claim 9, wherein evaluating the set of interim rules includes applying a user-selectable fitness function.

11. The method of claim 9, wherein evaluating the set of interim rules includes applying a fitness function based on at least one of a sensitivity, a positive predictive value, and a correlation coefficient of the interim rules.

12. A system for using machine learning to predict an outcome associated with a medical condition, the system comprising:

medical data including a plurality of records associating feature variables with outcome variables, wherein the medical data comprises a first data set associated with a first outcome and comprises a second data set associated with a second outcome substantially less likely than the first outcome, and wherein the feature variables include demographic data;

a processing module configured to identify within the first data set a third data set that consists essentially of nearby neighbors to the second data set; and

a plurality of machine learners configured to develop from the first, second and third data sets at least one set of computer-executable rules usable to predict the first outcome or the second outcome.

13. The system of claim 12, wherein the processing module is further configured to select a first subset of the first data set, select a second subset of the second data set, and select a third subset of the third data set, and wherein said plurality of machine learners is configured to develop from the first, second and third subsets the at least one set of computer-executable rules.

14. The system of claim 12, further comprising a computer readable medium configured to store at least a portion of the medical data.

15. The system of claim 12, wherein the first outcome is associated with a first range of medical costs less than a cost threshold, and wherein the second outcome is associated with a second range of medical costs at least as great as the cost threshold.

16. The system of claim 12, wherein the first outcome is at least thirty times more likely than the second outcome.

17. A computer system for using machine learning to predict an outcome associated with a medical condition, the computer system comprising:

means for storing training data including a plurality of records associating feature variables with outcome variables corresponding to at least one medical condition, wherein the training data comprises a first data set associated with a first outcome and comprises a second data set associated with a second outcome substantially less likely than the first outcome;

means for identifying within the first data set a third data set that consists essentially of nearby neighbors to the second data set;

means for developing from the first, second and third data sets at least one set of computer-executable rules usable to predict the first outcome or the second outcome; and

means for processing a portion of the training data, wherein said means for processing is configured to select a first subset of the first data set, select a second subset of the second data set, and select a third subset of the third data set, and wherein said means for developing is configured to use the first, second and third subsets to develop the at least one set of computer-executable rules.