Method for generating healthcare-related validated prediction models from multiple sources

• US 9,934,361 B2
• Filed: 09/29/2012
• Issued: 04/03/2018
• Est. Priority Date: 09/30/2011
• Status: Active Grant

• Current Assignee: Univfy Inc.
• Original Assignee: Univfy Inc.
• Inventors: Yao, Mylene, Wong, Wing H., Choi, Bokyung
• Primary Examiner(s): Kanaan, Maroun
• Application Number:
  US13/631,988
• Publication Number:
  US 20130085773A1
• Time in Patent Office: 2,012 Days
• Field of Search: 705 2, 705 3
• CPC Class Codes: F04C 2270/041 Controlled or regulated
• CPC Class Codes: G06Q 10/04 Forecasting or optimisation specially adapted for administrative or management purposes, e.g. linear programming or "cutting stock problem" market predictions or forecasting for commercial activities G06Q30/0202
• CPC Class Codes: G16H 10/60 for patient-specific data, e.g. for electronic patient records
• CPC Class Codes: G16H 50/20 for computer-aided diagnosis, e.g. based on medical expert systems
• CPC Class Codes: G16H 50/50 for simulation or modelling of medical disorders
• CPC Class Codes: G16H 50/70 for mining of medical data, e.g. analysing previous cases of other patients
• CPC Class Codes: G16H 70/20 relating to practices or guidelines

Abstract

Provided is a method for generating prediction models from multiple healthcare centers. The method allows a third party to use data sets from multiple sources to build prediction models. By entering the data sets in a Model Deconstruction and Transfer (MDT) platform, a healthcare center may provide data to a third party without the need to de-identify data or to physically transfer any identifying or de-identified data from the healthcare center. The MDT platform includes a variable library, which allows the healthcare center to select variables that will be used to generate and validate the prediction model. Also provided is a method for compensating sources that contribute data sets based upon the percentage of clinical data that is used to generate a prediction model.

Claims

1. A system for generating a prediction model for a health outcome of interest comprising:

(a) a plurality of electronic storage media, each comprising a Model Deconstruction and Transfer platform (MDT), wherein each electronic storage media configured for receiving data from only one of a plurality of healthcare centers, wherein the MDT is comprised of a variable library (VL) comprised of variables relevant to a health outcome of interest, wherein upon completion of data entry by a healthcare center, the MDT generates a first prediction model (PM₀) for the health outcome of interest, wherein the PM₀ represents a statistical relationship among the variables that are predictive of the health outcome of interest, and further wherein upon generation of the PM₀, the MDT automatically destroys the data entered into the electronic storage media; and

(b) a server configured to receive the MDT and deconstruct the PM₀ from the electronic storage media received from each of the plurality of healthcare centers, wherein the server is configured to perform the following functions,

(i) combine the PM₀ from the plurality of healthcare centers;

(ii) generate a Model Component Library, MCL from each PM₀, wherein the MCL comprises components that result from deconstruction of each PM₀,

(iii) generate at least one second prediction model, PM₁, from the MCL based upon differential weighting of the MCL components relevant to the health outcome of interest, wherein the at least one PM₁ provides a prediction of the health outcome of interest for individuals in a broad population, wherein the broad population is represented by the data entered into the MDT by the plurality of healthcare centers;

wherein the system expands predictive scope of a healthcare center that does not have sufficient data to develop its own predictions for a health outcome of interest.

2. The system of claim 1, wherein upon completion of the data entry and the generation of the PM₀, the MDT automatically destroys the entered data.

3. The system of claim 1, wherein the health outcome of interest is selected from the group consisting of poor treatment response, worsening of a condition, rate of disease progression, probability of success following an interventional procedure, probability of failure following an interventional procedure, probability of complications following an interventional procedure, readmission to emergency services for a specific condition, readmission to a hospital for a specific condition, development of co-morbidities, and combinations of any of the foregoing.

4. The system of claim 1, wherein the MCL components are selected from the group consisting of patient demographics, patient insurance policy status, patient billing data, patient family history, patient laboratory results, patient imaging results, patient pathology results (tissue biopsies), patient immunopathology results, patient cytology results, patient cytogenetic results, patient gene expression, patient metabolic panels, patient metabolic profiles, patient genomic data, patient response to procedures, patient medications, patient therapeutic regimens, patient acute care parameters, patient ambulatory monitoring;

frequency of outpatient visits, frequency of emergency visits, frequency of hospital admissions, frequency of hospital re-admissions;

quality of life assessments;

clinical characteristics (clinical signs and symptoms), clinical diagnoses, clinic inpatient procedures, clinic outpatient procedures;

monitoring for cancer recurrence, psychiatric assessments; and

CPT (current procedural technology) diagnostic codes.

5. The system of claim 1, wherein the PM₀ and the at least one PM₁ are calculated using a statistical techniques selected from the group consisting of machine learning, logistic regression, linear regression, non-linear regression, and combinations of any of the foregoing.

6. The system of claim 5, wherein the machine learning techniques are selected from the group consisting of classification tree methods, LASSO (least absolute shrinkage and selection operator), Bayesian network modeling, and combinations of any of the foregoing.

7. The system of claim 6, wherein the classification tree method is a boosted tree method.

8. The system of claim 1, wherein the PM₀ is validated by a test data set originating from one or more of the healthcare centers.

9. The system of claim 8, wherein performance of the validated PM₀ is measured by comparison against performance of a control model.

10. The system of claim 1, wherein the at least one PM₁ is validated by a test data set originating from one or more of the healthcare centers.

11. The system of claim 10, wherein performance of the validated PM₀ is measured by comparison against performance of a control model.

12. The system of claim 9, wherein the comparison is carried out by applying statistical criteria selected from the group consisting of predictive power based on posterior log likelihoods, predictive power based upon posterior probability of an event, predictive power based upon estimated errors of prediction, dynamic range of predicted probabilities, reclassification, frequency of predicted probabilities, cross-validation error, number of model components required, ease of implementation, computing time, discrimination, calibration, and reclassification.

13. The system of claim 11, wherein the comparison is carried out by applying statistical criteria selected from the group consisting of predictive power based on posterior log likelihoods, predictive power based upon posterior probability of an event, predictive power based upon estimated errors of prediction, dynamic range of predicted probabilities, reclassification, frequency of predicted probabilities, cross-validation error, number of model components required, ease of implementation, computing time, discrimination, calibration, and reclassification.

14. The system of claim 1, wherein the at least one PM₁ is generated from the MCL by using individual MCL components as variables in the at least one PM₁ with differential weighting.

15. The system of claim 14, wherein a single MCL component may be used more than once in a single PM₁.

16. The system of claim 15, wherein each use of a single MCL component is subject to a different statistical function.

17. The system of claim 1, wherein one of the plurality of healthcare centers provides an additional data set comprising healthcare data for a group of patients that are not represented in the MCL, wherein the additional data set and is non-redundant and unrelated to the data used to generate the MCL.

18. The system of claim 1, wherein one of the plurality of healthcare centers provides an additional data set comprising healthcare data for a group of patients that are represented in the MCL, wherein the additional data set is non-redundant and unrelated to the data used to generate the MCL.

19. The system of claim 17, wherein the additional data set comprises a training set.

20. The system of claim 18, wherein the additional data set comprises a training set.

21. The system of claim 17, wherein the additional data set comprises a test set.

22. The system of claim 18, wherein the additional data set comprises a test set.

23. The system of claim 17, wherein the additional data set comprises a combination of a test set and a training set.

24. The system of claim 18, wherein the additional data set comprises a combination of a test set and a training set.

25. The system of claim 23, wherein the additional data set is divided into training and test sets chronologically according to the year of origin of the healthcare data.

26. The system of claim 24, wherein the additional data set is divided into training and test sets chronologically according to the year of origin of the healthcare data.

27. The system of claim 19, wherein the at least one PM₁ comprises MCL components that are selected based upon the performance of the at least one PM₁ to predict probabilities of the healthcare outcome in the training set.

28. The system of claim 20, wherein the at least one PM₁ comprises MCL components that are selected based upon the performance of the at least one PM₁ to predict probabilities of the healthcare outcome in the training set.

29. The system of claim 21, wherein the at least one PM₁ comprises MCL components that are selected based upon the performance of the at least one PM₁ to predict probabilities of the healthcare outcome in the test set.

30. The system of claim 22, wherein the at least one PM₁ comprises MCL components that are selected based upon the performance of the at least one PM₁ to predict probabilities of the healthcare outcome in the test set.

31. The system of claim 1, wherein at least two PM₁ are generated, ranked, and selected according to model performance parameters selected from the group consisting of posterior log likelihoods, predictive power based upon posterior probability of an event, predictive power based upon estimated errors of prediction, dynamic range of predicted probabilities, reclassification, frequency of predicted probabilities, cross-validation error, number of model components required, and discrimination.

32. A system for compensating a healthcare center according to claim 1, comprising compensating the healthcare center based upon contribution to the at least one PM₁ made by the healthcare center.

33. The system of claim 1, wherein the VL comprises variables selected from the group consisting of patient demographics, patient insurance policy status, patient billing data, patient family history, patient laboratory results, patient imaging results, patient pathology results, patient immunopathology results, patient cytology results, patient cytogenetic results, patient gene expression, patient metabolic panels, patient metabolic profiles, patient genomic data, patient response to procedures, patient medications, patient therapeutic regimens, patient acute care parameters, patient ambulatory monitoring;

frequency of outpatient visits, frequency of emergency visits, frequency of hospital admissions, frequency of hospital re-admissions;

quality of life assessments;

clinical characteristics, clinical diagnoses, clinic inpatient procedures, clinic outpatient procedures;

monitoring for cancer recurrence, psychiatric assessments; and

current procedural technology (CPT) diagnostic codes.

34. The system of claim 32, wherein the healthcare center'"'"'s compensation is based upon factors selected from the group consisting of size or existence of the healthcare center'"'"'s data, amount or variety of variables selected by the healthcare center, amount and/or quality of data missing from the data provided by the healthcare center, and MCL components derived from the PM₀ of the healthcare center that contribute to the generation of the at least one PM₁.

35. The system of claim 32, wherein the healthcare center is compensated according to a percentage of revenue share based upon contribution of MCL components derived from the PM₀ that are instrumental to performance of the at least one PM₁.

36. The system of claim 32, wherein the healthcare center is compensated according to a fixed payment per measure of MCL components used in the at least one PM₁.