Method and system for adaptively imputing sparse and missing data for predictive models

• US 10,409,789 B2
• Filed: 09/18/2017
• Issued: 09/10/2019
• Est. Priority Date: 09/16/2016
• Status: Active Grant

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: Oracle International Corporation (Oracle Corporation)
• Inventors: Zoll, Michael, Suleiman, Yaser I., Basu, Subhransu, Pruscino, Angelo, Lohwasser, Wolfgang, Miyoshi, Wataru, Breidt, Thomas, Herter, Thomas, Thielen, Klaus, Kumar, Sahil
• Primary Examiner(s): Dwivedi, Mahesh H
• Application Number:
  US15/707,500
• Publication Number:
  US 20180081914A1
• Time in Patent Office: 722 Days
• CPC Class Codes: G06F 11/3409 for performance assessment
• CPC Class Codes: G06F 11/3452 Performance evaluation by statistical analysis
• CPC Class Codes: G06F 16/215 Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 20/20 Ensemble learning
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks
• CPC Class Codes: H04L 41/069 using logs of notifications; Post-processing of notifications
• CPC Class Codes: H04L 41/0695 the faulty arrangement being the maintenance, administration or management system
• CPC Class Codes: H04L 41/0823 characterised by the purposes of a change of settings, e.g. optimising configuration for enhancing reliability for optimising operational conditions of wireless networks H04W24/02
• CPC Class Codes: H04L 41/142 using statistical or mathematical methods
• CPC Class Codes: H04L 43/02 Capturing of monitoring data

Abstract

Described is an approach that provides an adaptive solution to missing data for machine learning systems. A gradient solution is provided that is attentive to imputation needs at each of several missingness levels. This multilevel approach treats data missingness at any of multiple severity levels while utilizing, as much as possible, the actual observed data.

Claims

1. A method for imputing data for a learning system, comprising:

collecting data from a monitored target system;

determining one or more levels of missingness for the data collected from the monitored target system;

selecting, from among a plurality of imputation techniques, a selected imputation technique based at least in part upon the one or more levels of missingness for the data, wherein expectation maximization (EM) is selected as the selected imputation technique if it is determined that both an overall level of missing data and individual levels of missing data for signals are at one or more designated thresholds, and an external data source is accessed to generate an EM seed for the expectation maximization when insufficient seed data exists within the data collected from the monitored target system;

imputing missing data using the selected imputation technique to generate training data; and

performing model training with the training data.

2. The method of claim 1, wherein the one or more levels of missingness for the data comprise a first factor corresponding to an overall degree of missingness for the data, a second factor corresponding to one or more degrees of missingness for individual signals within a dataset, and a third factor corresponding to missingness degrees for different signal patterns in the data.

3. The method of claim 1, wherein the plurality of imputation techniques comprises some or all of a first imputation technique that performs expectation maximization to impute the missing data at a first level of missingness, a second imputation technique that performs the expectation maximization with external data at a second level of missingness, a third imputation technique that generates the training data using predicted values from a predictive model at a third level of missingness, or a fourth imputation technique that performs simulation to generate the training data at a fourth level of missingness.

4. The method of claim 1, wherein a second imputation technique is selected to impute the missing data when a first imputation technique does not successfully generate the missing data.

5. The method of claim 1, wherein the model training generates a predictive model that is employed for health monitoring of a database system.

6. A system for imputing data for a machine learning system, comprising:

a processor;

a memory for holding programmable code; and

wherein the programmable code includes instructions for collecting data from a monitored target system;

determining one or more levels of missingness for the data collected from the monitored target system;

selecting, from among a plurality of imputation techniques, a selected imputation technique based at least in part upon the one or more levels of missingness for the data, wherein expectation maximization (EM) is selected as the selected imputation technique if it is determined that both an overall level of missing data and individual levels of missing data for signals are at one or more designated thresholds, and an external data source is accessed to generate an EM seed for the expectation maximization when insufficient seed data exists within the data collected from the monitored target system;

imputing missing data using the selected imputation technique to generate training data; and

performing model training with the training data.

7. The system of claim 6, wherein the one or more levels of missingness for the data comprise a first factor corresponding to an overall degree of missingness for the data, a second factor corresponding to one or more degrees of missingness for individual signals within a dataset, and a third factor corresponding to missingness degrees for different signal patterns in the data.

8. The system of claim 6, wherein the plurality of imputation techniques comprises some or all of a first imputation technique that performs expectation maximization to impute the missing data at a first level of missingness, a second imputation technique that performs the expectation maximization with external data at a second level of missingness, a third imputation technique that generates the training data using predicted values from a predictive model at a third level of missingness, or a fourth imputation technique that performs simulation to generate the training data at a fourth level of missingness.

9. The system of claim 6, wherein a second imputation technique is selected to impute the missing data when a first imputation technique does not successfully generate the missing data.

10. The system of claim 6, wherein the model training generates a predictive model that is employed for health monitoring of a database system.

11. A computer program product embodied on a non-transitory computer readable medium, the non-transitory computer readable medium having stored thereon a sequence of instructions which, when executed by a processor, executes a method comprising:

collecting data from a monitored target system;

determining one or more levels of missingness for the data collected from the monitored target system;

selecting, from among a plurality of imputation techniques, a selected imputation technique based at least in part upon the one or more levels of missingness for the data, wherein expectation maximization (EM) is selected as the selected imputation technique if it is determined that both an overall level of missing data and individual levels of missing data for signals are at one or more designated thresholds, and an external data source is accessed to generate an EM seed for the expectation maximization when insufficient seed data exists within the data collected from the monitored target system;

imputing missing data using the selected imputation technique to generate training data; and

performing model training with the training data.

12. The computer program product of claim 11, wherein the one or more levels of missingness for the data comprise a first factor corresponding to an overall degree of missingness for the data, a second factor corresponding to one or more degrees of missingness for individual signals within a dataset, and a third factor corresponding to missingness degrees for different signal patterns in the data.

13. The computer program product of claim 11, wherein the plurality of imputation techniques comprises some or all of a first imputation technique that performs expectation maximization to impute the missing data at a first level of missingness, a second imputation technique that performs the expectation maximization with external data at a second level of missingness, a third imputation technique that generates the training data using predicted values from a predictive model at a third level of missingness, or a fourth imputation technique that performs simulation to generate the training data at a fourth level of missingness.

14. The computer program product of claim 11, wherein a second imputation technique is selected to impute the missing data when a first imputation technique does not successfully generate the missing data.

15. The computer program product of claim 11, wherein the model training generates a predictive model that is employed for health monitoring of a database system.