Collaborative targeted maximum likelihood learning

• US 8,996,445 B2
• Filed: 04/07/2010
• Issued: 03/31/2015
• Est. Priority Date: 04/07/2009
• Status: Expired due to Fees

• Current Assignee: Regents of the University of California (University of California)
• Original Assignee: Regents of the University of California (University of California)
• Inventors: van der Laan, Mark, Gruber, Susan
• Primary Examiner(s): Chang, Li-Wu
• Assistant Examiner(s): Bharadwaj, Kalpana
• Application Number:
  US13/263,252
• Publication Number:
  US 20120271791A1
• Time in Patent Office: 1,819 Days
• US Class Current: 706/52
• CPC Class Codes: G06N 20/00 Machine learning

Abstract

A method is provided comprising: determining one or more initial distribution estimators of a true probability distribution; determining for each series of estimators of a nuisance parameter; generating candidate targeted estimators of the probability distribution for a targeted feature using an iterative modification of the initial distribution estimator(s) designed to reduce bias in the estimate of the target feature with respect to the initial distribution estimator, based on the series of estimators and the targeted feature; selecting one of the candidate estimators as an estimator for the true probability distribution for the targeted feature; and applying a mapping to the estimator for the true probability distribution or relevant portion thereof to obtain an estimated value for the targeted feature.

Claims

1. A method comprising:

determining one or more initial distribution estimators of a true probability distribution;

determining a series of estimators of a nuisance parameter;

generating targeted candidate estimators of the probability distribution for a targeted feature using an iterative modification of the one or more initial distribution estimators based on the series of estimators and the targeted feature; and

selecting one of the candidate estimators as an estimator for the true probability distribution for the targeted feature.

2. The method of claim 1, further including applying a mapping to the selected estimator to determine an estimated value for the targeted feature.

3. The method of claim 1, wherein generating the candidate estimators comprises determining a first candidate estimator for a first estimator of the nuisance parameter.

4. The method of claim 3, further comprising:

using the first candidate estimator and a second estimator of the nuisance parameter to determine a second candidate estimator.

5. The method of claim 1, wherein selecting one of the candidate estimators includes applying a likelihood based analysis to an update of an initial distribution estimator, wherein the update depends on the nuisance parameter estimator.

6. The method of claim 5, wherein the update of the initial distribution estimator includes a targeted maximum likelihood estimator.

7. The method of claim 5, wherein the likelihood based analysis includes using a likelihood criterion applied to an update of an initial distribution estimator using a candidate nuisance parameter estimator to determine the selection.

8. The method of claim 7, further comprising:

determining a penalized likelihood criterion, wherein a penalty to be applied to the likelihood criterion is determined based on a function of the candidate estimators of the probability distribution for the targeted feature.

9. The method of claim 8, wherein the function includes an estimate of the mean square error of the targeted feature of the candidate estimators of the probability distribution.

10. The method of claim 1, wherein a plurality of estimators for the true probability distribution are determined, the method further comprising:

selecting one of the plurality of estimators based on a criterion.

11. The method of claim 10, wherein the selecting is performed based on a likelihood based analysis.

12. A method for obtaining an estimator of a true probability distribution, and a corresponding estimator of a target feature, the method comprising:

selecting one or more initial estimators ({circumflex over (P)}) of a true probability distribution (P_TRUE) of a dataset based on one or more candidate estimators (P), a feature map (Ψ

), and the dataset;

determining one or more series of estimators (g-estimators) of a nuisance parameter (g_TRUE); and

employing the one or more series of estimators (g-estimators) to modify the one or more initial estimators ({circumflex over (P)}) and to provide one or more candidate targeted estimators ({circumflex over (P)}*) in response thereto.

13. The method of claim 12, further including applying the feature map to the selected targeted estimator ({circumflex over (P)}*) to obtain an estimated value for the target feature.

14. The method of claim 12, wherein employing further includes using a targeted feature and the one or more series of estimators (g-estimators) to provide the one or more candidate targeted estimators ({circumflex over (P)}*).

15. The method of claim 14, wherein employing further includes iteratively modifying one or more initial estimators ({circumflex over (P)}) by updating the one or more series of estimators (g-estimators) with one or more updated versions of the initial estimators ({circumflex over (P)}) to yield further updated one or more estimators and providing a series of iteratively modified estimators ({circumflex over (P)}*) in response thereto.

16. The method of claim 15, wherein a series of the one or more series of estimators (g-estimators) includes a first estimator that is less data-adaptive than a second estimator.

17. The method of claim 15, wherein each successive iteratively modified estimator ({circumflex over (P)}*) in the series of iteratively modified estimators more closely fits the data generated by the true probability distribution (P_TRUE) than each previous iteratively modified estimator of the series.

18. The method of claim 15, further including selecting a preferred modified estimator from the series of iteratively modified estimators ({circumflex over (P)}*) based on a predetermined selection method.

19. The method of claim 18, wherein the predetermined selection method includes likelihood-based cross-validation of candidate modified estimators that are based on one or more series of nuisance parameter estimators ({ĝ

}_i).

20. The method of claim 12, wherein an initial P-estimator module is adapted to receive the dataset, the one or more candidate estimators (P), and the feature map (Ψ

) as input thereto and to provide one or more initial distribution estimators as output in response thereto.

21. The method of claim 20, wherein a collaborative g-estimator module is adapted to provide a first series of estimators of the nuisance parameter (g_TRUE)based on the one or more initial distribution estimators and to further provide one or more subsequent series of estimators of the nuisance parameter (g_TRUE ) based on one or more updated initial distribution estimators.

22. The method of claim 20, wherein a targeted P-estimator ({circumflex over (P)}*) is adapted to employ a series of estimators of a nuisance parameter output by a collaborative g-estimator module to provide the one or more updated initial distribution estimators.

23. The method of claim 12, further including a targeted P-selector module in communication with a targeted P-estimator module, wherein the targeted P-estimator module is adapted to select one or more iteratively modified estimators output from the targeted P-estimator module.

24. A method comprising:

obtaining a dataset;

determining a first set of one or more candidate estimators (P) of a true probability distribution (P_TRUE) of the dataset;

employing a feature map, the dataset, and the one or more candidate estimators to determine a set of one or more initial distribution estimators ({circumflex over (P)});

using the feature map and the set of one or more initial distribution estimators ({circumflex over (P)}) to determine a series of estimators (g-estimators) of a nuisance parameter (g_TRUE); and

modifying the set of one or more initial distribution estimators ({circumflex over (P)}) via the series of estimators (g-estimators) of a nuisance parameter, yielding one or more modified estimators ({circumflex over (P)}*) in response thereto.

25. A method comprising:

selecting an initial estimator (ĝ

₁*) of a nuisance parameter, wherein the nuisance parameter is characterized by a true distribution (g_TRUE);

employing the initial estimator (ĝ

₁*) to construct a set of tentative candidate estimators of the nuisance parameter based on a sequence of estimators of a nuisance parameter;

selecting an estimator from the set of tentative candidate estimators based on a first predetermined selection method and providing one or more selected estimators of the nuisance parameter in response thereto; and

choosing a preferred candidate estimator of the nuisance parameter from the selected estimators of the nuisance parameter based on a second predetermined selection method.

26. A method comprising:

generating a sequence of targeted maximum likelihood estimators, wherein the sequence includes estimators with increasing likelihood, and wherein each estimator that has a higher likelihood than a previous estimator is indexed by a nuisance parameter estimator that is more nonparametric than a previous nuisance parameter estimator used to index the previous estimator;

updating one or more estimators of the sequence of targeted maximum likelihood estimators and providing one or more updated estimators in response thereto, wherein updating includes adjusting one or more nuisance parameters that index targeted maximum likelihood estimators of the sequence; and

selecting one or more estimators from the updated estimators and providing one or more selected estimators.

27. The method of claim 26, wherein selecting includes selecting one or more estimators from the updated estimators and providing one or more selected estimators and the result of applying mapping (Ψ

) to the one or more selected estimators to obtain an estimate of the target feature.