Hosting predictive models

• US 8,364,613 B1
• Filed: 09/27/2011
• Issued: 01/29/2013
• Est. Priority Date: 07/14/2011
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Lin, Wei-Hao, Kaplow, Robert, Fu, Gang, Mann, Gideon S., Green, Travis H.
• Primary Examiner(s): Chaki, Kakali
• Assistant Examiner(s): Bharadwaj, Kalpana
• Application Number:
  US13/246,596
• Time in Patent Office: 490 Days
• Field of Search: 706/12
• US Class Current: 706/12
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks

Abstract

Methods include the actions of storing a first predictive model in computer-readable memory, the first predictive model having been defined based on a first training dataset provided by an owner of the first predictive model and being operable to generate an output based on a query, enabling access to the first predictive model based on permissions defined by the owner, while inhibiting access to the first training dataset, receiving a second training dataset from a user, the second training dataset being distinct from the first training dataset, modifying the first predictive model based on the second training dataset to provide a second predictive model, storing the second predictive model in computer-readable memory, and enabling access to the second predictive model.

Claims

1. A computer-implemented method, comprising:

storing a first predictive model in computer-readable memory, the first predictive model having been defined based on a first training dataset provided by an owner of the first predictive model and being operable to generate an output based on a query;

enabling access for a user to the first predictive model based on permissions defined by the owner, while inhibiting access for the user to the first training dataset such that the first training dataset is inaccessible to the user;

receiving a second training dataset from the user, the second training dataset being distinct from the first training dataset;

modifying the first predictive model based on the second training dataset to provide a second predictive model;

storing the second predictive model in computer-readable memory; and

enabling access for the user to the second predictive model.

2. The method of claim 1, wherein storing a first predictive model comprises receiving the first predictive model from an owner.

3. The method of claim 1, wherein storing a first predictive model comprises:

receiving the first training dataset from an owner; and

applying the first training dataset to a first machine learning algorithm to generate the first predictive model.

4. The method of claim 1, wherein the first training dataset comprises a first set of training examples and the second training dataset comprises a second set of training examples, the second set of training examples comprising at least one training example that is divergent from the first set of training examples.

5. The method of claim 1, further comprising modifying the second training dataset by applying one or more favoring operations on the second training dataset.

6. The method of claim 1, wherein modifying the first predictive model comprises

generating a copy of the first predictive model;

and

updating the copy of the first predictive model.

7. The method of claim 6, wherein updating the copy of the first predictive model comprises applying the second training dataset and the copy of the first predictive model to a second machine learning algorithm.

8. The method of claim 7, wherein the first predictive model comprises an ensemble of combined predicted models, and wherein updating the copy of the first predictive model comprises:

applying the second training dataset to a second machine learning algorithm to generate a supplementary predictive model representative of the second training dataset; and

incorporating the supplementary predictive model into the ensemble of combined predictive models.

9. The method of claim 6, wherein updating the copy of the first predictive model comprises:

combining the first and second training datasets to form an updated training dataset; and

applying the updated training dataset to a second machine learning algorithm.

12. The method of claim 1, wherein modifying the first predictive model comprises:

receiving an instruction from the user through an application programming interface (API); and

modifying the first predictive model in response to the instruction.

13. The method of claim 6, wherein updating the copy of the first predictive model comprises modifying one or more hyper-parameters associated with the first predictive model.

14. The method of claim 8, incorporating the supplementary predictive model into the ensemble of combined predictive models comprises combining output from the supplementary predictive model with output from the ensemble of predictive models using a combining rule.

15. The method of claim 14, wherein combining output from the supplementary predictive model with output from the ensemble of predictive models comprises at least one of weighting or devaluing output from the supplementary predictive model.

16. The method of claim 2, wherein receiving the first predictive model from an owner comprises receiving a model representation of the first predictive model in the form of a Predictive Model Markup Language (PMML) document.

10. A computer storage medium encoded with a computer program, the program comprising instructions that when executed by one or more computers cause the one or more computers to perform operations comprising:

storing a first predictive model in computer-readable memory, the first predictive model having been defined based on a first training dataset provided by an owner of the first predictive model and being operable to generate an output based on a query;

enabling access for a user to the first predictive model based on permissions defined by the owner, while inhibiting access for the user to the first training dataset such that the first training dataset is inaccessible to the user;

receiving a second training dataset from the user, the second training dataset being distinct from the first training dataset;

modifying the first predictive model based on the second training dataset to provide a second predictive model;

storing the second predictive model in computer-readable memory; and

enabling access for the user to the second predictive model.

17. The computer storage medium of claim 10, further comprising modifying the second training dataset by applying one or more favoring operations on the second training dataset.

18. The computer storage medium of claim 10, wherein modifying the first predictive model comprises

generating a copy of the first predictive model;

and

updating the copy of the first predictive model.

11. A system, comprising:

one or more computing devices; and

one or more computer-readable media coupled to the one or more computing devices and having instructions stored thereon which, when executed by the one or more computing devices, cause the one or more computing devices to perform operations comprising;

storing a first predictive model in computer-readable memory, the first predictive model having been defined based on a first training dataset provided by an owner of the first predictive model and being operable to generate an output based on a query;

enabling access for a user to the first predictive model based on permissions defined by the owner, while inhibiting access for the user to the first training dataset such that the first training dataset is inaccessible to the user;

receiving a second training dataset from the user, the second training dataset being distinct from the first training dataset;

modifying the first predictive model based on the second training dataset to provide a second predictive model;

storing the second predictive model in computer-readable memory; and

enabling access for the user to the second predictive model.

19. The system of claim 11, further comprising modifying the second training dataset by applying one or more favoring operations on the second training dataset.

20. The system of claim 11, wherein modifying the first predictive model comprises

generating a copy of the first predictive model;

and

updating the copy of the first predictive model.