Predictive analytical model matching

• US 8,311,967 B1
• Filed: 09/27/2011
• Issued: 11/13/2012
• Est. Priority Date: 05/14/2010
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Lin, Wei-Hao, Green, Travis H. K., Kaplow, Robert, Fu, Gang, Mann, Gideon S.
• Primary Examiner(s): FERNANDEZ RIVAS, OMAR F
• Application Number:
  US13/246,229
• Time in Patent Office: 413 Days
• Field of Search: None
• US Class Current: 706/45
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 5/04 Inference or reasoning models

Abstract

Methods, systems, and apparatus, for selecting a trained predictive models. A request is received from a client-subscriber computing system for access to a trained predictive model that can generate a predictive output in response to receiving input data having one or more input types. Information that describes each of the trained predictive models in a predictive model repository can be used to determine that one or more models included in the repository match the request. Determining a match can be based (at least in part) on a comparison of the one or more input types to input types included in the information that describes the trained predictive models. Access is provided to at least one of the models to the client-subscriber computing system. The models that match the request are models that were trained using training data provided by a computing system other than the client-subscriber computing system.

Claims

1. A computer-implemented system comprising:

one or more computers;

one or more data storage devices coupled to the one or more computers, storing;

a predictive model repository of a plurality of trained predictive models,

information that describes each of the trained predictive models, which information includes an indication for each trained predictive model of one or more input types of input data that are compatible with the trained predictive model and an output type of a predictive output that can be generated using the trained predictive model, and

instructions that, when executed by the one or more computers, cause the one or more computers to perform operations comprising;

receiving a request from a client-subscriber computing system for access to a trained predictive model from the predictive model repository, which trained predictive model can generate a predictive output in response to receiving input data having one or more input types, wherein the one or more input types can be determined from the request;

determining from the information that describes each of the trained predictive models that one or more models included in the predictive model repository match the request from the client-subscriber computing system, wherein determining a match is based at least in part on a comparison of the one or more input types determined from the request to input types included in the information that describes the trained predictive models; and

providing access to at least one of the one or more models to the client-subscriber computing system;

wherein the one or more models that match the request are models that were trained using training data provided by a computing system other than the client-subscriber computing system.

2. The system of claim 1, the operations further comprising:

determining an output type for predictive output for the client-subscriber computing system, wherein;

if the output type is included in the request, then determining the output type comprises receiving the output type in the request; and

if the output type is not included in the request, then determining the output type comprises determining from the information that describes each of the trained predictive models that one or more models included in the predictive model repository are compatible with the one or more input types determined from the request to input types and that the one or more models can generate output of one or more output types, and selecting the output type from the one or more output types.

3. The system of claim 2, wherein selecting the output type from the one or more output types comprises selecting the output type in response to input received from the client-subscriber computing system that identifies an output type to select.

4. The system of claim 2, wherein selecting the output type from the one or more output types comprises selecting the output type that is associated with a model from the one or more models that is best matched to the request.

5. The system of claim 2, wherein determining from the information that describes each of the trained predictive models that one or more models included in the predictive model repository match the request from the client-subscriber computing system is further based on a comparison of the output type determined for the request to output types included in the information that describes the trained predictive models.

6. The system of claim 5, wherein a first model of the one or more models that are determined to match the request comprises a model that requires less input types to generate a predictive output than a number of input types determined from the request.

7. The system of claim 5, wherein determining from the information that describes each of the trained predictive models that one or more models included in the predictive model repository match the request from the client-subscriber computing system includes determining that a first model that generates the output type determined for the request requires a first input type to generate a predictive output which first input type is not included in the one or more input types determined from the request, the operations further comprising:

identifying an intermediate model included in the predictive model repository that can be used with at least some of the one or more input types determined from the request to generate a predictive output that is of the first input type;

wherein the one or more models determined to match the request from the client-subscriber computing system includes a set of models, which set includes the first model and the intermediate model that together match the request.

8. The system of claim 1, wherein the one or more input types are indicated in the request or can be derived from one or more samples of input data included in the request.

9. The system of claim 1, wherein:

at least one of the one or more input types determined from the request is included in a hierarchy of input types, wherein the at least one input type can be represented by incrementally broader input types by incrementing to higher levels in the hierarchy; and

determining a match based at least in part on a comparison of the one or more input types determined from the request to input types included in the information that describes the trained predictive models includes incrementing to one or more higher levels in the hierarchy until an input type that describes a trained predictive model matches the at least one input type determined from the request as represented by incrementally broader input types.

10. A computer-implemented method comprising:

receiving a request from a client-subscriber computing system for access to a trained predictive model from a repository of trained predictive models, which trained predictive model can generate a predictive output in response to receiving input data having one or more input types, wherein the one or more input types can be determined from the request;

determining from information that describes each of the trained predictive models in the predictive model repository that one or more models included in the predictive model repository match the request from the client-subscriber computing system, wherein determining a match is based at least in part on a comparison of the one or more input types determined from the request to input types included in the information that describes the trained predictive models; and

providing access to at least one of the one or more models to the client-subscriber computing system;

wherein the one or more models that match the request are models that were trained using training data provided by a computing system other than the client-subscriber computing system.

11. The method of claim 10, further comprising:

determining an output type for predictive output for the client-subscriber computing system, wherein;

if the output type is included in the request, then determining the output type comprises receiving the output type in the request; and

if the output type is not included in the request, then determining the output type comprises determining from the information that describes each of the trained predictive models that one or more models included in the predictive model repository are compatible with the one or more input types determined from the request to input types and that the one or more models can generate output of one or more output types, and selecting the output type from the one or more output types.

12. The method of claim 11, wherein selecting the output type from the one or more output types comprises selecting the output type in response to input received from the client-subscriber computing system that identifies an output type to select.

13. The method of claim 11, wherein selecting the output type from the one or more output types comprises selecting the output type that is associated with a model from the one or more models that is best matched to the request.

14. The method of claim 11, wherein determining from the information that describes each of the trained predictive models that one or more models included in the predictive model repository match the request from the client-subscriber computing system is further based on a comparison of the output type determined for the request to output types included in the information that describes the trained predictive models.

15. The method of claim 14, wherein a first model of the one or more models that are determined to match the request comprises a model that requires less input types to generate a predictive output than a number of input types determined from the request.

16. The method of claim 14, wherein determining from the information that describes each of the trained predictive models that one or more models included in the predictive model repository match the request from the client-subscriber computing system includes determining that a first model that generates the output type determined for the request requires a first input type to generate a predictive output which first input type is not included in the one or more input types determined from the request, the method further comprising:

identifying an intermediate model included in the predictive model repository that can be used with at least some of the one or more input types determined from the request to generate a predictive output that is of the first input type;

wherein the one or more models determined to match the request from the client-subscriber computing system includes a set of models, which set includes the first model and the intermediate model that together match the request.

17. The method of claim 10, wherein the one or more input types are indicated in the request or can be derived from one or more samples of input data included in the request.

18. The method of claim 10, wherein:

at least one of the one or more input types determined from the request is included in a hierarchy of input types, wherein the at least one input type can be represented by incrementally broader input types by incrementing to higher levels in the hierarchy; and

determining a match based at least in part on a comparison of the one or more input types determined from the request to input types included in the information that describes the trained predictive models includes incrementing to one or more higher levels in the hierarchy until an input type that describes a trained predictive model matches the at least one input type determined from the request as represented by incrementally broader input types.

19. A computer-readable storage device encoded with a computer program product, the computer program product comprising instructions that when executed on one or more computers cause the one or more computers to perform operations comprising:

receiving a request from a client-subscriber computing system for access to a trained predictive model from a repository of trained predictive models, which trained predictive model can generate a predictive output in response to receiving input data having one or more input types, wherein the one or more input types can be determined from the request;

determining from information that describes each of the trained predictive models in the predictive model repository that one or more models included in the predictive model repository match the request from the client-subscriber computing system, wherein determining a match is based at least in part on a comparison of the one or more input types determined from the request to input types included in the information that describes the trained predictive models; and

providing access to at least one of the one or more models to the client-subscriber computing system;

wherein the one or more models that match the request are models that were trained using training data provided by a computing system other than the client-subscriber computing system.

20. The computer-readable storage device of claim 19, the operations further comprising:

determining an output type for predictive output for the client-subscriber computing system, wherein;

if the output type is included in the request, then determining the output type comprises receiving the output type in the request; and

if the output type is not included in the request, then determining the output type comprises determining from the information that describes each of the trained predictive models that one or more models included in the predictive model repository are compatible with the one or more input types determined from the request to input types and that the one or more models can generate output of one or more output types, and selecting the output type from the one or more output types.

21. The computer-readable storage device of claim 20, wherein selecting the output type from the one or more output types comprises selecting the output type in response to input received from the client-subscriber computing system that identifies an output type to select.

22. The computer-readable storage device of claim 20, wherein selecting the output type from the one or more output types comprises selecting the output type that is associated with a model from the one or more models that is best matched to the request.

23. The computer-readable storage device of claim 20, wherein determining from the information that describes each of the trained predictive models that one or more models included in the predictive model repository match the request from the client-subscriber computing system is further based on a comparison of the output type determined for the request to output types included in the information that describes the trained predictive models.

24. The computer-readable storage device of claim 23, wherein a first model of the one or more models that are determined to match the request comprises a model that requires less input types to generate a predictive output than a number of input types determined from the request.

25. The computer-readable storage device of claim 23, wherein determining from the information that describes each of the trained predictive models that one or more models included in the predictive model repository match the request from the client-subscriber computing system includes determining that a first model that generates the output type determined for the request requires a first input type to generate a predictive output which first input type is not included in the one or more input types determined from the request, the operations further comprising:

identifying an intermediate model included in the predictive model repository that can be used with at least some of the one or more input types determined from the request to generate a predictive output that is of the first input type;

wherein the one or more models determined to match the request from the client-subscriber computing system includes a set of models, which set includes the first model and the intermediate model that together match the request.

26. The computer-readable storage device of claim 19, wherein the one or more input types are indicated in the request or can be derived from one or more samples of input data included in the request.

27. The computer-readable storage device of claim 19, wherein:

at least one of the one or more input types determined from the request is included in a hierarchy of input types, wherein the at least one input type can be represented by incrementally broader input types by incrementing to higher levels in the hierarchy; and

determining a match based at least in part on a comparison of the one or more input types determined from the request to input types included in the information that describes the trained predictive models includes incrementing to one or more higher levels in the hierarchy until an input type that describes a trained predictive model matches the at least one input type determined from the request as represented by incrementally broader input types.