Predictive model caching

• US 8,626,791 B1
• Filed: 06/14/2011
• Issued: 01/07/2014
• Est. Priority Date: 06/14/2011
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Lin, Wei-Hao, Green, Travis H. K., Kaplow, Robert, Fu, Gang, Mann, Gideon S.
• Primary Examiner(s): SYED, FARHAN M
• Application Number:
  US13/159,882
• Time in Patent Office: 938 Days
• US Class Current: 707/778
• CPC Class Codes: G06F 16/9574 of access to content, e.g. by caching
• CPC Class Codes: G06N 20/00 Machine learning

Abstract

Methods, systems, and apparatus, including computer programs encoded on one or more computer storage devices, for caching predictive models are described. Records are obtained, each record including a time of a previously submitted predictive request and an identifier of a trained predictive model. A trained scheduling model is generated using the records as training data. A set of identifiers of trained predictive models are determined from a plurality of trained predictive models that are stored in a secondary memory of a computing system. The target time is inputted to the trained scheduling model. In response, a second predictive output is received that comprises the set of identifiers. A set of trained predictive models are obtained that correspond to the set of identifiers from the secondary memory. The set of trained predictive models are stored in a primary memory of the computing system.

Claims

1. A computer-implemented method comprising:

maintaining, in a secondary memory of a computing system, a collection of trained predictive models, wherein each trained predictive model in the collection has a respective identifier;

obtaining a plurality of records, wherein each record includes a time of a previously submitted predictive request and an identifier of a trained predictive model that provided a predictive output in response to the previously submitted predictive request, and wherein each of the plurality of records identifies the respective previously submitted predictive request as having been submitted by a first user;

generating a trained scheduling model using the plurality of records as training data, wherein the trained scheduling model is a trained individual scheduling model that is specific to the first user;

identifying, using the trained scheduling model, a particular set of trained predictive models that are most likely to receive predictive requests at a target time;

selecting, for storing, one or more of the trained predictive models in the particular set of trained predictive models that have a likelihood of receiving a predictive request at the target time that is higher relative to others of the trained predictive models in the particular set of trained predictive models; and

storing the selected one or more of the trained predictive models in the particular set of trained predictive models in a primary memory of the computing system, wherein the primary memory comprises volatile memory and the secondary memory comprises a non-volatile memory.

2. The method of claim 1, wherein the target time is a date, a day of a week, a time of the day, a date and time of day, or a day or week and time of day.

3. The method of claim 1,

wherein the first user is a client entity.

12. The method of claim 1, further comprising:

using the trained scheduling model to identify one or more other trained predictive models that are most likely to be used in tandem with the particular set of trained predictive models;

obtaining, from the secondary memory, prior to the target time, the one or more other trained predictive models; and

storing the one or more other trained predictive models in the primary memory.

4. A computer storage medium encoded with computer program instructions that when executed by one or more computers cause the one or more computers to perform operations comprising:

maintaining, in a secondary memory of the one or more computers, a collection of trained predictive models, wherein each trained predictive model in the collection has a respective identifier;

obtaining a plurality of records, wherein each record includes a time of a previously submitted predictive request and an identifier of a trained predictive model that provided a predictive output in response to the previously submitted predictive request, and wherein each of the plurality of records identifies the respective previously submitted predictive request as having been submitted by a first user;

generating a trained scheduling model using the plurality of records as training data, wherein the trained scheduling model is a trained individual scheduling model that is specific to the first user;

identifying, using the trained scheduling model, a particular set of trained predictive models that are most likely to receive predictive requests at a target time;

selecting, for storing, one or more of the trained predictive models in the particular set of trained predictive models that have a likelihood of receiving a predictive request at the target time that is higher relative to others of the trained predictive models in the particular set of trained predictive models; and

storing the selected one or more of the trained predictive models in the particular set of trained predictive models in a primary memory of the one or more computers, wherein the primary memory comprises volatile memory and the secondary memory comprises a non-volatile memory.

5. The medium of claim 4, wherein the target time is a date, a day of a week, a time of the day, a date and time of day, or a day or week and time of day.

6. The medium of claim 4,

wherein the first user is a client entity.

7. The medium of claim 4,

wherein each of the plurality of records identifies the respective previously submitted predictive request as having been submitted from a particular geographic region;

and

wherein the trained scheduling model is specific to the particular geographic region.

8. A system comprising:

one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising;

maintaining, in a secondary memory of a computing system, a collection of trained predictive models, wherein each trained predictive model in the collection has a respective identifier;

obtaining a plurality of records, wherein each record includes a time of a previously submitted predictive request and an identifier of a trained predictive model that provided a predictive output in response to the previously submitted predictive request, and wherein each of the plurality of records identifies the respective previously submitted predictive request as having been submitted by a first user;

generating a trained scheduling model using the plurality of records as training data, wherein the trained scheduling model is a trained individual scheduling model that is specific to the first user;

identifying, using the trained scheduling model, a particular set of trained predictive models that are most likely to receive predictive requests at a target time;

selecting, for storing, one or more of the trained predictive models in the particular set of trained predictive models that have a likelihood of receiving a predictive request at the target time that is higher relative to others of the trained predictive models in the particular set of trained predictive models; and

storing the selected one or more of the trained predictive models in the particular set of trained predictive models in a primary memory of the computing system, wherein the primary memory comprises volatile memory and the secondary memory comprises a non-volatile memory.

9. The system of claim 8, wherein the target time is a date, a day of a week, a time of the day, a date and time of day, or a day or week and time of day.

10. The system of claim 8,

wherein the first user is a client entity.

11. The system of claim 8,

wherein each of the plurality of records identifies the respective previously submitted predictive request as having been submitted from a particular geographic region;

and

wherein the trained scheduling model is specific to the particular geographic region.

13. The system of claim 8, wherein the operations further comprise:

using the trained scheduling model to identify one or more other trained predictive models that are most likely to be used in tandem with the particular set of trained predictive models;

obtaining, from the secondary memory, prior to the target time, the one or more other trained predictive models; and

storing the one or more other trained predictive models in the primary memory.

14. The system of claim 8, wherein generating the trained scheduling model comprises:

training a scheduling model using, for each of the plurality of records, the respective time of the previously submitted request as input and the respective identifier of the trained predictive models that satisfied the request as output.

15. The system of claim 8, wherein the operations further comprise:

determining that the primary memory does not have sufficient resources available to store the selected one or more of the trained predictive models; and

removing, from the primary memory, one or more trained predictive models to free up resources.

16. The system of claim 15, wherein the operations further comprise:

selecting, for removing, the one or more trained predictive models that are less likely to receive predictive models at the target time.

17. A computer-implemented method comprising:

maintaining, in a secondary memory of a computing system, a collection of trained predictive models, wherein each trained predictive model in the collection has a respective identifier;

obtaining a plurality of records, wherein each record includes a time of a previously submitted predictive request and an identifier of a trained predictive model that provided a predictive output in response to the previously submitted predictive request, and wherein each of the plurality of records identifies the respective previously submitted predictive request as having been submitted from a particular geographic region;

generating a trained scheduling model using the plurality of records as training data, wherein the trained scheduling model is specific to the particular geographic region;

identifying, using the trained scheduling model, a particular set of trained predictive models that are most likely to receive predictive requests at a target time;

obtaining, from the secondary memory, prior to the target time, the particular set of trained predictive models that are most likely to receive predictive requests at the target time;

selecting, for storing, one or more of the trained predictive models in the particular set of trained predictive models that have a likelihood of receiving a predictive request at the target time that is higher relative to others of the trained predictive models in the particular set of trained predictive models; and

storing the selected one or more of the trained predictive models in the particular set of trained predictive models in a primary memory of the computing system, wherein the primary memory comprises volatile memory and the secondary memory comprises a non-volatile memory.

18. A computer storage medium encoded with computer program instructions that when executed by one or more computers cause the one or more computers to perform operations comprising:

maintaining, in a secondary memory of the one or more computers, a collection of trained predictive models, wherein each trained predictive model in the collection has a respective identifier;

obtaining a plurality of records, wherein each record includes a time of a previously submitted predictive request and an identifier of a trained predictive model that provided a predictive output in response to the previously submitted predictive request, and wherein each of the plurality of records identifies the respective previously submitted predictive request as having been submitted from a particular geographic region;

generating a trained scheduling model using the plurality of records as training data, wherein the trained scheduling model is specific to the particular geographic region;

identifying, using the trained scheduling model, a particular set of trained predictive models that are most likely to receive predictive requests at a target time;

obtaining, from the second memory, prior to the target time, the particular set of trained predictive models that are most likely to receive predictive requests at the target time;

selecting, for storing, one or more of the trained predictive models in the particular set of trained predictive models that have a likelihood of receiving a predictive request at the target time that is higher relative to others of the trained predictive models in the particular set of trained predictive models; and

storing the selected one or more of the trained predictive models in the particular set of trained predictive models in a primary memory of the one or more computers, wherein the primary memory comprises volatile memory and the secondary memory comprises a non-volatile memory.

19. A system comprising:

one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising;

maintaining, in a secondary memory of a computing system, a collection of trained predictive models, wherein each trained predictive model in the collection has a respective identifier;

obtaining a plurality of records, wherein each record includes a time of a previously submitted predictive request and an identifier of a trained predictive model that provided a predictive output in response to the previously submitted predictive request, and wherein each of the plurality of records identifies the respective previously submitted predictive request as having been submitted from a particular geographic region;

generating a trained scheduling model using the plurality of records as training data, wherein the trained scheduling model is specific to the particular geographic region;

identifying, using the trained scheduling model, a particular set of trained predictive models that are most likely to receive predictive requests at a target time;

obtaining, from the secondary memory, prior to the target time, the particular set of trained predictive models that are most likely to receive predictive requests at a target time;

selecting, for storing, one or more of the trained predictive models in the particular set of trained predictive models that have a likelihood of receiving a predictive request at the target time that is higher relative to others of the trained predictive models in the particular set of trained predictive models; and

storing the selected one or more of the trained predictive models in the particular set of trained predictive models in a primary memory of the computing system, wherein the primary memory comprises volatile memory and the secondary memory comprises a non-volatile memory.