Reducing latency by caching derived data at an edge server

• US 10,102,133 B2
• Filed: 05/31/2016
• Issued: 10/16/2018
• Est. Priority Date: 05/31/2016
• Status: Active Grant

• Current Assignee: Salesforce.com, Inc.
• Original Assignee: Salesforce.com, Inc.
• Inventors: Pack, III, Richard Perry
• Primary Examiner(s): Song, Jasmine
• Application Number:
  US15/169,574
• Publication Number:
  US 20170344484A1
• Time in Patent Office: 868 Days
• Field of Search: 711130
• CPC Class Codes: G06F 12/0833 in combination with broadcast means (e.g. for invalidation or updating)
• CPC Class Codes: G06F 12/0862 with prefetch
• CPC Class Codes: G06F 12/128 adapted to multidimensional cache systems, e.g. set-associative, multicache, multiset or multilevel
• CPC Class Codes: G06F 2212/6026 Prefetching based on access pattern detection, e.g. stride based prefetch
• CPC Class Codes: G06F 2212/621 Coherency control relating to peripheral accessing, e.g. from DMA or I/O device
• CPC Class Codes: G06F 2212/70 Details relating to dynamic memory management
• CPC Class Codes: G06F 9/54 Interprogram communication
• CPC Class Codes: G06F 9/546 Message passing systems or structures, e.g. queues
• CPC Class Codes: H04L 67/568 Storing data temporarily at an intermediate stage, e.g. caching

Abstract

To deliver up-to-date, coherent user data to applications upon request, the disclosed technology includes systems and methods for caching data and metadata after it has been synchronously loaded—for future retrieval with a page load time close to zero milliseconds. To provide this experience, data needs to be stored as locally to a user as possible, in the cache on the local device or in an edge cache located geographically nearby, for use in responding to requests. Applications which maintain caches of API results can be notified of their invalidation, and can detect the invalidation, propagate the invalidation to any further client tiers with the appropriate derivative type mapping, and refresh their cached values so that clients need not synchronously make the API requests again—ensuring that the client has access to the most up-to-date copy of data as inexpensively as possible—in terms of bandwidth and latency.

Claims

1. A method of caching derived data at an edge cache coupled to an edge server, for use in responding to requests from a data consuming application via an application program interface (abbreviated API) to at least one application program running via an edge server, wherein the application program uses core data as input and returns derived data, the method of caching derived data including:

a core cache manager, running on a caching system, receiving information regarding core data instances of interest requested from a data source and regarding requests for derived data of interest;

the core cache manager;

monitoring and detecting changes to the core data instances of interest;

inferring impact on the derived data of the changed core data instances;

pre-caching an updated version of the impacted derived data by replaying at least one API access that is dependent on the changed core data instances;

informing an edge cache manager, running on a same device as or connected by a bus or local area network to an edge server, that responds to requests from the data consuming application, of the changes to the core data instances of interest and of the impact on derived data returned by at least one API; and

the core cache manager, responsive to information from the data consuming application regarding the changes to core data instances of interest and to the impacted derived data, determining whether follow-up requests from the edge cache manager for the changed core data instances and the impacted derived data can be satisfied by the core cache manager without accessing the data source and satisfying the requests for the changed core data and the impacted derived data from cache memory.

2. The method of claim 1, further including the core cache manager monitoring a time to live counter or timestamp for the impacted derived data in the cache memory to determine whether the impacted derived data can be used to satisfy follow-up requests from the edge cache manager.

3. The method of claim 1, further including the core cache manager relying on an invalidation listener to monitor and detect the changes to the core data instances of interest;

and the invalidation listener receiving invalidation messages reflecting changes to the core data at the data source.

4. The method of claim 1, wherein the inferring of impact on the derived data, at least in part, includes determining from an impact map one or more API calls that use the changed core data as API input.

5. The method of claim 1, wherein the inferring of impact on the derived data, at least in part, includes determining from cache memory one or more API calls made by the data consuming application using the changed core data instances.

6. The method of claim 1, wherein the inferring of impact on the derived data, at least in part, includes using a user-specific key to access cache memory and retrieve one or more API calls made by the data consuming application that used the changed core data instances as API input.

7. The method of claim 1, further including a core cache API through which the core cache manager communicates with an external customer process, including the core cache manager:

monitoring and detecting changes to the core data instances of interest to the external customer process;

inferring impact on the derived data of the changed core data instances to the external customer process;

informing the external customer process of the changes to the core data instances of interest and of the impact on derived data returned by at least one API; and

responsive to information regarding the changes to core data instances of interest and the impacted derived data, determining whether follow-up requests from the external customer process for the changed core data instances and the impacted derived data can be satisfied by the core cache manager without accessing the data source and satisfying the requests for the changed core data and the impacted derived data from cache memory.

8. A system of computing devices, including:

an edge cache that stores both core data and derived data, wherein the derived data is generated by passing some of the core data to at least one application program interface (abbreviated API) and receiving the derived data in return;

a data source from which core data is obtained;

at least one application program running on a server accessed via the API, wherein the application program receives the core data as input and returns the derived data;

an edge cache manager, local to an edge server instance, that refreshes an edge cache of core data and derived data requested from the application program;

a core cache manager running on a caching system that;

accesses interest indications of core data instances and derived data of interest to the edge cache manager;

monitors and detects changes to the core data instances of interest;

infers impact on the derived data of the changed core data instances;

pre-caches an updated version of the impacted derived data by replaying at least one API access that is dependent on the changed core data instances; and

informs the edge cache manager of the changes to the core data instances of interest and of the impact on derived data returned by the at least one API; and

wherein the edge cache manager, responsive to information from the core cache manager regarding the changes to the interesting core data and the impact on derived data determines whether the derived data is of continuing interest and, if so, replays the at least one API access using the changed core data instances to refresh cached derived data.

9. The system of claim 8, further including the core cache manager further includes monitoring a time to live counter or timestamp for the impacted derived data in cache memory to determine whether the impacted derived data can be used to satisfy follow-up requests from the edge cache manager.

10. The system of claim 8, further including the core cache manager relying on an invalidation listener to monitor and detect the changes to the core data instances of interest;

and the invalidation listener receiving invalidation messages reflecting changes to the core data at the data source.

11. The system of claim 8, wherein the inferring of impact on the derived data, at least in part, includes determining from an impact map one or more API calls that use the changed core data as API input.

12. The system of claim 8, wherein the inferring of impact on the derived data, at least in part, includes determining from cache memory one or more API calls made by the data consuming application using the changed core data instances.

13. The system of claim 8, wherein the inferring of impact on the derived data, at least in part, includes using a user-specific key to access cache memory and retrieve one or more API calls made by the data consuming application that used the changed core data instances as API input.

14. The system of claim 8, further including a core cache API through which the core cache manager communicates with an external customer process, wherein the core cache manager further:

monitors and detects changes to the core data instances of interest to the external customer process;

infers impact on the derived data of the changed core data instances to the external customer process;

informs the external customer process of the changes to the core data instances of interest and of the impact on derived data returned by at least one API; and

responsive to information regarding the changes to core data instances of interest and to the impacted derived data, determines whether follow-up requests from the to the external customer process for the changed core data instances and the impacted derived data can be satisfied by the core cache manager without accessing the data source and satisfying the requests for the changed core data and the impacted derived data from cache memory.

15. A non-transitory computer readable storage medium loaded with computer instructions that, when executed, cause a computer system to cache derived data for a client cache, including:

an edge cache that stores both core data and derived data, wherein the derived data is generated by passing some of the core data to at least one application program interface (abbreviated API) and receiving the derived data in return;

a data source from which core data is obtained;

at least one application program running on a server accessed via the API, wherein the application program receives the core data as input and returns the derived data;

an edge cache manager, local to an edge server instance, that refreshes an edge cache of core data and derived data requested from the application program;

a core cache manager running on a caching system that;

accesses interest indications of core data instances and derived data of interest to the edge cache manager;

monitors and detects changes to the core data instances of interest;

infers impact on the derived data of the changed core data instances;

pre-caches an updated version of the impacted derived data by replaying at least one API access that is dependent on the changed core data instances; and

informs the edge cache manager of the changes to the core data instances of interest and of the impact on derived data returned by at least one API; and

wherein the edge cache manager, responsive to information from the core cache manager regarding the changes to the interesting core data and to the impacted derived data, determines whether the impacted derived data is of continuing interest and, if so, replays the at least one API access with the changed core data to refresh cached derived data from the API access.

16. The non-transitory computer readable storage medium of claim 15, causing the core cache manager to take actions further including monitoring a time to live counter or timestamp for the impacted derived data in cache memory to determine whether the impacted derived data is of continuing interest.

17. The non-transitory computer readable storage medium of claim 15, wherein the inferring of impact on the derived data, at least in part, includes determining from an impact map one or more API calls that use the changed core data as API input.

18. The non-transitory computer readable storage medium of claim 15, wherein the inferring of impact on the derived data, at least in part, includes determining from cache memory one or more API calls made by the data consuming application using the changed core data instances.

19. The non-transitory computer readable storage medium of claim 15, wherein the inferring of impact on the derived data, at least in part, includes using a user-specific key to access cache memory and retrieve one or more API calls made by the data consuming application that used the changed core data instances as API input.

20. The non-transitory computer readable storage medium of claim 15, causing the core cache manager to take actions further including refreshing the impacted derived data after replaying at least one API access using the changed core data instances.

21. The non-transitory computer readable storage medium of claim 15, wherein the core cache manager further includes a core cache API through which the core cache manager communicates with an external customer process, further including causing the core cache manager to:

monitor and detect changes to core data instances of interest to the external customer process;

infer impact on derived data of the changed core data instances of interest to the external customer process;

inform the external customer process of the changes to the core data instances of interest and of the impact on derived data returned by at least one API; and

responsive to information regarding the changes to core data instances of interest and to the impacted derived data, determine whether follow-up requests from the to the external customer process for the changed core data instances and the impacted derived data can be satisfied by the core cache manager without accessing the data source and satisfy the requests for the changed core data and the impacted derived data from cache memory.

22. A cache manager including a core cache API through which the cache manager communicates with an external customer process, the cache manager running on an edge caching system that includes a processor and computer instructions that when executed the processor cause the cache manager to carry out a method including:

monitoring and detecting changes to core data instances of interest to the external customer process;

inferring impact on derived data of the changed core data instances to the external customer process;

pre-caching an updated version of the impacted derived data by replaying at least one API access that is dependent on the changed core data instances;

informing the external customer process of the changes to the core data instances of interest and of the impact on derived data returned by at least one API; and

responsive to information regarding the changes to core data instances of interest and the impacted derived data, determining whether follow-up requests from the external customer process for the changed core data instances and the impacted derived data can be satisfied by the cache manager without accessing a data source and satisfying the requests for the changed core data and the impacted derived data from cache memory.