Allocating cache request in distributed cache system based upon cache object and marker identifying mission critical data

• US 9,584,617 B2
• Filed: 12/31/2013
• Issued: 02/28/2017
• Est. Priority Date: 12/31/2013
• Status: Active Grant

• Current Assignee: Successfactors Incorporated (SAP SE)
• Original Assignee: Successfactors Incorporated (SAP SE)
• Inventors: Zhao, Peter, Lu, Edward, Yang, Jessica, Ke, Tim, Han, Xiao, Liu, Kun
• Primary Examiner(s): WANG, LIANG CHE A
• Application Number:
  US14/144,948
• Publication Number:
  US 20150189033A1
• Time in Patent Office: 1,155 Days
• Field of Search: 709/203, 709/223, 709/224, 709/226
• US Class Current: 1/1
• CPC Class Codes: H04L 67/288 Distributed intermediate devices, i.e. intermediate devices for interaction with other intermediate devices on the same level
• CPC Class Codes: H04L 67/568 Storing data temporarily at an intermediate stage, e.g. caching

Abstract

Various embodiments of the present disclosure provide improved systems and techniques for intelligently allocating cache requests to caches based upon the nature of the cache objects associated with the cache requests, and/or the condition of the caches in the distributed cache system, to facilitate cache system utilization and performance. In some embodiments, an allocation of cache requests may be performed in response to detection of a problem at one of a group of cache servers. For example, in some embodiments, a particular cache server may be entered into a safe mode of operations when the cache'"'"'s ability to service cache requests is impaired. In other embodiments, an allocation of cache requests may be performed based on an cache object data type associated with the cache requests.

Claims

1. A method for intelligently allocating cache requests among caches in a distributed cache system, the method comprising:

allocating, at a computing device, a cache request to a first cache of the distributed cache system based on a marker identifying a mission critical data type associated with a cache object associated with the cache request, wherein the mission critical data type is identified based upon information gathered by monitoring operations concerning a number of cache objects of different types generated by user requests, and wherein the first cache comprises a local cache;

monitoring the caches in the distributed cache system;

when a slowdown in service is detected at the first cache, switching the first cache to a safe cache mode of operation;

periodically performing a test to determine whether the first cache has resumed normal operations;

when the first cache has resumed normal operations, switching the first cache to a normal cache mode of operations.

2. The method of claim 1, further comprising:

measuring a time period a cache takes to respond to each of a plurality of cache requests;

for each of the cache requests, when the time period the cache takes to respond to the cache request exceeds a first time threshold, throwing a time out exception;

counting a number of time out exceptions thrown with respect to the cache over a period of time;

when the number of time out exceptions thrown with respect to the cache exceeds a predetermined number over a predetermined time frame, entering the cache into cache safe mode of operations;

at predetermined time intervals, performing a test to determine whether cache requests to the cache are again being performed in a timely manner; and

when a determination is made that requests to the cache are again being timely executed, reentering the cache into a normal mode of operations.

3. The method of claim 2, wherein performing a test to determine whether cache requests to the cache are again being performed in a timely manner comprises:

allowing a first time period of a first predetermined length to pass;

sending a predetermined number of test cache requests to the cache server;

allowing a second time period of a second predetermined length to pass;

after the second time period has passed, determining whether any time out exceptions were thrown in connection with any of the test cache requests;

upon determining after the second time period that one or more such time out exception were thrown, restarting the process of determining whether cache requests to the cache area being performed in a timely manner.

4. The method of claim 3, wherein performing a test to determine whether cache requests to the cache server are again being performed in a timely manner further comprises:

when no time out exceptions were thrown in the second time period in connection with any of the test cache requests, reentering the cache into a normal mode of operations.

5. The method of claim 1, further comprising:

monitoring the health of a cache server of a plurality of cache servers by periodically sending a heartbeat signal to the cache server;

measuring a time period between a time that the heartbeat signal was sent, and a time a response to the heartbeat signal was received; and

when the cache server does not provide a responsive signal within a predetermined heartbeat time frame, removing the cache server from operating in the distributed cache system in response.

6. The method of claim 5, further comprising:

determining whether the cache server has recovered normal operations; and

after determining that the cache server has recovered normal operations, allowing the cache server to operate in the cache system again.

7. The method of claim 1, wherein allocating a cache request to a first cache of the distributed cache system based on a data type associated with a cache object associated with the cache request, comprises:

determining whether the cache request falls into one of a set of predetermined categories;

identifying a caching strategy for the cache request based on a first category determined to apply to the cache request;

allocating the cache request to one of the caches of the distributed cache system based upon the identified caching strategy; and

transmitting the cache request to the allocated cache.

8. The method of claim 7, wherein when the cache request is associated with frequently accessed data, the cache request is transmitted to the local cache.

9. The method of claim 7, further comprising:

gathering information at runtime concerning one or more of the following;

a quantity of cache requests generated for each of the predetermined categories, a frequency with which cache objects associated with a first category are accessed, and a frequency with which cache objects associated with the first category are modified.

10. The method of claim 9, further comprising:

analyzing the runtime information to determine whether improvements can be made to a caching strategy for the first category of cache requests;

wherein making adjustments to the caching strategy comprises changing the allocation of the first category of cache requests from a first cache to a second cache.

11. A computer system for intelligently allocating cache requests among caches in a distributed cache system, comprising:

one or more processors; and

a non-transitory computer readable medium having stored thereon one or more programs, which when executed by the one or more processors, causes the one or more processor to singly or in combination;

allocate a cache request to a first cache of the distributed cache system based on a marker identifying a mission critical data type associated with a cache object associated with the cache request, wherein the mission critical data type is identified based upon information gathered by monitoring operations concerning a number of cache objects of different types generated by user requests, and wherein the first cache comprises a local cache;

monitor the caches in the distributed cache system;

when a slowdown in service is detected at the first cache, switch the first cache to a safe cache mode of operation;

periodically perform a test to determine whether the first cache has resumed normal operations;

when the first cache has resumed normal operations, switch the first cache to a normal cache mode of operations.

12. The computer system of claim 11, wherein the one or more processors, singly or in combination, further:

measure a time period a cache takes to respond to each of a plurality of cache requests;

for each of the cache requests, when the time period the cache takes to respond to the cache request exceeds a first time threshold, throw a time out exception;

count a number of time out exceptions thrown with respect to the cache over a period of time;

when the number of time out exceptions thrown with respect to the cache exceeds a predetermined number over a predetermined time frame, enter the cache into cache safe mode of operations;

at predetermined time intervals, perform a test to determine whether cache requests to the cache are again being performed in a timely manner; and

when a determination is made that requests to the cache are again being timely executed, reenter the cache into a normal mode of operations.

13. The computer system of claim 12, wherein performing a test to determine whether cache requests to the cache are again being performed in a timely manner comprises:

allowing a first time period of a first predetermined length to pass;

sending a predetermined number of test cache requests to the cache server;

allowing a second time period of a second predetermined length to pass;

after the second time period has passed, determining whether any time out exceptions were thrown in connection with any of the test cache requests;

upon determining after the second time period that one or more such time out exception were thrown, restarting the process of determining whether cache requests to the cache area being performed in a timely manner.

14. The computer system of claim 13, wherein performing a test to determine whether cache requests to the cache server are again being performed in a timely manner further comprises:

when no time out exceptions were thrown in the second time period in connection with any of the test cache requests, reentering the cache into a normal mode of operations.

15. The computer system of claim 11, wherein the one or more processors, singly or in combination, further:

monitor the health of a cache server of a plurality of cache servers by periodically sending a heartbeat signal to the cache server;

measure a time period between a time that the heartbeat signal was sent, and a time a response to the heartbeat signal was received;

when the cache server does not provide a responsive signal within a predetermined heartbeat time frame, remove the cache server from operating in the distributed cache system in response.

16. The computer system of claim 15, wherein the one or more processors, singly or in combination, further:

determine whether the cache server has recovered normal operations; and

after determining that the cache server has recovered normal operations, allow the cache server to operate in the cache system again.

17. The computer system of claim 11, wherein allocating a cache request to a first cache of the distributed cache system based on a data type associated with a cache object associated with the cache request, comprises:

determining whether the cache request falls into one of a set of predetermined categories;

identifying a caching strategy for the cache request based on a first category determined to apply to the cache request;

allocating the cache request to one of the caches of the distributed cache system based upon the identified caching strategy; and

transmitting the cache request to the allocated cache.

18. The computer system of claim 17, wherein the one or more processors, singly or in combination, further:

gather information at runtime concerning one or more of the following;

a quantity of cache requests generated for each of the predetermined categories, a frequency with which cache objects associated with a first category are accessed, and a frequency with which cache objects associated with the first category are modified.

19. The method of claim 18, wherein the one or more processors, singly or in combination, further:

analyze the runtime information to determine whether improvements can be made to a caching strategy for the first category of cache requests;

wherein making adjustments to the caching strategy comprises changing the allocation of the first category of cache requests from a first cache to a second cache.

20. A non-transitory computer readable storage medium storing one or more programs, the one or more programs comprising instructions for:

allocating a cache request to a first cache of the distributed cache system based on a marker identifying a mission critical data type associated with a cache object associated with the cache request, wherein the mission critical data type is identified based upon information gathered by monitoring operations concerning a number of cache objects of different types generated by user requests, and wherein the first cache comprises a local cache;

monitoring the caches in the distributed cache system;

when a slowdown in service is detected at a first cache, switching the first cache to a safe cache mode of operation;

periodically performing a test to determine whether the first cache has resumed normal operations;

when the first cache has resumed normal operations, switching the first cache to a normal cache mode of operations.