Distributed caching cluster client configuration

• US 9,602,614 B1
• Filed: 11/26/2012
• Issued: 03/21/2017
• Est. Priority Date: 11/26/2012
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Shankaran, Nishanth, Shou, Dong, Sbisa, Clint Joseph, Thiruchi Loganathan, Rajaprabhu, Krishnamoorthy, Shyam, Arya, Rajat
• Primary Examiner(s): Christensen, Scott B
• Application Number:
  US13/685,607
• Time in Patent Office: 1,576 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 12/0806 Multiuser, multiprocessor or multiprocessing cache systems
• CPC Class Codes: G06F 12/0868 Data transfer between cache memory and other subsystems, e.g. storage devices or host systems
• CPC Class Codes: G06F 15/177 Initialisation or configuration control processor initialisation G06F9/4405
• CPC Class Codes: G06F 16/184 implemented as replicated file system
• CPC Class Codes: G06F 2212/285 Redundant cache memory
• CPC Class Codes: G06F 2212/314 In storage network, e.g. network attached cache
• CPC Class Codes: H04L 41/0816 the condition being an adaptation, e.g. in response to network events
• CPC Class Codes: H04L 41/0856 by backing up or archiving configuration information
• CPC Class Codes: H04L 43/0858 One way delays
• CPC Class Codes: H04L 43/087 Jitter
• CPC Class Codes: H04L 47/125 by balancing the load, e.g. traffic engineering
• CPC Class Codes: H04L 67/34 involving the movement of software or configuration parameters network booting or remote initial program loading [RIPL] G06F9/4416
• CPC Class Codes: H04L 67/568 Storing data temporarily at an intermediate stage, e.g. caching

Abstract

A cache or other cluster is configuration-aware such that initialization and changes to the underlying structure of the cluster can be dynamically updated for use by a client. A client may use a client driver as an intermediary that is responsible for managing the communication with the cluster. For example, a client driver may resolve an alias from a static configuration endpoint to a storage node. The client driver may request an initial configuration from the storage node and then update configuration from one or more storage nodes that store current configuration of the cluster.

Claims

1. A computer-implemented method for a distributed memory caching configuration, comprising:

receiving, by a client device from a first memory caching node in a cache cluster, a current configuration of the cache cluster, the cache cluster comprising a set of memory caching nodes, each of the memory caching nodes in the set of memory caching nodes comprising a reserved memory space that stores the current configuration, the set of memory caching nodes being part of a first network, the client device being part of a second network that is separate from the first network, and the reserved memory space being exempt from rules of a caching protocol;

defining, by the client device, a key space for each memory caching node in the set of memory caching nodes in the cache cluster based at least in part on the configuration;

requesting, by the client device utilizing the caching protocol, a first set of values corresponding to a first set of keys from the cache cluster using the key space to determine which memory caching node to contact for each key of the first set of keys;

monitoring, by the client device, responses from the cache cluster;

determining, from the monitoring, that a latency of a response from a memory caching node has increased beyond a threshold;

transmitting, by the client device utilizing an extension of the caching protocol, a provisioning request for a new memory caching node to be added to the cache cluster, the transmitting of the provisioning request being based at least in part on the determining that the latency has increased beyond the threshold, and the extension of the caching protocol being configured to modify the reserved memory space;

receiving, by the client device from a second memory caching node in the cache cluster, an updated configuration of the cache cluster identifying changes to the cache cluster, the second memory caching node being different from the first memory caching node in the cache cluster, and the updated configuration being received independent from a request by the client device;

determining a revised key space for each memory caching node in the set of memory caching nodes in the cache cluster affected by the updated configuration; and

requesting a second set of values corresponding to a second set of keys from the cache cluster using the revised key space to determine which memory caching node to contact for each key of the second set of keys.

2. The computer-implemented method of claim 1, wherein the defining the key space for each memory caching node in the set of memory caching nodes further comprises determining one or more caching angles to assign to each memory caching node in the set of memory caching nodes.

3. The computer-implemented method of claim 1, wherein defining the key space for each memory caching node in the set of memory caching nodes further comprises defining a hash function to be used with keys stored in the cache cluster.

4. The computer-implemented method of claim 1, wherein receiving the current configuration of the cache cluster from the first memory caching node in the cache cluster further comprises:

resolve an alias provided by a configuration endpoint to the first memory caching node; and

requesting a configuration from the first memory caching node.

5. The computer-implemented method of claim 1, wherein the first memory caching node and the second memory caching node are peers in the cache cluster, and wherein the first memory caching node and the second memory caching node are communicatively coupled.

6. The computer-implemented method of claim 1, further comprising:

accessing, by the client device utilizing the extension of the caching protocol, a plurality of configurations of the cache cluster, the plurality of configurations being stored in a plurality of memory caching nodes of the cache cluster;

determining a newest configuration from the plurality of configurations;

determining that the updated configuration differs from the newest configuration; and

loading the newest configuration as the updated configuration based at least in part on determining that the updated configuration differs from the newest configuration.

7. The computer-implemented method of claim 1, wherein the caching protocol comprises a memcached protocol.

8. A computer-implemented method for communicating with a distributed caching system, comprising:

receiving, by a client device from a storage node in a cache cluster, a configuration describing a set of storage nodes in the cache cluster, the set of storage nodes individually comprising a reserved memory space that stores a current configuration of the cache cluster, the set of storage nodes being part of a first network, the client device being a part of a second network that is separate from the first network, and the reserved memory space being exempt from rules of a caching protocol;

determining, by the client device, that particular data of the set of storage nodes is accessed with a frequency above a threshold;

transmitting, by the client device utilizing an extension of the caching protocol, a provisioning request for a new storage node to be added to the cache cluster, the transmitting of the provisioning request being based at least in part on determining that the particular data is accessed with the frequency above the threshold, and the extension of the caching protocol being configured to modify the reserved memory space;

receiving, by the client device from a different storage node in the cluster, an updated configuration describing the set of storage nodes in the cluster, the set of storage nodes comprising the new storage node;

receiving a request for a key; and

determining, by the client device, an assignment of the key to the storage node in the cluster based at least in part on the configuration.

9. The computer-implemented method of claim 8, further comprising sending a request to the storage node to retrieve an updated configuration from a reserved configuration space in the storage node.

10. The computer-implemented method of claim 8, further comprising receiving a configuration update request from the storage node when the cluster has been updated.

11. The computer-implemented method of claim 8, further comprising:

requesting, by the client device, a new storage node be provisioned; and

assigning, by the client device, the storage node a portion of a hashing function result.

12. A computer system for providing communication with a cache cluster, comprising:

one or more computing resources having one or more processors and memory including executable instructions that, when executed by the one or more processors, cause the one or more processors to implement at least;

a driver of a client device of a first network that updates a configuration retrieved from a plurality of memory caching nodes of a cache cluster that utilizes a second network that is separate from the first network, each memory caching node of the cache cluster storing the configuration in a reserved memory space of the each memory caching node that is exempt from rules of a caching protocol, the configuration identifying the plurality of memory caching nodes in the cache cluster, the driver providing access to data within the cache cluster based at least in part on the configuration, the driver being configured to transmit a provisioning request for a new memory caching node to be added to the cache cluster when it is determined that a latency of a response from a memory caching node has increased beyond a threshold, the driver being configured to transmit the provisioning request utilizing an extension of the caching protocol, and the extension being configured to modify the reserved memory space; and

an application configured to communicate with the cache cluster through the driver such that changes to the cache cluster are resolved by the driver.

13. The computer system of claim 12, further comprising a database that provides requested data when the data from the cache cluster does not contain the requested data.

14. The computer system of claim 12, wherein the plurality of memory caching nodes further comprise random access memory to hold cached data from the database.

15. The computer system of claim 12, wherein the application continues to operate while the driver updates configuration.

16. The computer system of claim 12, further comprising a configuration endpoint providing an alias to at least one memory caching node in the cache cluster.

17. One or more non-transitory computer-readable storage media having collectively stored thereon executable instructions that, when executed by one or more processors of a computer system, cause the computer system to at least:

receive, by a client device of a first network, from a first memory caching node in a cache cluster of a second network that is separate from the first network, a configuration describing a set of memory caching nodes in the cache cluster, the set of memory caching nodes individually storing configuration data in a reserved memory space, and the reserved memory space being exempt from rules of a caching protocol,

determine, by the client device, a first assignment of results of a hashing function to the set of memory caching nodes based at least in part on the configuration;

transmit, by the client device utilizing an extension of the caching protocol, a provisioning request for a new memory caching node to be added to the cache cluster, the transmitting of the provisioning request being based at least in part on determining that a latency of a response from the first memory caching node has increased beyond a threshold, the extension being configured to modify the reserved memory space;

receive, by the client device from a second memory caching node in the cache cluster, an updated configuration describing the set of memory caching nodes having been modified, the first memory caching node being different than the second memory caching node, the updated configuration being received independent from a request; and

determine a second assignment of results of the hashing function to the set of memory caching nodes based at least in part on the updated configuration.

18. The one or more non-transitory computer-readable storage media of claim 17, wherein the executable instructions that determine the second assignment of results include further instructions that cause the computer system to determine the second assignment of results based at least in part on a deletion of a node from the set of memory caching nodes.

19. The one or more non-transitory computer-readable storage media of claim 17, wherein the executable instructions that determine the second assignment of results include further instructions that cause the computer system to determine the second assignment of results based at least in part on an addition of a node to the set of memory caching nodes.

20. The one or more non-transitory computer-readable storage media of claim 17, wherein the executable instructions that determine the second assignment of results include further instructions that cause the computer system to determine the second assignment of results based at least in part on a replacement of a node from the set of memory caching nodes.

21. The one or more non-transitory computer-readable storage media of claim 20, including further executable instructions that, when executed, cause the computer system to at least:

service requests from an application for keys stored by the set of memory caching nodes; and

switch use of the configuration to use of the updated configuration without notifying the application.

22. The one or more non-transitory computer-readable storage media of claim 17, including further instructions that cause the computer system to periodically request stored configuration from the second memory caching node.

23. The one or more non-transitory computer-readable storage media of claim 17, wherein the first memory caching node is the second memory caching node.

24. The one or more non-transitory computer-readable storage media of claim 17, wherein the executable instructions that determine the first assignment of results of the hashing function include further instructions that cause the computer system to receive an indication from a management system of one or more assignments of memory caching nodes for use with a consistent hashing method.