Virtual multi-cluster clouds

• US 8,712,982 B2
• Filed: 10/21/2011
• Issued: 04/29/2014
• Est. Priority Date: 03/08/2011
• Status: Expired due to Fees

• Current Assignee: Rackspace US Incorporated (Apollo Global Management, Inc.)
• Original Assignee: Rackspace US Incorporated (Apollo Global Management, Inc.)
• Inventors: Holt, Gregory Lee, Gerrard, Clay, Goetz, David Patrick, Barton, Michael
• Primary Examiner(s): Wong, Leslie
• Application Number:
  US13/278,876
• Publication Number:
  US 20120233251A1
• Time in Patent Office: 921 Days
• Field of Search: 707/698, 707/831, 707/959, 709/220, 709/238, 709/240, 709/251
• US Class Current: 707/698
• CPC Class Codes: G06F 11/1004 to protect a block of data words, e.g. CRC or checksum G06F11/1076 takes precedence; security arrangements for protecting computers or computer systems against unauthorized activity G06F21/00
• CPC Class Codes: G06F 11/2097 maintaining the standby controller/processing unit updated initialisation or re-synchronisation thereof G06F11/1658 and subgroups
• CPC Class Codes: G06F 16/1744 using compression, e.g. sparse files
• CPC Class Codes: G06F 16/1748 De-duplication implemented within the file system, e.g. based on file segments de-duplication techniques in storage systems for the management of data blocks G06F3/0641
• CPC Class Codes: G06F 16/178 Techniques for file synchronisation in file systems
• CPC Class Codes: G06F 16/2219 Large Object storage; Management thereof
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: G06F 16/273 Asynchronous replication or reconciliation
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/9566 URL specific, e.g. using aliases, detecting broken or misspelled links
• CPC Class Codes: G06F 2201/85 Active fault masking without idle spares active fault masking without idle spare hardware where processing functionality is redundant G06F11/2035
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0626 Reducing size or complexity of storage systems
• CPC Class Codes: G06F 3/0644 Management of space entities, e.g. partitions, extents, pools
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: H04L 43/106 using time related information in packets, e.g. by adding timestamps
• CPC Class Codes: H04L 65/403 Arrangements for multi-party communication, e.g. for conferences data switching systems for conference H04L12/18; arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities H04M3/56; television conferencing systems H04N7/15
• CPC Class Codes: H04L 67/06 specially adapted for file transfer, e.g. file transfer protocol [FTP]
• CPC Class Codes: H04L 67/1095 : Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
• CPC Class Codes: H04L 67/1097 : for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 67/5651 : Reducing the amount or size of exchanged application data
• CPC Class Codes: H04L 69/04 : Protocols for data compression, e.g. ROHC

Abstract

An improved scalable object storage system allows multiple clusters to work together. Users working with a first cluster, or with a multi-cluster gateway, can ask for services and have the request or data transparently proxied to a second cluster. This gives transparent cross-cluster replication, as well as multi-cluster compute or storage farms based upon spot availability or various provisioning policies. Vendors providing a cloud storage “frontend” can provide multiple backends simultaneously. In one embodiment, a multi-cluster gateway can have a two, three, or higher-level ring that transparently matches an incoming request with the correct cluster. In the ring, a request is first mapped to an abstract “partition” based on a consistent hash function, and then one or more constrained mappings map the partition number to an actual resource. In another embodiment, the multi-cluster gateway is a dumb gateway, and the rings are located only at the cluster level.

Claims

1. A system for coordinating a plurality of clusters, the system comprising:

a multi-cluster gateway, the multi-cluster gateway communicatively coupled to a first cluster and a second cluster, wherein the first cluster includes a first plurality of information processing devices and the second cluster includes a second plurality of information processing devices;

wherein the multi-cluster gateway includes a multi-cluster ring, wherein the multi-cluster ring performs a consistent hashing function responsive to a request to interact with the system, and wherein an output of the consistent hashing function is a partition identifier corresponding to computing resources available at one of the first cluster and the second cluster;

wherein the multi-cluster ring performs a constrained mapping function on the partition identifier, and wherein an output of the constrained mapping function maps the partition identifier to a responsive cluster from one of the first cluster and the second cluster in accordance with a desired storage characteristic; and

wherein the multi-cluster gateway sends a message to the responsive cluster.

2. The system of claim 1, wherein the output of the constrained mapping function further identifies a specific resource within the responsive cluster.

3. The system of claim 1, wherein the message to the responsive cluster is addressed to the specific resource within the responsive cluster.

4. The system of claim 1, wherein the multi-cluster gateway receives a return message from the responsive cluster responsive to the request to interact with the system.

5. The system of claim 1, wherein the multi-cluster gateway further includes a proxy module.

6. The system of claim 5, wherein the proxy module sends a return message from the responsive cluster in response to the request to interact with the system.

7. The system of claim 1, wherein the consistent hashing function is a first-level hashing function and the constrained mapping function is a second-level hashing function, wherein one of the first cluster and the second cluster further comprises a cluster-level ring that performs a third-level hashing function responsive to a message identifying the cluster as a responsive cluster, wherein an output of the third-level hashing function is a cluster-level partition associated with the resources available within the cluster.

8. The system of claim 7, wherein the cluster-level ring performs a fourth-level hashing function, wherein an output of the fourth-level hashing function identifies a specific responsive resource associated with the cluster-level partition.

13. The system of claim 1, wherein the constrained mapping function uses geographic distance to choose which cluster is the responsive cluster.

14. The system of claim 1, wherein the constrained mapping function uses network latency to choose which cluster is the responsive cluster.

17. The system of claim 1, wherein the first cluster is controlled by a first party and the second cluster is controlled by a second party.

19. The system of claim 1, wherein the multi-cluster ring divides the partition identifier into a plurality of parts and performs the constrained mapping function on each part of the plurality of parts, and wherein each output of the constrained mapping function identifies a partition associated with the first cluster or the second cluster.

20. The system of claim 1, wherein the request includes an object and the multi-cluster ring performs the consistent hashing function on an attribute of the object.

21. The system of claim 1, wherein the request includes an object and the multi-cluster ring performs the consistent hashing function on the object.

9. A system for coordinating a plurality of clusters, the system comprising:

a multi-cluster gateway, the multi-cluster gateway communicatively coupled to a first cluster and a second cluster, wherein the first cluster includes a first plurality of information processing devices and the second cluster includes a second plurality of information processing devices;

wherein the multi-cluster gateway sends a message to one of the first cluster, second cluster, or both clusters responsive to a request to interact with the system;

wherein one of the first cluster and the second cluster includes a cluster-level ring, wherein the cluster-level ring performs a consistent hashing function responsive to a message, wherein an output of the consistent hashing function is a partition identifier corresponding to computing resources available at one of the first cluster and the second cluster; and

wherein the cluster-level ring performs a constrained mapping function on the partition identifier, and wherein an output of the constrained mapping function maps the partition identifier to a specific responsive resource associated with the partition in accordance with a desired storage characteristic.

10. The system of claim 9, wherein the multi-level gateway receives a responsive message from a cluster and sends a response to the request.

11. The system of claim 9, wherein only one of the first cluster and the second cluster is a responsive cluster, and wherein a response to the request is sent by the responsive cluster.

12. The system of claim 9, wherein either of the first cluster and the second cluster can be a responsive cluster, and wherein the first cluster to formulate a response sends the response.

15. The system of claim 9, wherein the multi-cluster gateway sends the message to the cluster that is closer geographically to the source of the request.

16. The system of claim 9, wherein the multi-cluster gateway sends the message to the cluster that has lower latency between the cluster and the source of the request.

18. The system of claim 9, wherein the first cluster is controlled by a first party and the second cluster is controlled by a second party.

22. A system for coordinating a plurality of clusters, the system comprising:

a multi-cluster gateway, the multi-cluster gateway communicatively coupled to a first cluster and a second cluster, wherein the first cluster includes a first plurality of information processing devices and the second cluster includes a second plurality of information processing devices;

wherein the multi-cluster gateway includes a multi-cluster ring, wherein the multi-cluster ring provides a mapping between one or more entities stored in the first or second clusters and locations of the one or more entities in one or more storage pools;

wherein the multi-cluster ring performs a consistent hash function responsive to a request to interact with the system, and wherein an output of the consistent hash function is a partition identifier corresponding to a partition that is mapped to a plurality of storage pools of the one or more storage pools;

wherein the multi-cluster ring performs a constrained mapping function on the output of the consistent hash function, and wherein the constrained mapping function ensures that each partition is mapped to a storage pool of the plurality of storage pools in a different zone; and

wherein the multi-cluster gateway sends a message to the responsive cluster.