Modification of an object replica

• US 8,712,975 B2
• Filed: 04/19/2011
• Issued: 04/29/2014
• Est. Priority Date: 03/08/2011
• Status: Expired due to Fees

• Current Assignee: Rackspace US Incorporated (Apollo Global Management, Inc.)
• Original Assignee: Rackspace US Incorporated (Apollo Global Management, Inc.)
• Inventors: Barton, Michael, Reese, Will, Dickinson, John A., Payne, Jay B., Thier, Charles B., Holt, Gregory
• Primary Examiner(s): Wong, Leslie
• Application Number:
  US13/089,487
• Publication Number:
  US 20120233134A1
• Time in Patent Office: 1,106 Days
• Field of Search: 707/610, 707/698, 707/692, 707/747, 707/999.203, 707/999.204
• US Class Current: 707/692
• CPC Class Codes: G06F 16/162 Delete operations erasing in storage systems G06F3/0652
• CPC Class Codes: G06F 16/1837 Management specially adapted to peer-to-peer storage networks topology management mechanisms of peer-to-peer networks H04L67/1042
• CPC Class Codes: G06F 16/219 Managing data history or versioning querying versioned data G06F16/2474; querying temporal data G06F16/2477
• CPC Class Codes: G06F 16/23 Updating
• CPC Class Codes: G06F 16/2322 using timestamps
• CPC Class Codes: H04L 67/1023 based on a hash applied to IP addresses or costs
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

Several different embodiments of a massively scalable object storage system are described. The object storage system is particularly useful for storage in a cloud computing installation whereby shared servers provide resources, software, and data to computers and other devices on demand. In several embodiments, the object storage system includes a ring implementation used to associate object storage commands with particular physical servers such that certain guarantees of consistency, availability, and performance can be met. In other embodiments, the object storage system includes a synchronization protocol used to order operations across a distributed system. In a third set of embodiments, the object storage system includes a metadata management system. In a fourth set of embodiments, the object storage system uses a structured information synchronization system. Features from each set of embodiments can be used to improve the performance and scalability of a cloud computing object storage system.

Claims

1. A method for managing data items in a distributed storage pool, comprising:

providing a plurality of physical storage pools, each storage pool including a plurality of storage nodes coupled to a network, each storage node further providing a non-transitory computer readable medium for data storage;

storing a first replica of a data item in a first physical storage pool, and storing a second replica of the data item in a second physical storage pool;

in response to receiving a modification instruction for the data item, selectively modifying the first replica of the data item, creating a first modification sentinel file and storing the first modification sentinel file in the first physical storage pool;

in response to encountering the first modification sentinel file during a data item replication process, modifying the second replica of the data item and creating a second modification sentinel file in the second physical storage pool; and

applying a hash function to a name of the first replica of the data item, wherein the first modification sentinel file includes a first output of the hash function.

2. The method of claim 1, further comprising:

waiting for a configurable time; and

deleting the first modification sentinel file in the first physical storage pool, and deleting the second modification sentinel file in the second physical storage pool.

3. The method of claim 1, wherein the modification instruction is a delete instruction, wherein the selective modification of a replica of the data item is performed by removing each data item from its physical storage pool;

and wherein the modification sentinel file is a file deletion marker.

4. The method of claim 1, wherein the modification instruction is a purge instruction, wherein the selective modification of a replica of the data item is performed by deleting and marking as free the physical drive space associated with each data item;

and wherein the modification sentinel file is a file purge marker.

5. The method of claim 1, wherein the modification instruction is a version instruction, wherein the selective modification of a replica of the data item is performed by saving a copy of the existing data item in a second location, creating a new copy of each data item;

and wherein the modification sentinel file is a version marker.

6. The method of claim 1, wherein the modification instruction is a time-to-live (ttl) instruction, wherein the selective modification of a replica of the data item is performed by selectively checking a third location for an update if the time since the last update is greater than the time indicated in the ttl instruction, and copying the data item from the third location if the data item in the third location has been changed;

and wherein the modification sentinel file is a ttl marker.

7. The method of claim 2, wherein the configurable time is longer than the worst-case replication propagation time between the first physical storage pool and the second physical storage pool.

8. The method of claim 1, wherein the first modification sentinel file is a 0 byte file.

9. The method of claim 1, further comprising:

applying the hash function to a name of the second replica of the data item, wherein the second modification sentinel file includes a second output of the hash function, and the first output is the same as the second output.

10. A system for out-of-band communication of object storage metadata, the system comprising:

a distributed storage system coupled to a network, the distributed storage including a first storage pool and a second storage pool from a plurality of storage pools, the first and second storage pools each including at least one processor, a computer readable medium, and a communications interface;

wherein the first storage pool includes a first replica of a data item, and the second storage pool includes a second replica of the data item;

an object service responsive to modification instructions and responsive to an out-of-band instruction; and

a replicator that creates the second replica of the data item in the second storage pool and responsive to encountering a first modification sentinel file during a data item replication process;

wherein the object service selectively modifies the first replica of the data item, creates the first modification sentinel file, applies a hash function to a name of the first replica of the data item, and stores the first modification sentinel file in the first physical storage pool, wherein the first modification sentinel file includes a first output of the hash function; and

wherein the replicator modifies the second replica of the data item and creates a second modification sentinel file in the second physical storage pool.

11. The system of claim 10, further comprising:

a timer triggering an action at a configurable time; and

wherein the replicator includes a triggerable action that deletes the first modification sentinel file in the first physical storage pool and deletes the second modification sentinel file in the second physical storage pool.

12. The system of claim 10, wherein the out-of-band instruction is a delete instruction, wherein the object service modifies the first replica of the data item and the replicator modifies the second replica of the data item by removing the respective data items from their respective physical storage pools;

and wherein the modification sentinel files are file deletion markers.

13. The system of claim 10, wherein the out-of-band instruction is a purge instruction, wherein the object service modifies the first replica of the data item and the replicator modifies the second replica of the data item by deleting and marking as free the respective data items from their respective physical storage pools;

and wherein the modification sentinel files are file purge markers.

14. The system of claim 10, wherein the out-of-band instruction is a version instruction, wherein the object service modifies the first replica of the data item and the replicator modifies the second replica of the data item by saving a copy of the existing data item in a second location in their respective physical storage pools;

and wherein the modification sentinel files are version markers.

15. The system of claim 10, wherein the out-of-band instruction is a time-to-live (ttl) instruction, wherein the object service modifies the first replica of the data item and the replicator modifies the second replica of the data item by checking a third location for an update if the time since the last update is greater than the time indicated in the ttl instruction, and copying the data item from the third location if the data item in the third location has been changed;

and wherein the modification sentinel files are ttl markers.

16. The system of claim 10, wherein the configurable time is longer than the worst-case replication propagation time between the first physical storage pool and the second physical storage pool.

17. A non-transitory computer readable medium containing executable instructions, which when executed on a processor:

at a first time, run a replication procedure that takes a first copy of a data item in a first location and makes an identical second copy of the data item in a second location;

at a second time, run an out-of-band modification procedure to selectively modify the first copy of the data item, create a first modification sentinel file, apply a hash function to a name of the first copy of the data item, and store the first modification sentinel file in the first location, wherein the first modification sentinel file includes a first output of the hash function;

at a third time, change the execution of the replication procedure to modify the second copy of the data item and create a second modification sentinel file in the second location.

18. The non-transitory computer readable medium of claim 17, further comprising instructions that, at a fourth time, delete the first modification sentinel file in the first location and deleting the second modification sentinel file in the second location.

19. The non-transitory computer readable medium of claim 17, wherein the out-of-band modification procedure deletes the first copy of the data item in the first location;

wherein the execution of the replication procedure is changed to delete the second copy of the data item in the second location; and

wherein the modification sentinel files are file deletion markers.

20. The non-transitory computer readable medium of claim 17, wherein the out-of-band modification procedure creates an immutable copy of the data item in the first location and marks it with a version identification tag;

wherein the execution of the replication procedure is changed to create an immutable copy of the data item in the second location and mark it with the version identification tag; and

wherein the modification sentinel files are version markers.

21. The non-transitory computer readable medium of claim 17, wherein the out-of-band modification procedure:

compares the last file modification time of the first copy of the data item with a time-to-live (ttl) value;

checks a third location for an updated version of the data item if the last file modification time is greater than the ttl value;

copies the updated version in the first location if the updated version is newer than first copy of the data item;

updates the file modification time on the first copy of the data item if the updated version is the same as the first copy of the data item;

wherein the execution of the replication procedure is changed to modify the contents and last file modification time of the second copy of the data item to match the contents and last file modification time on the first copy of the data item; and

wherein the modification sentinel files are ttl markers.

22. The non-transitory computer readable medium of claim 18, wherein the span between the third time and the fourth time is longer than the worst-case replication propagation time between the first location and the second location.