Cloud storage system with distributed metadata

• US 8,533,231 B2
• Filed: 08/12/2011
• Issued: 09/10/2013
• Est. Priority Date: 08/12/2011
• Status: Active Grant

• Current Assignee: Nexenta by DDN, Inc.
• Original Assignee: Nexenta Systems Incorporated (DataDirect Networks Incorporated)
• Inventors: Aizman, Alexander, Bestler, Caitlin
• Primary Examiner(s): Lu, Charles
• Application Number:
  US13/209,342
• Publication Number:
  US 20130041872A1
• Time in Patent Office: 760 Days
• Field of Search: 707/827, 707/793
• US Class Current: 707/793
• CPC Class Codes: G06F 16/182 Distributed file systems

Abstract

A method and system is disclosed for providing a cloud storage system supporting existing APIs and protocols. The method of storing cloud storage system (CSS) object metadata separates object metadata that describes each CSS object as a collection of named chunks with chunk locations specified as a separate part of the metadata. Chunks are identified using globally unique permanent identifiers that are never re-used to identify different chunk payload. While avoiding the bottleneck of a single metadata server, the disclosed system provides ordering guarantees to clients such as guaranteeing access to the most recent version of an object. The disclosed system also provides end-to-end data integrity protection, inline data deduplication, configurable replication, hierarchical storage management and location-aware optimization of chunk storage.

Claims

1. A computer implemented method for avoiding single metadata server bottlenecks on processing cloud storage system (CSS) object metadata, the computer executing the following operations:

dividing a metadata storage for a cloud storage system between Object metadata that encodes information on an object and versions of the object and Chunk metadata that tracks the locations of chunks;

identifying a reference to each chunk in the object metadata using a globally unique permanent chunk identifier which is never re-used to identify a different payload;

maintaining the CSS object metadata in a shared global name space, wherein the CSS object metadata is distributed over a plurality of object metadata servers (OMDS);

wherein one or more Enhanced Chunk Servers (ECS) or one or more Chunk Metadata Servers (CMDS) encode a chunk as a file named based upon a chunk class of storage and the permanent chunk identifier, wherein data of the file holds a compressed payload for a respective chunk as data, and the metadata of the file encodes the identifiers of zero or more other ECSs known to hold the payload for the respective chunk;

wherein one of the ECSs will acknowledge each valid put of a chunk with a chunk cookie that encodes the permanent chunk identifier for the chunk, the length of the chunk data after compression and context information supplied by a Cloud Storage Access Module (CSAM), usable by an OMDS to validate a commit of the object; and

wherein an ECS that receives a request to put a chunk that is identified with a deferred chunk identifier will;

validate that a checksum encoded in the deferred chunk identifier is valid;

determine the permanent chunk identifier for the chunk payload;

store the chunk under the permanent chunk identifier, responsive to determining that the permanent chunk identifier is not already present; and

provide a chunk cookie as an acknowledgement.

2. The computer implemented method of claim 1, wherein end-to-end data integrity is ensured by one or more enhanced chunk servers (ECS) not accepting a chunk for storage when the chunk identifier is inconsistent with the chunk payload.

3. The computer implemented method of claim 1, wherein at least one of the OMDSs archives CSS object metadata to external storage services so that the CSS object metadata can be expunged from the local file systems and restored from an external storage system when needed.

4. The computer implemented method of claim 1, wherein one or more Chunk Metadata Servers (CMDS) archives chunks to external storage services so that they can be expunged from one or more ECSs and restored from the external storage system when needed.

5. The computer implemented method of claim 1, where each OMDS maintains the object metadata for each CSS object as metadata file within a local file system using a file name derived from a fully qualified object name.

6. The computer implemented method of claim 1, wherein the OMDSs create a global name space by publishing their object metadata files using distributed network attached storage (NAS) protocol.

7. The computer implemented method of claim 1, wherein a collective CMDS is responsible for storing and retrieving chunk metadata.

8. The computer implemented method of claim 1, wherein a permanent chunk identifier is used to permanently identify all persistently stored chunks, wherein the permanent chunk identifier is based upon a cryptographic hash algorithm that excludes the possibility of a preimage attack.

9. The computer implemented method of claim 1, wherein a permanent chunk identifier comprises a hash algorithm with a unique identifier assigned by either a CSAM or an ECS.

10. The computer implemented method of claim 1, wherein the identification of a chunk is deferred during a put transaction using a deferred chunk identifier to act as a temporary placeholder for a permanent chunk identifier.

11. The computer implemented method of claim 1, wherein each CSS object at any point in time has a designated home OMDS which is solely responsible for assigning new version numbers.

12. The computer implemented method of claim 11, wherein each CSS object is represented by an object directory in an OMDS local file system with a directory name derived from the fully qualified object name.

13. The computer implemented method of claim 12, wherein each version of a CSS object is represented by an object metadata file within the object directory with a name that encodes the version and contents that encode the set of permanent chunk identifiers to reconstruct the object and the required sequencing information for respective chunks.

14. The computer implemented method of claim 1, wherein each OMDS maintains transactional consistency of metadata derived from information in object metadata files.

15. The computer implemented method of claim 1, wherein an OMDS acts as a NAS client to access metadata files in the OMDS shared global name space.

16. The computer implemented method of claim 1, wherein a request by a requester to get a chunk that has no local payload stored for the chunk is answered with a chunk referral response redirecting the requester to another ECS that does have the chunk payload stored.

17. The computer implemented method of claim 1, wherein one of the ECSs issues a chunk referral response when it has determined that processing a get request itself would cause excessive delay.

18. The computer implemented method of claim 7, wherein an ECS will report an error when the payload read for a chunk is inconsistent with its chunk identifier.