Large scale data storage system with fault tolerance

• US 8,713,282 B1
• Filed: 03/31/2011
• Issued: 04/29/2014
• Est. Priority Date: 03/31/2011
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: Rajimwale, Abhishek, Hsu, Windsor W.
• Primary Examiner(s): Bradley, Matthew
• Assistant Examiner(s): GOLDSCHMIDT, CRAIG S
• Application Number:
  US13/077,679
• Time in Patent Office: 1,125 Days
• Field of Search: 711/173
• US Class Current: 711/173
• CPC Class Codes: G06F 16/1748 De-duplication implemented within the file system, e.g. based on file segments de-duplication techniques in storage systems for the management of data blocks G06F3/0641
• CPC Class Codes: G06F 16/184 implemented as replicated file system
• CPC Class Codes: G06F 3/0644 Management of space entities, e.g. partitions, extents, pools
• CPC Class Codes: G06F 9/5077 Logical partitioning of resources; Management or configuration of virtualized resources specific details on emulation or internal functioning of virtual machines G06F9/455

Abstract

According to one embodiment, a large scale data storage system with fault tolerance is described. In one example, a system includes a plurality of storage partitions, each partition including a portion of a large scale data store, and metadata corresponding to the portion of the large scale data store. At least one partition includes namespace metadata for the large scale data store including namespace data for a portion of the large scale data store outside the at least one partition.

Claims

1. An apparatus comprising:

a plurality of storage partitions, each partition including;

a portion of a large scale data store; and

metadata corresponding to the portion of the large scale data store,

the apparatus configured to;

select one of the plurality of storage partitions as a target partition;

write data of a large scale data store to be backed up into an active partition, the active partition being one of the plurality of storage partitions;

write namespace data for the large scale data store into the active partition, the namespace data including namespace data for a portion of the large scale data store outside the active partition;

back up data from the active partition into the target partition;

deduplicate the data written into the target partition within the partition but not across other partitions;

seal the data written to the target partition against writing any additional data;

select a new partition of the plurality of storage partitions that is not sealed as a new target partition; and

back up new data of the large scale data store from the active partition to the new target partition.

2. A computer-implemented method for configuring storage partitions of a file storage system, the method comprising:

selecting a storage partition as a target partition;

writing data of a large scale data store to be backed up into an active partition;

writing namespace data for the large scale data store into the active partition, the namespace data including namespace data for a portion of the large scale data store outside the active partition;

backing up data from the active partition into the target partition;

deduplicating the data written into the target partition within the partition but not across other partitions;

sealing the data written to the target partition against writing any additional data;

selecting a new partition that is not sealed as a new target partition; and

backing up new data of the large scale data store from the active partition to the new target partition.

3. The method of claim 2, further comprising grouping a plurality of storage nodes into a single logical entity to form a partition.

4. The method of claim 3, wherein the storage nodes comprise physical storage devices.

5. The method of claim 2 wherein sealing the data comprises writing an identifier to the active partition.

6. The method of claim 2, wherein sealing the data comprises copying the namespace data from the active partition into the target partition.

7. A computer-readable storage medium having instructions stored therein, which when executed by a computer, cause the computer to perform a method for configuring storage partitions of a file storage system, the method comprising:

selecting a storage partition as a target partition;

writing data of a large scale data store to be backed up into an active partition;

writing namespace data for the large scale data store into the active partition, the namespace data including namespace data for a portion of the large scale data store outside the active partition;

backing up data from the active partition into the target partition;

deduplicating the data written into the target partition within the partition but not across other partitions;

sealing the data written to the target partition against writing any additional data;

selecting a new partition that is not sealed as a new target partition; and

backing up new data of the large scale data store from the active partition to the new target partition.

8. The computer-readable storage medium of claim 7, wherein the method further comprises grouping a plurality of storage nodes into a single logical entity to form a partition.

9. The computer-readable storage medium of claim 8, wherein the storage nodes comprise physical storage devices.

10. The computer-readable storage medium of claim 7, wherein sealing the data comprises writing an identifier to the active partition.

11. The computer-readable storage medium of claim 7, wherein sealing the data comprises copying the namespace data from the active partition into the target partition.

12. The computer-readable storage medium of claim 7, wherein backing up data comprises backing up data during writing data into the active partition.

13. The computer-readable storage medium of claim 12, the method further comprising copying namespace data to the target archive at specified intervals during backing up data.