System and method for providing long-term storage for data

• US 20100332452A1
• Filed: 06/25/2009
• Published: 12/30/2010
• Est. Priority Date: 06/25/2009
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: Data Domain, Inc. (Dell Technologies Inc.)
• Inventors: Patterson, R. Hugo, Hsu, Windsor W.
• Granted Patent: US 8,635,184 B2
• Time in Patent Office: Days
• US Class Current: 707/640
• CPC Class Codes: G06F 11/2094 Redundant storage or storage space G06F11/2056 takes precedence
• CPC Class Codes: G06F 16/1748 De-duplication implemented within the file system, e.g. based on file segments de-duplication techniques in storage systems for the management of data blocks G06F3/0641
• CPC Class Codes: G06F 16/185 Hierarchical storage management [HSM] systems, e.g. file migration or policies thereof details of archiving G06F16/11
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0641 De-duplication techniques
• CPC Class Codes: G06F 3/0647 Migration mechanisms
• CPC Class Codes: G06F 3/0685 Hybrid storage combining heterogeneous device types, e.g. hierarchical storage, hybrid arrays
• CPC Class Codes: G06F 3/0689 Disk arrays, e.g. RAID, JBOD

Abstract

A system for storing files comprises a processor and a memory. The processor is configured to break a file into one or more segments; store the one or more segments in a first storage unit; and add metadata to the first storage unit so that the file can be accessed independent of a second storage unit, wherein a single namespace enables access for files stored in the first storage unit and the second storage unit. The memory is coupled to the processor and configured to provide the processor with instructions

Claims

1. A system for storing files, comprising:

a processor configured to;

break a file into one or more segments;

store the one or more segments in a first storage unit; and

add metadata to the first storage unit so that the file can be accessed independent of a second storage unit, wherein a single namespace enables access for files stored in the first storage unit and the second storage unit; and

a memory coupled to the processor and configured to provide the processor with instructions.

2. A system as in claim 1, wherein the first storage unit includes metadata information to provide access to files stored in the first storage unit and the second storage unit.

3. A system as in claim 1, wherein the first storage unit comprises a deduplication domain.

4. A system as in claim 1, wherein in the event that the first storage unit is full, the processor is further configured to store the one or more segments in a third storage unit.

5. A system as in claim 1, wherein according to a transfer protocol, the processor is further configured to identify a transfer file to be transferred according to a migration policy and to transfer a set of segments comprising the transfer file to a third storage unit.

6. A system as in claim 5, wherein the transfer protocol comprises a rule for transferring the set of segments in the event that the first storage unit is above a predetermined capacity limit.

7. A system as in claim 5, wherein the transfer protocol comprises a rule for transferring the set of segments according to a schedule.

8. A system as in claim 5, wherein the transfer protocol comprises a rule for transferring the set of segments in the event that the first storage unit is not being accessed.

9. A system as in claim 5, wherein transferring includes storing in the third storage unit metadata associated with the transfer file, enabling the transfer file to be accessed from the third storage unit.

10. A system as in claim 5, wherein the migration policy comprises a rule for identifying a file in the event that the file has not been accessed for a predetermined period of time.

11. A system as in claim 5, wherein the migration policy comprises a rule for identifying a file in the event that the file has been modified for a predetermined period of time.

12. A system as in claim 10, wherein the predetermined period of time is determined based at least in part on the type of file.

13. A system as in claim 5, wherein the third storage unit is powered down while the second storage unit remains operational for accessing files stored on the second storage unit.

14. A system as in claim 13, wherein the third storage unit is powered down when the third storage unit is predicted to be idle for some time.

15. A system as in claim 13, wherein powering down comprises one or more of the following:

powering off some electronics of a disk in the third storage unit, spinning down a disk, powering down a disks, and powering down electronics in the third storage unit.

16. A system as in claim 1, wherein the processor is further configured to update the metadata in the first storage unit when an indication is received to delete a file.

17. A system as in claim 16, wherein segments associated only with deleted file are discarded during a space reclamation process.

18. A system as in claim 1, wherein the first storage unit is replicated by copying stored segments and metadata.

19. A system as in claim 1, wherein breaking the file into one or more segments comprises determining segment boundaries.

20. A system as in claim 19, wherein segment boundaries are determined based at least in part on the content of the file.

21. A system as in claim 20, wherein segment boundaries are determined based at least in part on a value calculated using a function applied to at least a portion of the file.

22. A method for storing data, comprising:

breaking a file into one or more segments;

storing the one or more segments in a first storage unit; and

adding metadata to the first storage unit so that the file can be accessed independent of a second storage unit, wherein a single namespace enables access for files stored in the first storage unit and the second storage unit.

23. A computer program product for storing data, the computer program product being embodied in a computer readable storage medium and comprising computer instructions for:

breaking a file into one or more segments;

storing the one or more segments in a first storage unit; and

adding metadata to the first storage unit so that the file can be accessed independent of a second storage unit, wherein a single namespace enables access for files stored in the first storage unit and the second storage unit.