System and method for maintaining consistent points in file systems

• US 9,104,616 B1
• Filed: 07/24/2014
• Issued: 08/11/2015
• Est. Priority Date: 03/31/2011
• Status: Active Grant

• Current Assignee: EMC Corporation (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: Mitra, Soumyadeb, Hsu, Windsor W.
• Primary Examiner(s): Tseng, Cheng-Yuan
• Assistant Examiner(s): Goldschmidt, Craig
• Application Number:
  US14/340,363
• Time in Patent Office: 383 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 11/1435 using file system or storage system metadata
• CPC Class Codes: G06F 11/1446 Point-in-time backing up or restoration of persistent data
• CPC Class Codes: G06F 11/1448 Management of the data involved in backup or backup restore
• CPC Class Codes: G06F 16/11 File system administration, e.g. details of archiving or snapshots file system backup G06F11/14
• CPC Class Codes: G06F 16/113 Details of archiving lifecycle management in storage systems G06F3/0649; backup systems G06F11/1446
• CPC Class Codes: G06F 16/1748 De-duplication implemented within the file system, e.g. based on file segments de-duplication techniques in storage systems for the management of data blocks G06F3/0641
• CPC Class Codes: G06F 16/185 Hierarchical storage management [HSM] systems, e.g. file migration or policies thereof details of archiving G06F16/11
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity
• CPC Class Codes: G06F 2201/82 Solving problems relating to consistency ensuring consistency in mirrored systems G06F11/2064
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data
• CPC Class Codes: G06F 3/0647 Migration mechanisms
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

According to one embodiment, in response to a request to write a prime segment of a file system of a storage system, one or more of the storage units are identified based on a prime segment write-map (PSWM). The PSWM includes storage unit identifiers (IDs) identifying the storage units to which a new prime segment should be written. The prime segment is then written in the one or more storage units identified from the PSWM, without writing the prime segment to a remainder of the storage units. The first prime segment is one of a plurality of prime segments collectively representing a prime. The prime contains metadata representing a consistent point of data stored in the file system, including a reference to a root of the file system. The first prime segment can be utilized to restore the file system back to a state represented by the consistent point.

Claims

1. A computer-implemented method for maintaining a consistent point of a file system, the method comprising:

receiving a request to write a first prime segment of a file system of a storage system, the storage system having a plurality of storage units that archive data from one or more clients;

retrieving one or more storage unit identifiers from a prime segment write-map (PSWM), the one or more storage unit identifiers (IDs) identifying one or more of the storage units to which a new prime segment should be written, the storage unit IDs including a first storage unit ID identifying a first storage unit and a second storage unit ID identifying a second storage unit; and

writing the first prime segment in the first storage unit and the second storage unit based on the first storage unit ID and second storage unit ID respectively, without writing the first prime segment to a remainder of the storage units, wherein the first prime segment is one of a plurality of prime segments collectively representing a prime, the prime containing metadata representing a consistent point of data stored in the file system, including a reference to a root of the file system, and wherein the first prime segment is utilized to restore the file system back to a state represented by the consistent point.

2. The method of claim 1, further comprising maintaining the PSWM within a segment storage engine of the storage system that is configured to segment a data file into a plurality of deduplicated segments before being stored in the storage system.

3. The method of claim 2, wherein each of the storage units is configured to store segments of data files, and wherein the segments are deduplicated within the respective storage unit.

4. The method of claim 1, wherein each of the storage units is configured to store data associated with one of a plurality of file system partitions.

5. The method of claim 4, wherein the file system partitions collectively represent a single logical file system of the storage system.

6. The method of claim 5, wherein each of the storage units maintains metadata having sufficient information to enable the corresponding file system partition to be self-identifiable and configurable.

7. The method of claim 6, wherein each file system partition can be individually started up and removed without having a major impact on operations of remaining file system partitions of the logical file system.

8. The method of claim 1, wherein the PSWM is stored in a persistent storage location of the storage system, and wherein in the event that the file system crashes, the PSWM is recovered from the last known installable prime during restart of the storage system.

9. A non-transitory machine-readable medium having instructions stored therein, which when executed by a processing system, cause the processing system to perform operations for maintaining a consistent point of a file system, the operations comprising:

receiving a request to write a first prime segment of a file system of a storage system, the storage system having a plurality of storage units that archive data from one or more clients;

retrieving one or more storage unit identifiers from a prime segment write-map (PSWM), the one or more storage unit identifiers (IDs) identifying one or more of the storage units to which a new prime segment should be written, the storage unit IDs including a first storage unit ID identifying a first storage unit and a second storage unit ID identifying a second storage unit; and

writing the first prime segment in the first storage unit and the second storage unit based on the first storage unit ID and second storage unit ID respectively, without writing the first prime segment to a remainder of the storage units, wherein the first prime segment is one of a plurality of prime segments collectively representing a prime, the prime containing metadata representing a consistent point of data stored in the file system, including a reference to a root of the file system, and wherein the first prime segment is utilized to restore the file system back to a state represented by the consistent point.

10. The non-transitory machine-readable medium of claim 9, wherein the operations further comprise maintaining the PSWM within a segment storage engine of the storage system that is configured to segment a data file into a plurality of deduplicated segments before being stored in the storage system.

11. The non-transitory machine-readable medium of claim 10, wherein each of the storage units is configured to store segments of data files, and wherein the segments are deduplicated within the respective storage unit.

12. The non-transitory machine-readable medium of claim 9, wherein each of the storage units is configured to store data associated with one of a plurality of file system partitions.

13. The non-transitory machine-readable medium of claim 12, wherein the file system partitions collectively represent a single logical file system of the storage system.

14. The non-transitory machine-readable medium of claim 13, wherein each of the storage units maintains metadata having sufficient information to enable the corresponding file system partition to be self-identifiable and configurable.

15. The non-transitory machine-readable medium of claim 14, wherein each file system partition can be individually started up and removed without having a major impact on operations of remaining file system partitions of the logical file system.

16. The non-transitory machine-readable medium of claim 9, wherein the PSWM is stored in a persistent storage location of the storage system, and wherein in the event that the file system crashes, the PSWM is recovered from the last known installable prime during restart of the storage system.

17. A storage system, comprising:

a processor;

a memory;

a plurality of storage units to store segments of data files of a file system;

a prime segment write-map (PSWM) maintained by in the memory to store one or more storage unit identifiers (IDs) identifying one or more of the storage units to which a new prime segment should be written; and

an archiver executed from the memory by the processor, in response to a request to write a first prime segment of the file system, configured to retrieve the one or more storage unit IDs from the PSWM, the storage unit IDs including a first storage unit ID identifying a first storage unit and a second storage unit ID identifying a second storage unit, and to write the first prime segment in the first storage unit and the second storage unit based on the first storage unit ID and second storage unit ID respectively, without writing the first prime segment to a remainder of the storage units, wherein the first prime segment is one of a plurality of prime segments collectively representing a prime, the prime containing metadata representing a consistent point of data stored in the file system, including a reference to a root of the file system, and wherein the first prime segment is utilized to restore the file system back to a state represented by the consistent point.

18. The system of claim 17, wherein the PSWM is maintained within a segment storage engine of the storage system that is configured to segment a data file into a plurality of deduplicated segments before being stored in the storage system.

19. The system of claim 18, wherein each of the storage units is configured to store segments of data files, and wherein the segments are deduplicated within the respective storage unit.

20. The system of claim 17, wherein each of the storage units is configured to store data associated with one of a plurality of file system partitions.

21. The system of claim 20, wherein the file system partitions collectively represent a single logical file system of the storage system.

22. The system of claim 21, wherein each of the storage units maintains metadata having sufficient information to enable the corresponding file system partition to be self-identifiable and configurable.

23. The system of claim 22, wherein each file system partition can be individually started up and removed without having a major impact on operations of remaining file system partitions of the logical file system.

24. The system of claim 17, wherein the PSWM is stored in a persistent storage location of the storage system, and wherein in the event that the file system crashes, the PSWM is recovered from the last known installable prime during restart of the storage system.