System and method for maintaining consistent points in file systems

• US 8,832,394 B2
• Filed: 03/31/2011
• Issued: 09/09/2014
• Est. Priority Date: 03/31/2011
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: Mitra, Soumyadeb, Hsu, Windsor W.
• Primary Examiner(s): Bradley, Matthew
• Assistant Examiner(s): GOLDSCHMIDT, CRAIG S
• Application Number:
  US13/077,795
• Publication Number:
  US 20120254565A1
• Time in Patent Office: 1,258 Days
• Field of Search: 711/161
• US Class Current: 711/161
• CPC Class Codes: G06F 11/1435 using file system or storage system metadata
• CPC Class Codes: G06F 11/1446 Point-in-time backing up or restoration of persistent data
• CPC Class Codes: G06F 11/1448 Management of the data involved in backup or backup restore
• CPC Class Codes: G06F 16/11 File system administration, e.g. details of archiving or snapshots file system backup G06F11/14
• CPC Class Codes: G06F 16/113 Details of archiving lifecycle management in storage systems G06F3/0649; backup systems G06F11/1446
• CPC Class Codes: G06F 16/1748 De-duplication implemented within the file system, e.g. based on file segments de-duplication techniques in storage systems for the management of data blocks G06F3/0641
• CPC Class Codes: G06F 16/185 Hierarchical storage management [HSM] systems, e.g. file migration or policies thereof details of archiving G06F16/11
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity
• CPC Class Codes: G06F 2201/82 Solving problems relating to consistency ensuring consistency in mirrored systems G06F11/2064
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data
• CPC Class Codes: G06F 3/0647 Migration mechanisms
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

According to one embodiment, in response to a request to write a prime segment of a file system of a storage system having a plurality of storage units, one or more of the storage units are identified based on a prime segment write-map (PSWM). The PSWM includes information indicating which of the storage units to which a next prime should be written. The prime segment is then written in the one or more storage units identified from the PSWM, without writing the prime segment to a remainder of the storage units. The prime segment represents at least a portion of a prime that contains metadata representing a consistent point of data stored in the file system.

Claims

1. A computer-implemented method for maintaining a consistent point of a file system, the method comprising:

in response to a request to write a first prime segment of a file system of a storage system having a plurality of storage units that archive data from one or more clients, identifying one or more of the storage units based on a prime segment write-map (PSWM), the PSWM having information indicating which of the storage units to which a next prime segment should be written;

writing the first prime segment in the one or more storage units identified from the PSWM, without writing the first prime segment to a remainder of the storage units, wherein the first prime segment represents at least a portion of a prime that contains metadata representing a consistent point of data stored in the file system, wherein the first prime segment includes information identifying files that constitute the consistent point of the file system, and wherein the first prime segment is utilized to restore the file system back to a state represented by the consistent point;

wherein in response to a request to migrate data from a first of the storage units to a second of the storage units, modifying the PSWM to include a first storage unit identifier (ID) and a second storage unit ID identifying the first and second storage units, respectively; and

performing the data migration from the first storage unit to the second storage unit; and

in response to fresh data received from a client to be archived, writing a second prime segment to the first and second storage units based on the first and second storage unit IDs retrieved from the PSWM, while the data migration is being performed from the first storage unit to the second storage unit.

2. The method of claim 1, further comprising maintaining the PSWM within a segment storage engine of the storage system that is configured to segment a data file into a plurality of deduplicated segments before being stored in the storage system.

3. The method of claim 1, wherein each of the storage units is configured to store segments of data files, and wherein the segments are deduplicated within the respective storage unit.

4. The method of claim 1, further comprising in response to a signal indicating an end of data migration, modifying the PSWM to remove the second storage unit ID.

5. The method of claim 4, wherein a subsequent new prime segment is to be written to the first storage unit based on the first storage unit ID of the PSWM, without writing the subsequent new prime segment to the second storage unit.

6. The method of claim 4, wherein the first storage unit is an active storage unit that receives fresh user data from the file system, and wherein the second storage unit is a target storage unit to archive data of the active storage unit.

7. The method of claim 1, wherein the PSWM is stored in a persistent storage location of the storage system, and wherein in the event that the file system crashes, the PSWM is recovered from the last known installable prime during restart of the storage system.

8. A non-transitory computer-readable storage medium having instructions stored therein, which when executed by a computer, cause the computer to perform operations for maintaining a consistent point of a file system, the operations comprising:

in response to a request to write a first prime segment of a file system of a storage system having a plurality of storage units that archive data from one or more clients, identifying one or more of the storage units based on a prime segment write-map (PSWM), the PSWM having information indicating which of the storage units to which a next prime segment should be written;

writing the first prime segment in the one or more storage units identified from the PSWM, without writing the first prime segment to a remainder of the storage units, wherein the first prime segment represents at least a prime that contains metadata representing a consistent point of data stored in the file system˜

wherein the first prime segment includes information identifying files that constitute the consistent point of the file system, and wherein the first prime segment is utilized to restore the file system back to a state represented by the consistent point;

in response to a request to migrate data from a first of the storage units to a second of the storage units, modifying the PSWM to include a first storage unit identifier (ID) and a second storage unit ID identifying the first and second storage units, respectively; and

performing the data migration from the first storage unit to the second storage unit; and

in response to fresh data received from a client to be archived, writing a second prime segment to the first and second storage units based on the first and second storage unit IDs retrieved from the PSWM, while the data migration is being performed from the first storage unit to the second storage unit.

9. The non-transitory computer-readable storage medium of claim 8, wherein the operations further comprise maintaining the PSWM within a segment storage engine of the storage system that is configured to segment a data file into a plurality of deduplicated segments before being stored in the storage system.

10. The non-transitory computer-readable storage medium of claim 8, wherein each of the storage units is configured to store segments of data files, and wherein the segments are deduplicated within the respective storage unit.

11. The non-transitory computer-readable storage medium of claim 8, wherein the operations further comprise in response to a signal indicating an end of data migration, modifying the PSWM to remove the second storage unit ID.

12. The non-transitory computer-readable storage medium of claim 11, wherein a subsequent new prime segment is to be written to the first storage unit based on the first storage unit ID of the PSWM, without writing the subsequent new prime segment to the second storage unit.

13. The non-transitory computer-readable storage medium of claim 11, wherein the first storage unit is an active storage unit that receives fresh user data from the file system, and wherein the second storage unit is a target storage unit to archive data of the active storage unit.

14. The non-transitory computer-readable storage medium of claim 8, wherein the PSWM is stored in a persistent storage location of the storage system, and wherein in the event that the file system crashes, the PSWM is recovered from the last known installable prime during restart of the storage system.

15. A storage system, comprising:

a processor;

a memory coupled to the processor;

a plurality of storage units to store segments of data files of a file system, wherein the storage units archive data from one or more clients;

a prime segment write-map (PSWM) maintained in the memory to store information indicating which of the storage units to which a next prime segment should be written; and

an archiver coupled to the PSWM and the storage units, executed by the processor, in response to a request to write a first prime segment of the file system, configured to identify one or more of the storage units based on the PSWM and to write the first prime in the one or more storage units identified from the PSWM, without writing the first prime segment to a remainder of the storage units, wherein the first prime segment represents at least a portion of a prime that contains metadata representing a consistent point of data stored in the file system, wherein the first prime segment includes information identifying files that constitute the consistent point of the file system, and wherein the first prime segment is utilized to restore the file system back to a state represented by the consistent point;

wherein in response to a request to migrate data from a first of the storage units to a second of the storage units, the archiver is configured to modify the PSWM to include a first storage unit identifier (ID) and a second storage unit ID identifying the first and second storage units, respectively;

wherein the archiver is configured to perform the data migration from the first storage unit to the second storage unit; and

wherein in response to fresh data received for archiving, the archiver is configured to write a second prime segment to the first and second storage units based on the first and second storage unit IDs retrieved from the PSWM, while the data migration is being performed from the first storage unit to the second storage unit.

16. The system of claim 15, wherein the PSWM is maintained within a segment storage engine of the storage system that is configured to segment a data file into a plurality of deduplicated segments before being stored in the storage system.

17. The system of claim 15, wherein the segments of data files are deduplicated within each of the storage units.

18. The system of claim 15, wherein the archiver, in response to a signal indicating an end of data migration, is configured to modify the PSWM to include only the first storage unit ID, such that a subsequent new prime segment is to be written to the first storage unit based on the first storage unit ID of the PSWM, without writing the subsequent new prime segment to the second storage unit.

19. The system of claim 18, wherein the first storage unit is an active storage unit that receives fresh user data from the file system, and wherein the second storage unit is a target storage unit to archive data of the active storage unit.