System and method for maintaining consistent points in file systems

• US 9,740,565 B1
• Filed: 07/10/2015
• Issued: 08/22/2017
• Est. Priority Date: 03/31/2011
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Inventors: Mitra, Soumyadeb, Hsu, Windsor W.
• Primary Examiner(s): Yi, David X
• Assistant Examiner(s): Goldschmidt, Craig
• Application Number:
  US14/796,817
• Time in Patent Office: 774 Days
• Field of Search: None
• CPC Class Codes: G06F 11/1435 using file system or storage system metadata
• CPC Class Codes: G06F 11/1446 Point-in-time backing up or restoration of persistent data
• CPC Class Codes: G06F 11/1448 Management of the data involved in backup or backup restore
• CPC Class Codes: G06F 16/11 File system administration, e.g. details of archiving or snapshots file system backup G06F11/14
• CPC Class Codes: G06F 16/113 Details of archiving lifecycle management in storage systems G06F3/0649; backup systems G06F11/1446
• CPC Class Codes: G06F 16/1748 De-duplication implemented within the file system, e.g. based on file segments de-duplication techniques in storage systems for the management of data blocks G06F3/0641
• CPC Class Codes: G06F 16/185 Hierarchical storage management [HSM] systems, e.g. file migration or policies thereof details of archiving G06F16/11
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity
• CPC Class Codes: G06F 2201/82 Solving problems relating to consistency
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data
• CPC Class Codes: G06F 3/0647 Migration mechanisms
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

A request is received to determine a consistent point of data stored in a file system of a storage system having storage units. In response to the request, a prime dependency list is retrieved from a first prime segment stored in a first storage unit, the prime dependency list including information identifying at least a second prime segment stored in a second storage unit. The first and second prime segments are identified by a first prime segment identifier (ID) and a second prime segment ID, respectively, which collectively identify a prime representing a first consistent view of the file system. The consistent point of data is determined based the prime segments listed in the prime dependency list, where the consistent point of data represents a file system state at a point in time for restoration of the file system back to a prior known state.

Claims

1. A computer-implemented method for maintaining a consistent point of a file system, the method comprising:

receiving a request to determine a consistent point of data stored in a file system of a storage system, the storage system having a plurality of storage units including a first storage unit and a second storage unit;

in response to the request, retrieving a prime dependency list from a first prime segment stored in the first storage unit, the prime dependency list including information identifying at least a second prime segment stored in the second storage unit, wherein the first and second prime segments are identified by a first prime segment identifier (ID) and a second prime segment ID, respectively, which collectively identify a prime representing a first consistent view of the file system; and

determining the consistent point of data based the prime segments listed in the prime dependency list, wherein the consistent point of data represents a file system state that provides a valid view of the data at a point in time of the file system for restoration of the file system back to a prior known state.

2. The method of claim 1, wherein the first and second prime segment IDs represent different versions of the prime.

3. The method of claim 2, wherein the prime dependency list comprises a first entry and a second entry, wherein the first entry includes the first prime segment ID associated with a first storage unit ID that identifies the first storage unit, and wherein the second entry includes the second prime segment ID associated with a second storage unit ID that identifies the second storage unit.

4. The method of claim 2, wherein the first and second prime segment IDs were determined based on a sequence order in which the first and second prime segments were written.

5. The method of claim 1, wherein determining the consistent point of data based the prime segments listed in the prime dependency list comprises:

verifying each of the prime segments listed in the prime dependency list in their respective storage units; and

identifying the prime represented by the prime segments listed in the prime dependency list as an installable prime if each of the prime segments has been verified.

6. The method of claim 5, wherein the installable prime can be used to construct the consistent point of data of the file system.

7. The method of claim 1, further comprising:

receiving a second request to write a second prime to the first and second storage units, the second prime being identified by a third prime segment ID;

in response to the second request, generating a second prime dependency list including the third prime segment ID associated with the first and second storage units; and

writing a third prime segment to at least the first storage unit, wherein the third prime segment includes the second prime dependency list.

8. The method of claim 7, wherein the second prime dependency list comprises a first entry and a second entry, wherein the first entry includes the third prime segment ID and a first storage unit ID that identifies the first storage unit, and wherein the second entry includes the third prime segment ID and a second storage unit ID that identifies the second storage unit.

9. A non-transitory machine-readable medium having instructions stored therein, which when executed by a processor, cause the processor to perform operations of maintaining a consistent point of a file system, the operations comprising:

receiving a request to determine a consistent point of data stored in a file system of a storage system, the storage system having a plurality of storage units including a first storage unit and a second storage unit;

in response to the request, retrieving a prime dependency list from a first prime segment stored in the first storage unit, the prime dependency list including information identifying at least a second prime segment stored in the second storage unit, wherein the first and second prime segments are identified by a first prime segment identifier (ID) and a second prime segment ID, respectively, which collectively identify a prime representing a first consistent view of the file system; and

determining the consistent point of data based the prime segments listed in the prime dependency list, wherein the consistent point of data represents a file system state that provides a valid view of the data at a point in time of the file system for restoration of the file system back to a prior known state.

10. The non-transitory machine-readable medium of claim 9, wherein the first and second prime segment IDs represent different versions of the prime.

11. The non-transitory machine-readable medium of claim 10, wherein the prime dependency list comprises a first entry and a second entry, wherein the first entry includes the first prime segment ID associated with a first storage unit ID that identifies the first storage unit, and wherein the second entry includes the second prime segment ID associated with a second storage unit ID that identifies the second storage unit.

12. The non-transitory machine-readable medium of claim 10, wherein the first and second prime segment IDs were determined based on a sequence order in which the first and second prime segments were written.

13. The non-transitory machine-readable medium of claim 9, wherein determining the consistent point of data based the prime segments listed in the prime dependency list comprises:

verifying each of the prime segments listed in the prime dependency list in their respective storage units; and

identifying the prime represented by the prime segments listed in the prime dependency list as an installable prime if each of the prime segments has been verified.

14. The non-transitory machine-readable medium of claim 13, wherein the installable prime can be used to construct the consistent point of data of the file system.

15. The non-transitory machine-readable medium of claim 9, wherein the operations further comprise:

receiving a second request to write a second prime to the first and second storage units, the second prime being identified by a third prime segment ID;

in response to the second request, generating a second prime dependency list including the third prime segment ID associated with the first and second storage units; and

writing a third prime segment to at least the first storage unit, wherein the third prime segment includes the second prime dependency list.

16. The non-transitory machine-readable medium of claim 15, wherein the second prime dependency list comprises a first entry and a second entry, wherein the first entry includes the third prime segment ID and a first storage unit ID that identifies the first storage unit, and wherein the second entry includes the third prime segment ID and a second storage unit ID that identifies the second storage unit.

17. A storage system, comprising:

a processor;

a memory;

a plurality of storage units coupled to the processor to store segments of data files of a file system, including a first storage unit and a second storage unit;

an archiver coupled to the processor and the memory, the archiver configured to

receive a request to determine a consistent point of data stored in the file system of the storage system,

in response to the request, retrieve a prime dependency list from a first prime segment stored in the first storage unit, the prime dependency list including information identifying at least a second prime segment stored in the second storage unit, wherein the first and second prime segments are identified by a first prime segment identifier (ID) and a second prime segment ID, respectively, which collectively identify a prime representing a first consistent view of the file system, and

determine the consistent point of data based the prime segments listed in the prime dependency list, wherein the consistent point of data represents a file system state that provides a valid view of the data at a point in time of the file system for restoration of the file system back to a prior known state.

18. The system of claim 17, wherein the first and second prime segment IDs represent different versions of the prime.

19. The system of claim 18, wherein the prime dependency list comprises a first entry and a second entry, wherein the first entry includes the first prime segment ID associated with a first storage unit ID that identifies the first storage unit, and wherein the second entry includes the second prime segment ID associated with a second storage unit ID that identifies the second storage unit.

20. The system of claim 18, wherein the first and second prime segment IDs were determined based on a sequence order in which the first and second prime segments were written.

21. The system of claim 17, wherein determining the consistent point of data based the prime segments listed in the prime dependency list comprises:

verifying each of the prime segments listed in the prime dependency list in their respective storage units; and

identifying the prime represented by the prime segments listed in the prime dependency list as an installable prime if each of the prime segments has been verified.

22. The system of claim 21, wherein the installable prime can be used to construct the consistent point of data of the file system.

23. The system of claim 17, wherein the archiver is further configured to

receive a second request to write a second prime to the first and second storage units, the second prime being identified by a third prime segment ID,

in response to the second request, generate a second prime dependency list including the third prime segment ID associated with the first and second storage units, and

write a third prime segment to at least the first storage unit, wherein the third prime segment includes the second prime dependency list.

24. The system of claim 23, wherein the second prime dependency list comprises a first entry and a second entry, wherein the first entry includes the third prime segment ID and a first storage unit ID that identifies the first storage unit, and wherein the second entry includes the third prime segment ID and a second storage unit ID that identifies the second storage unit.