System and method for maintaining consistent points in file systems using a prime dependency list

• US 9,996,540 B2
• Filed: 03/31/2011
• Issued: 06/12/2018
• Est. Priority Date: 03/31/2011
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Inventors: Mitra, Soumyadeb, Hsu, Windsor W.
• Primary Examiner(s): HARPER, ELIYAH STONE
• Application Number:
  US13/077,790
• Publication Number:
  US 20120254130A1
• Time in Patent Office: 2,630 Days
• Field of Search: 707999101, 707999102, 707650
• CPC Class Codes: G06F 16/128 Details of file system snapshots on the file-level, e.g. snapshot creation, administration, deletion use of snapshots for error detection or correction G06F11/14, G06F11/16
• CPC Class Codes: G06F 16/13 File access structures, e.g. distributed indices arrangements of input from, or output to, record carriers G06F3/06
• CPC Class Codes: G06F 16/1805 Append-only file systems, e.g. using logs or journals to store data
• CPC Class Codes: G06F 16/1873 Versioning file systems, temporal file systems, e.g. file system supporting different historic versions of files
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor

Abstract

According to one embodiment, a request is received for obtaining a consistent point of data stored in a file system of a storage system having a plurality of storage units. In response to the request, retrieving a prime dependency list from a first prime segment stored in a first of the storage units, where the prime dependency list includes information identifying at least a second prime segment stored in a second of the storage units. The first and second prime segments collectively form a prime segment representing a consistent view of the file system. Each of the prime segments listed in the prime dependency list is ascertained in an attempt to generate the consistent point of data.

Claims

1. A computer-implemented method for maintaining a consistent point of a file system, the method comprising:

receiving a request, by an archiver loaded in a memory and executed by a processor of a storage system having a plurality of storage units, to determine a consistent point of data stored in a file system of the storage system, wherein a first of the plurality of the storage units is an active storage unit that stores a first prime segment, and a second of the plurality of storage units is an archive storage unit that stores a second prime segment, wherein the first prime segment and the second prime segment collectively represent a first consistent view of the file system, wherein the first prime segment is a latest prime segment that was written to the active storage unit after the second prime segment was written to the archive storage unit;

in response to the request, retrieving by the processor, from the latest prime segment, a prime dependency list that includes information identifying all prime segments that collectively represent the first consistent view of the file system, the prime dependency list including information identifying the first prime segment stored in the active storage unit and information identifying the second prime segment stored in the archive storage unit, wherein the first and second prime segments are identified by a first prime segment identifier (ID) and a second prime segment ID, respectively, wherein the first and second prime segment IDs collectively identify a prime representing the first consistent view of the file system, wherein the active storage unit and the archive storage unit are separate physical storage units;

ascertaining, by the processor, each of the prime segments listed in the prime dependency list in an attempt to determine the consistent point of data, wherein the consistent point of data is a file system state which provides a valid view of the data at a point in time of the file system for restoration of the file system back to a prior known state, wherein the consistent point of data includes a first prime segment stored in the first storage unit and a second prime segment stored in the second storage unit, wherein the prime dependency list comprises a first entry and a second entry, wherein the first entry includes the first prime segment ID associated with a first storage unit ID that identifies the first storage unit, and wherein the second entry includes the second prime segment ID associated with a second storage unit ID that identifies the second storage unit, wherein the first storage unit stores a plurality of files that are backed up from a plurality of clients, wherein the second storage unit periodically archives at least a portion of the files by migrating the files from the first storage unit, while the first storage unit receives fresh backup data from the clients, and wherein the prime dependency list is used to identify the archived files stored in the second storage unit in order to identify all files corresponding to a particular consistent point of a file system stored both in the first storage unit and the second storage unit such that the first and the second portions of the consistent point of data do not need to reside on the same storage unit; and

restoring the file system back to a prior known state based on the identified first and second prime segments of the prime dependent list, including parsing the prime dependency list to determine storage units containing the first and second prime segments to retrieve the first and second prime segments.

2. The method of claim 1, wherein the first and second prime segment IDs represent different versions of the prime.

3. The method of claim 2, wherein the first and second prime segment IDs were determined based on a sequence order in which the first and second prime segments were written.

4. The method of claim 1, wherein ascertaining each of the prime segments comprises:

verifying each of the prime segments listed in the prime dependency list in their respective storage units; and

identifying the prime represented by the prime segments listed in the prime dependency list as an installable prime if each of the prime segments has been verified.

5. The method of claim 4, wherein the installable prime can be used to construct the consistent point of data of the file system.

6. The method of claim 1, further comprising:

receiving a second request to write a second prime to the first and second storage units, the second prime being identified by a third prime segment ID;

in response to the second request, generating a second prime dependency list including the third prime segment ID associated with the first and second storage units; and

writing a third prime segment to at least the first storage unit, wherein the third prime segment includes the second prime dependency list.

7. The method of claim 6, wherein the second prime dependency list comprises a first entry and a second entry, wherein the first entry includes the third prime segment ID and a first storage unit ID that identifies the first storage unit, and wherein the second entry includes the third prime segment ID and a second storage unit ID that identifies the second storage unit.

8. A non-transitory computer-readable storage medium having instructions stored therein, which when executed by a computer, cause the computer to perform operations comprising:

receiving a request, by an archiver loaded in a memory and executed by a processor of a storage system having a plurality of storage units, to determine a consistent point of data stored in a file system of the storage system, wherein a first of the plurality of the storage units is an active storage unit that stores a first prime segment, and a second of the plurality of storage units is an archive storage unit that stores a second prime segment, wherein the first prime segment and the second prime segment collectively represent a first consistent view of the file system, wherein the first prime segment is a latest prime segment that was written to the active storage unit after the second prime segment was written to the archive storage unit;

in response to the request, retrieving by the processor, from the latest prime segment, a prime dependency list that includes information identifying all prime segments that collectively represent the first consistent view of the file system, the prime dependency list including information identifying the first prime segment stored in the active storage unit and information identifying the second prime segment stored in the archive storage unit, wherein the first and second prime segments are identified by a first prime segment identifier (ID) and a second prime segment ID, respectively, wherein the first and second prime segment IDs collectively identify a prime representing the first consistent view of the file system, wherein the active storage unit and the archive storage unit are separate physical storage units;

ascertaining, by the processor, each of the prime segments listed in the prime dependency list in an attempt to determine the consistent point of data, wherein the consistent point of data is a file system state which provides a valid view of the data at a point in time of the file system for restoration of the file system back to a prior known state, wherein the consistent point of data includes a first prime segment stored in the first storage unit and a second prime segment stored in the second storage unit, wherein the prime dependency list comprises a first entry and a second entry, wherein the first entry includes the first prime segment ID associated with a first storage unit ID that identifies the first storage unit, and wherein the second entry includes the second prime segment ID associated with a second storage unit ID that identifies the second storage unit, wherein the first storage unit stores a plurality of files that are backed up from a plurality of clients, wherein the second storage unit periodically archives at least a portion of the files by migrating the files from the first storage unit, while the first storage unit receives fresh backup data from the clients, and wherein the prime dependency list is used to identify the archived files stored in the second storage unit in order to identify all files corresponding to a particular consistent point of a file system stored both in the first storage unit and the second storage unit such that the first and the second portions of the consistent point of data do not need to reside on the same storage unit; and

restoring the file system back to a prior known state based on the identified first and second prime segments of the prime dependent list, including parsing the prime dependency list to determine storage units containing the first and second prime segments to retrieve the first and second prime segments.

9. The computer-readable storage medium of claim 8, wherein the first and second prime segment IDs represent different versions of the prime.

10. The computer-readable storage medium of claim 9, wherein the first and second prime segment IDs were determined based on a sequence order in which the first and second prime segments were written.

11. The computer-readable storage medium of claim 8, wherein ascertaining each of the prime segments comprises:

verifying each of the prime segments listed in the prime dependency list in their respective storage units; and

identifying the prime represented by the prime segments listed in the prime dependency list as an installable prime if each of the prime segments has been verified.

12. The computer-readable storage medium of claim 11, wherein the installable prime can be used to construct the consistent point of data of the file system.

13. The computer-readable storage medium of claim 8, wherein the method further comprises:

receiving a second request to write a second prime to the first and second storage units, the second prime being identified by a third prime segment ID;

in response to the second request, generating a second prime dependency list including the third prime segment ID associated with the first and second storage units; and

writing a third prime segment to at least the first storage unit, wherein the third prime segment includes the second prime dependency list.

14. The computer-readable storage medium of claim 13, wherein the second prime dependency list comprises a first entry and a second entry, wherein the first entry includes the third prime segment ID and a first storage unit ID that identifies the first storage unit, and wherein the second entry includes the third prime segment ID and a second storage unit ID that identifies the second storage unit.

15. A storage system, comprising:

a processor;

a memory;

a plurality of storage units, wherein a first of the plurality of the storage units is an active storage unit that stores a first prime segment, and a second of the plurality of storage units is an archive storage unit that stores a second prime segment, wherein the first prime segment and the second prime segment collectively represent a first consistent view of the file system, wherein the first prime segment is a latest prime segment that was written to the active storage unit after the second prime segment was written to the archive storage unit; and

an archiver loaded in the memory and executed by the processor to

in response to a request for obtaining a consistent point of data stored in a file system, retrieve, from the latest prime segment, a prime dependency list that includes information identifying all prime segments that collectively represent the first consistent view of the file system, the prime dependency list including information identifying the first prime segment stored in the active storage unit and information identifying the second prime segment stored in the archive storage unit, wherein the first and second prime segments are identified by a first prime segment identifier (ID) and a second prime segment ID, respectively, wherein the first and second prime segment IDs collectively identify a prime representing the first consistent view of the file system, wherein the active storage unit and the archive storage unit are separate physical storage units,

ascertain each of the prime segments listed in the prime dependency list in an attempt to determine the consistent point of data, wherein the consistent point of data is a file system state which provides a valid view of the data at a point in time of the file system for restoration of the file system back to a prior known state, wherein the consistent point of data includes a first prime segment stored in the first storage unit and a second prime segment stored in the second storage unit, wherein the prime dependency list comprises a first entry and a second entry, wherein the first entry includes the first prime segment ID associated with a first storage unit ID that identifies the first storage unit, and wherein the second entry includes the second prime segment ID associated with a second storage unit ID that identifies the second storage unit, wherein the first storage unit stores a plurality of files that are backed up from a plurality of clients, wherein the second storage unit periodically archives at least a portion of the files by migrating the files from the first storage unit, while the first storage unit receives fresh backup data from the clients, and wherein the prime dependency list is used to identify the archived files stored in the second storage unit in order to identify all files corresponding to a particular consistent point of a file system stored both in the first storage unit and the second storage unit such that the first and the second portions of the consistent point of data do not need to reside on the same storage unit, and

restoring the file system back to a prior known state based on the identified first and second prime segments of the prime dependent list, including parsing the prime dependency list to determine storage units containing the first and second prime segments to retrieve the first and second prime segments.

16. The system of claim 15, wherein the first and second prime segment IDs represent different versions of the prime.

17. The system of claim 15, wherein during ascertaining each of the prime segments, the archiver is configured to

verify each of the prime segments listed in the prime dependency list in their respective storage units, and

identify the prime represented by the prime segments listed in the prime dependency list as an installable prime if each of the prime segments has been verified.

18. The system of claim 15, wherein the archiver is further configured to

receive a second request to write a second prime to the first and second storage units, the second prime being identified by a third prime segment ID,

in response to the second request, generate a second prime dependency list including the third prime segment ID associated with the first and second storage units, and

write a third prime segment to at least the first storage unit, wherein the third prime segment includes the second prime dependency list.