Deterministic recovery of a file system built on a thinly provisioned logical volume having redundant metadata

• US 8,037,345 B1
• Filed: 03/31/2010
• Issued: 10/11/2011
• Est. Priority Date: 03/31/2010
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: Bhagwat, Rahul, Dharmadhikari, Amit, Potnis, Ajay, Patel, Dixit, DeForest, Miles A., Iyer, Sairam
• Primary Examiner(s): McCarthy; Christopher
• Application Number:
  US12/752,031
• Time in Patent Office: 559 Days
• Field of Search: None
• US Class Current: 714/6.12
• CPC Class Codes: G06F 11/1004 to protect a block of data words, e.g. CRC or checksum G06F11/1076 takes precedence; security arrangements for protecting computers or computer systems against unauthorized activity G06F21/00
• CPC Class Codes: G06F 11/1435 using file system or storage system metadata
• CPC Class Codes: G06F 11/186 Passive fault masking when reading multiple copies of the same data
• CPC Class Codes: G06F 11/2094 Redundant storage or storage space G06F11/2056 takes precedence
• CPC Class Codes: G06F 16/183 Provision of network file services by network file servers, e.g. by using NFS, CIFS network file access protocols H04L67/1097

Abstract

A file server architecture decouples logical storage from physical storage and provides proactive detection and containment of faults, errors, and corruptions in a file system, in order to enable in place (online) and non-intrusive recovery. The file system is built upon a thinly provisioned logical volume, and there are stored three copies of the metadata defining the logical volume in order to provide quick, deterministic, and reliable recovery from a faulted system. A first copy of the metadata is distributed among all of the slices of physical storage allocated to the logical volume. A second copy of the metadata is stored in a root slice of the logical volume. A third copy of the metadata is stored separate from the slices of physical storage allocated to the logical volume.

Claims

1. A file server comprising:

physical data storage; and

at least one data processor coupled to the physical data storage for accessing the physical data storage;

wherein said at least one data processor is programmed for maintaining a sparse metavolume of the physical data storage, the sparse metavolume includes slices of the physical data storage allocated to the sparse metavolume, the sparse metavolume provides logical data storage, some of the logical data storage is mapped to the slices of the physical data storage allocated to the sparse metavolume, some of the logical data storage does not have allocated physical storage and is not mapped to the slices of the physical data storage allocated to the sparse metavolume, and the sparse metavolume has slice metadata defining the allocation of the slices of the physical data storage to the sparse metavolume and the mapping of the logical data storage to the slices of the physical data storage that are allocated to the sparse metavolume, and

wherein the file server stores three copies of the slice metadata, and said at least one data processor is programmed with a recovery program executable by said at least one data processor to recover from a disruption by comparing the three copies of the slice metadata to detect and correct errors in the slice metadata.

2. The file server as claimed in claim 1, wherein one of the three copies of the slice metadata is stored in a kernel mode database.

3. The file server as claimed in claim 1, wherein a first one of the three copies of the slice metadata is distributed among the slices allocated to the sparse metavolume, a second one of the three copies of the slice metadata is stored in one of the slices allocated to the sparse metavolume, and a third one of the three copies of the slice metadata is stored separate from the slices allocated to the sparse metavolume.

4. The file server as claimed in claim 3, wherein the third one of the three copies of the slice metadata is stored in a kernel mode database.

5. The file server as claimed in claim 1, wherein each slice of physical data storage allocated to the sparse metavolume includes a slice mark containing a respective portion of the slice metadata defining allocation of said each slice to the sparse metavolume and mapping of the logical data storage of the sparse metavolume to said each slice, one of the three copies of the slice metadata is comprised of the respective portions of the slice metadata in slice marks of the slices of physical data storage allocated to the metavolume, the slice marks are chained together by links, and the recovery procedure is executable by said at least one data processor to recover from the disruption by following the links to collect the slice metadata contained in the slice marks.

6. The file server as claimed in claim 1, wherein each of the three copies of the slice metadata is organized as a series of entries, and each entry has a value and an indication of validity of the value, and the recovery program is executable by said at least one data processor to recover from a disruption by applying an iterative procedure to each entry to detect and correct any error in each entry in response to the values and indications of validity of said each entry in each of the three copies of the slice metadata.

7. The file server as claimed in claim 6, wherein the indications of validity are cyclic redundancy checks indicating whether the values have been corrupted.

8. The file server as claimed in claim 6, wherein the recovery procedure is executable by said at least one data processor to recover from a disruption by invoking a procedure to load the entries of each of the three copies of the slice metadata into a recovery table, and then to apply the iterative procedure to the entries of each of the three copies of the slice metadata in the recovery table.

9. The file server as claimed in claim 6, wherein the iterative procedure is executable by said at least one data processor to produce a log of errors detected and corrected in the three copies of the slice metadata.

10. A file server comprising:

physical data storage; and

at least one data processor coupled to the physical data storage for accessing the physical data storage;

wherein said at least one data processor is programmed for maintaining a sparse metavolume of the physical data storage, the sparse metavolume includes slices of the physical data storage allocated to the sparse metavolume, the sparse metavolume provides logical data storage, some of the logical data storage is mapped to the slices of the physical data storage allocated to the sparse metavolume, some of the logical data storage does not have allocated physical storage and is not mapped to the slices of the physical data storage allocated to the sparse metavolume, and the sparse metavolume has slice metadata defining the allocation of the slices of the physical data storage to the sparse metavolume and the mapping of the logical data storage to the slices of the physical data storage that are allocated to the sparse metavolume, and

wherein the file server stores three copies of the slice metadata, and said at least one data processor is programmed with a recovery procedure executable by the said at least one data processor to recover from a disruption by comparing the three copies of the slice metadata to detect and correct errors in the slice metadata,

wherein each slice of physical data storage allocated to the sparse metavolume includes a slice mark containing a respective portion of the slice metadata defining allocation of said each slice to the sparse metavolume and mapping of the logical data storage of the sparse metavolume to said each slice, a first one of the three copies of the slice metadata is comprised of the respective portions of the slice metadata in the slice marks of the slices of physical data storage allocated to the metavolume, the slice marks are chained together by links, and the recovery procedure is executable by said at least one data processor to recover from the disruption by following the links to collect the slice metadata contained in the slice marks,

wherein a second one of the three copies of the slice metadata is maintained in one of the slices allocated to the sparse metavolume, and

wherein a third one of the three copies of the slice metadata is stored in a kernel mode database separate from the slices of the physical data storage allocated to the sparse metavolume.

11. The file server as claimed in claim 10, wherein each of the three copies of the slice metadata is organized as a series of entries, and each entry has a value and a cyclic redundancy check indicating whether the value has been corrupted, and the recovery procedure is executable by said at least one data processor to recover from a disruption by applying an iterative procedure to each entry to detect and correct any error in each entry in response to the three respective values and the three respective cyclic redundancy checks in said each entry in the three copies of the slice metadata.

12. A computer-implemented method of operating a file server, the file server having physical data storage, and a sparse metavolume of the physical data storage, the sparse metavolume including slices of the physical data storage allocated to the sparse metavolume, the sparse metavolume providing logical data storage, some of the logical data storage being mapped to the slices of the physical data storage allocated to the sparse metavolume, and some of the logical data storage not having allocated physical storage and not being mapped to the slices of the physical data storage allocated to the sparse metavolume, and the sparse metavolume having slice metadata defining the allocation of the slices of the physical data storage to the sparse metavolume and the mapping of the logical data storage of the sparse metavolume to the slices of the physical data storage that are allocated to the sparse metavolume, said method comprising:

maintaining three copies of the slice metadata in the physical data storage, and

recovering from a disruption in operation of the file server by executing a recovery program in memory of the file server, the recovery program comparing the three copies of the slice metadata to detect and correct errors in the slice metadata.

13. The computer-implemented method as claimed in claim 12, which includes maintaining one of the three copies of the slice metadata as a kernel mode database in the physical data storage.

14. The computer-implemented method as claimed in claim 12, which includes maintaining a first one of the three copies of the slice metadata distributed among the slices allocated to the sparse metavolume, maintaining a second one of the three copies of the slice metadata in one of the slices allocated to the sparse metavolume, and maintaining and storing a third one of the three copies of the slice metadata separate from the slices allocated to the sparse metavolume.

15. The computer-implemented method as claimed in claim 14, which includes maintaining the third one of the three copies of the slice metadata in a kernel mode database in the physical data storage.

16. The computer-implemented method as claimed in claim 12, wherein each slice of physical data storage allocated to the sparse metavolume includes a slice mark containing a respective portion of the slice metadata defining allocation of said each slice to the sparse metavolume and mapping of the logical data storage of the sparse metavolume to said each slice, one of the three copies of the slice metadata is comprised of the respective portions of the slice metadata in the slice marks of the slices of physical data storage allocated to the metadata, and the slice marks are chained together by links, and the recovery procedure is executable by said at least one data processor to recover from the disruption by following the links to collect the slice metadata contained in the slice marks.

17. The computer-implemented method as claimed in claim 12, wherein each of the three copies of the slice metadata is organized as a series of entries, and each entry has a value and indication of validity of the value, and the method further includes applying an iterative procedure to each entry to detect and correct any error in each entry in response to the values and indications of validity of said each entry in each of the three copies of the slice metadata.

18. The computer-implemented method as claimed in claim 17, which includes computing cyclic redundancy checks to provide the indications of validity of the values.

19. The computer-implemented method as claimed in claim 17, which further includes invoking a procedure to load the entries of each of the three copies of the slice metadata into a recovery table, and then to apply the iterative procedure to the entries of each of the three copies of the slice metadata in the recovery table.

20. The computer-implemented method as claimed in claim 17, which further includes the iterative procedure producing a log of errors detected and corrected in the three copies of the slice metadata.