System and method for managing redundant storage consistency at a file system level

• US 7,617,259 B1
• Filed: 12/31/2004
• Issued: 11/10/2009
• Est. Priority Date: 12/31/2004
• Status: Active Grant

• Current Assignee: Veritas Technologies, LLC (Whitehouse Group Ltd.)
• Original Assignee: Symantec Operating Corporation (NortonLifeLock Inc.)
• Inventors: Muth, John A., Kiselev, Oleg, Harmer, Craig K., Borate, Milind, Bezbaruah, Angshuman
• Primary Examiner(s): Breene; John E
• Assistant Examiner(s): Bullock; Joshua
• Application Number:
  US11/027,624
• Time in Patent Office: 1,775 Days
• Field of Search: 707/200, 707/206, 707/204, 711/162, 711/114, 714/6
• US Class Current: 1/1
• CPC Class Codes: G06F 11/1471 involving logging of persistent data for recovery
• CPC Class Codes: G06F 11/2058 using more than 2 mirrored copies
• CPC Class Codes: G06F 11/2064 while ensuring consistency
• CPC Class Codes: G06F 11/2066 Optimisation of the communication load
• CPC Class Codes: G06F 11/2071 using a plurality of controllers
• CPC Class Codes: G06F 11/2082 Data synchronisation

Abstract

A file system may save dirty file information indicating portions of files, residing on a storage system, to be modified corresponding data modification requests received by the file system. The file system may update the saved dirty file information to indicate successful modification of the files across redundant storage of the storage system and may track redundancy consistency recovery information on a file-by-file basis or on a file portion basis. A file system may also retrieve saved dirty file information and recover redundancy consistency for portions of files determined, based on the saved dirty file information, not to have redundancy consistency on the storage system. Recovering redundancy consistency may include copying, either directly to via a storage system mechanism, portions of files between redundant storage of the storage system. The redundancy consistency for the saved dirty file information may be recovered prior to recovering redundancy consistency for individual files.

Claims

1. A method for a file system to perform a redundancy consistency recovery operation for a file on a storage system without using any redundancy consistency mechanism of the storage system, comprising:

a file system receiving data modification requests each indicating data modifications to a file on a storage system, wherein the storage system comprises a plurality of storage devices configured to store data and redundancy data for the file;

for each data modification request;

the file system saving dirty file information at a file system level, wherein the dirty file information is separate from the data and redundancy data for the file, wherein the dirty file information indicates one or more portions of the file to be modified, wherein the one or more portions correspond to the indicated data modifications; and

the file system performing the indicated data modifications on the storage system, wherein said performing comprises storing redundancy data for the indicated data modifications, wherein the data modifications and redundancy data are stored on the storage system and are stored separately from the dirty file information; and

performing a redundancy consistency recovery operation for the file on the storage system without using any redundancy consistency mechanism of the storage system, wherein the redundancy consistency recovery operation uses the dirty file information.

2. The method of claim 1, wherein said saving dirty file information comprises saving the dirty file information in a file system log.

3. The method of claim 1, further comprising:

determining that the modifications to the one or more portions of a file to be modified have been completed on the storage system; and

updating the saved dirty file information to indicate that the modifications to the one or more portions of the file to be modified have been completed on the storage system.

4. The method of claim 3, wherein said updating is performed as part of saving, at the file system level, metadata modifications for each file.

5. The method of claim 1, wherein said performing a redundancy consistency recovery operation for the file on the storage system without using any redundancy consistency mechanism of the storage system further comprises:

retrieving the saved dirty file information from the file system; and

recovering redundancy consistency for each file indicated by the dirty file information, wherein said recovering redundancy consistency comprises updating redundancy data for each indicated file.

6. The method of claim 5, wherein the redundancy consistency recovery operation further comprises recovering redundancy consistency for the file system dirty file information on the storage system prior to said retrieving the saved dirty file information.

7. The method of claim 5, wherein said retrieving the saved dirty file information from the file system comprises reading the saved dirty file information from a file system log.

8. The method of claim 7, further comprising:

performing a redundancy consistency recovery operation on the storage system, wherein the redundancy consistency recovery operation comprises;

retrieving information from the file system indicating metadata modifications for one or more files; and

modifying metadata for one or more files on the storage system as indicated by the information retrieved from the file system, wherein said modifying comprises modifying the metadata for each of the one or more files on the storage system.

9. The method of claim 7, further comprising:

recovering redundancy consistency for the file system log on the storage system without using any redundancy consistency mechanism of the storage system, wherein said recovering redundancy consistency for the file system log comprises;

identifying one or more log entries of the file system log that are known to have redundancy consistency; and

recovering redundancy consistency for one or more remaining log entries not known to have redundancy consistency, wherein said recovering redundancy consistency comprises updating redundancy data for the one or more remaining log entries.

10. The method of claim 9, wherein the storage system comprises a plurality of mirrors;

and wherein said updating redundancy data for the one or more remaining log entries comprises copying the one or more remaining log entries from one of the mirrors to each other mirror.

11. The method of claim 9, wherein the storage system comprises a RAID configuration;

and wherein said updating redundancy data for the one or more remaining log entries comprises updating redundancy data for one or more RAID stripes corresponding to the remaining log entries.

12. A device, comprising:

a processor; and

a memory coupled to the processor, wherein the memory comprises program instructions executable by the processor to implement a file system configured to;

receive data modification requests each indicating data modifications to a file on a storage system, wherein the storage system comprises a plurality of storage devices configured to store data and redundancy data for the file;

for each data modification request;

save dirty file information and metadata information for the data modification request to a log or journal of the file system, wherein the dirty file information and metadata information are separate from the data and redundancy data for the file, wherein the dirty file information indicates one or more portions of the file to be modified, wherein the one or more portions correspond to the indicated data modifications; and

wherein the metadata information is saved to the log or journal of the file system using an I/O operation and the dirty file information is saved to the log or journal of the file system without incurring an additional I/O operation;

perform the indicated data modifications on the storage system; and

perform a redundancy consistency recovery operation for the file on the storage system without using any redundancy consistency mechanism of the storage system, wherein the redundancy consistency recovery operation uses the dirty file information.

13. The device of claim 12, wherein the program instructions are further configured to:

determine that the modifications to the one or more portions of a file to be modified have been completed on the storage system; and

update the saved dirty file information to indicate that the modifications to the one or more portions of the file to be modified have been completed on the storage system.

14. The device of claim 12, wherein the program instructions are further configured to:

perform the redundancy consistency recovery operation on the storage system, wherein the redundancy consistency recovery operation comprises;

retrieving the saved dirty file information from the file system; and

recover redundancy consistency for each file indicated by the dirty file information, wherein said recovering redundancy consistency comprises updating redundancy data for each indicated file.

15. A method for recovering redundancy consistency for a storage system comprising a plurality of storage devices configured to store data and redundancy data, comprising:

recovering redundancy consistency for saved dirty file information on a storage system, wherein the dirty file information indicates one or more portions of one or more files that are known to have redundancy consistency on the storage system, wherein said recovering redundancy consistency comprises verifying redundancy data for the dirty file information to ensure a consistent state of the saved dirty file information;

retrieving said saved dirty file information from a file system; and

subsequent to said recovering redundancy consistency for saved dirty file information and subsequent to said retrieving said saved dirty file information, recovering redundancy consistency for one or more remaining portions of one or more files, wherein said recovering redundancy consistency comprises updating redundancy data for the one or more remaining portions of each file as indicated by the dirty file information.

16. The method of claim 15, wherein said retrieving saved dirty file information from a file system comprises reading the saved dirty file information from a file system log.

17. The method of claim 15, wherein the storage system comprises a plurality of mirrors;

and wherein said updating redundancy data comprises copying the one or more remaining portions of each file, as indicated by the dirty file information, from one mirror to each other mirror.

18. The method of claim 17, wherein said copying comprises saving information to enable a redundancy consistency mechanism of the storage system to copy the one or more remaining portions from one of the mirrors to each of the other mirrors.

19. The method of claim 15, wherein the storage system comprises a RAID configuration;

and wherein said updating redundancy data comprises updating redundancy data for one or more RAID stripes corresponding to the one or more remaining portions of the one or more files.