System and method for redundant storage consistency recovery

• US 7,415,488 B1
• Filed: 12/31/2004
• Issued: 08/19/2008
• Est. Priority Date: 12/31/2004
• Status: Active Grant

• Current Assignee: Veritas Technologies, LLC (Whitehouse Group Ltd.)
• Original Assignee: Symantec Operating Corporation (NortonLifeLock Inc.)
• Inventors: Kiselev, Oleg, Harmer, Craig K., Muth, John A.
• Primary Examiner(s): Breene; John
• Assistant Examiner(s): Bullock; Joshua
• Application Number:
  US11/027,650
• Time in Patent Office: 1,327 Days
• Field of Search: 707/200, 707/202, 711/162, 714/6
• US Class Current: 707/648
• CPC Class Codes: G06F 11/2082 Data synchronisation

Abstract

A file system may selectively perform redundancy consistency recovery and synchronization of mirrors and may store dirty file information in a log or journal that indicates the modification of a file. A file system may update saved dirty file information to indicate successful modification of data across mirrors and may keep track of redundancy consistency recovery information on a file-by-file basis or may track the redundancy consistency of portions of files. A file system may also instruct an underlying storage system not to track data modifications, possibly selectively for each I/O request. During synchronization the file system may synchronize its log and use saved dirty file information to determine files to be synchronized. A file system may selectively use file system logging for individual data modification requests and may expose an API allowing applications to specify a redundancy consistency mechanism.

Claims

1. A method, comprising:

receiving data modification requests each indicating one or more data or metadata modifications for a file on a storage system, wherein the storage system comprises a plurality of storage devices configured to store data and redundancy data for the file;

for each of a plurality of the data modification requests;

determining whether or not to save dirty file information at a file system level to track redundancy consistency information for the data modification request;

in response to determining that dirty file information should be saved at the file system level for at least one of the plurality of data modification requests;

saving dirty file information at the file system level, wherein the dirty file information indicates one or more portions of the file to be modified, wherein the one or more portions correspond to the indicated one or more data or metadata modifications;

instructing the storage system not to apply a redundancy consistency mechanism to the one or more data or metadata modifications indicated in the at least one data modification request; and

performing the indicated one or more data or metadata modifications on the storage system.

2. The method of claim 1, wherein said determining whether or not to save dirty file information comprises determining whether or not data modifications indicated by the data modification requests may cause the file to enter a potentially inconsistent state.

3. The method of claim 1, further comprising performing a redundancy consistency recovery operation on the storage system, wherein the redundancy consistency recovery operation comprises:

retrieving the saved dirty file information from the file system; and

recovering redundancy consistency for each file indicated by the dirty file information, wherein said recovering redundancy consistency comprises updating redundancy data for each indicated file.

4. The method of claim 3, further comprising not recovering redundancy consistency for one or more portions of one or more files, wherein the dirty file information indicates that the modifications to the one or more portions have been completed on the storage system.

5. The method of claim 3, wherein the storage system comprises a plurality of mirrors;

and wherein said updating redundancy data comprises copying the one or more portions of each file from one mirror to each other mirror.

6. The method of claim 5, wherein said copying comprises:

configuring the storage system to operate in a READ-WRITEBACK mode of I/O, wherein when operating in a READ-WRITEBACK mode of I/O, the storage system is configured to write any data read from any one of the mirrors to each of the other mirrors; and

reading the one or more portions of each file from one of the mirrors.

7. The method of claim 5, wherein said copying comprises saving information to enable a redundancy consistency mechanism of the storage system to copy the one or more portions from one of the mirrors to each of the other mirrors.

8. The method of claim 7, wherein the storage system comprises a volume manager, wherein said saving information to enable a redundancy consistency mechanism of the storage system to copy the one or more portions comprises marking one or more regions as dirty in a dirty region map of the storage system, and wherein the volume manager performs said copying according to the dirty region map.

9. The method of claim 8,

wherein the storage system comprises a volume manager;

wherein said saving information to enable a redundancy consistency mechanism of the storage system to copy the one or more portions comprises storing in a log on the storage system information about regions of the volume which are being modified; and

wherein the volume manager performs said copying according to the information stored in the log.

10. The method of claim 3, wherein the storage system comprises a RAID configuration;

and wherein said updating redundancy data comprises updating redundancy data for one or more RAID stripes corresponding to the files indicated by the dirty file information.

11. The method of claim 1, wherein said determining is based on information received from an application associated with each data modification request specifying whether the respective data modifications should be tracked by a redundancy consistency mechanism of the storage system.

12. The method of claim 1, further comprising:

wherein, for one or more of the plurality of the data modification requests, said determining is based on information received from an application associated with those one or more data modification requests;

wherein the received information indicates that the application will maintain recovery information for respective data modifications; and

in response to determining that the application will maintain recovery information;

not performing said saving for those data modification requests that the application will maintain recovery information; and

instructing the storage system not to apply the redundancy consistency mechanism to the respective data modifications.

13. The method of claim 1, further comprising:

determining for each of one or more of the data modification requests not to save redundancy consistency recovery information for the respective indicated data modifications, wherein said determining is based respectively on each of the one or more data modification requests; and

instructing the storage system not to apply a redundancy consistency mechanism to the respective data modifications indicated in the corresponding data modification request.

14. The method of claim 1, wherein said receiving, said determining, said saving, said instructing and said performing are performed by a file system on one of a host device, a storage device, or an object storage device.

15. A system, comprising:

a processor; and

a memory coupled to the processor, wherein the memory comprises program instructions configured to;

receive data modification requests each indicating one or more data modifications to a file on a storage system, wherein the storage system comprises a plurality of storage devices configured to store data and redundancy data for the file; and

for each of a plurality of the data modification requests;

determine whether or not to save dirty file information at a file system level to track redundancy consistency information for the data modification request;

in response to determining that dirty file information should be saved for at least one of the plurality of data modification requests;

save dirty file information at the file system level, wherein the dirty file information indicates one or more portions of the file to be modified, wherein the one or more portions correspond to the one or more data modifications indicated in the at least one data modification request; and

instruct the storage system not to apply a redundancy consistency mechanism for the indicated one or more data modifications; and

perform the indicated one or more data modifications on the storage system.

16. The system of claim 15, wherein the program instructions are further configured to perform a redundancy consistency recovery operation on the storage system, wherein as part of the redundancy consistency recovery operation the program instructions are further configured to:

retrieve the saved dirty file information from the file system; and

recover redundancy consistency for each file indicated by the dirty file information, wherein said recovering redundancy consistency comprises updating redundancy data for each indicated file.

17. The system of claim 16, wherein the storage system comprises a plurality of mirrors;

and wherein as part of said updating redundancy data the program instructions are further configured to copy the one or more portions of each file from one mirror to each other mirror.

18. The system of claim 15, wherein the program instructions are further configured to:

determine that the one or more modifications to the one or more portions of a file to be modified have been completed on the storage system; and

update the saved dirty file information to indicate that the one or more modifications to the one or more portions of the file to be modified have been completed on the storage system.

19. The system of claim 18, wherein the program instructions are further configured to perform a redundancy consistency recovery operation on the storage system, wherein as part of the redundancy consistency recovery operation the program instructions are further configured to:

read information from the file system log indicating metadata modifications for one or more files; and

modify the metadata for one or more files on the storage system as indicated by the information read from the file system log, wherein said modifying comprises modifying the metadata for each of the one or more files on the storage system.

20. A computer accessible storage medium, comprising program instructions configured to implement:

receiving data modification requests each indicating one or more data modifications to a file on a storage system, wherein the storage system comprises a plurality of storage devices configured to store data and redundancy data for the file; and

for each of a plurality of the data modification requests;

determining whether to save dirty file information at a file system level to track redundancy consistency information for the data modification request;

in response to determining that dirty file information should be saved for at least one of the plurality of data modification requests;

saving dirty file information at the file system level, wherein the dirty file information indicates one or more portions of the file to be modified, wherein the one or more portions correspond to the one or more data modifications indicated in the at least one data modification request; and

instructing the storage system not to apply a redundancy consistency mechanism to the one or more data modifications indicated in the data modification request; and

performing the indicated one or more data modifications on the storage system.