System and method for managing data versions in a file system

• US 7,707,165 B1
• Filed: 12/09/2004
• Issued: 04/27/2010
• Est. Priority Date: 12/09/2004
• Status: Active Grant

• Current Assignee: Network Appliance Incorporated (NetApp, Inc.)
• Original Assignee: NetApp, Inc.
• Inventors: Tummala, Narayana R., Jiang, Tianyu
• Primary Examiner(s): Alam; Hosain T
• Assistant Examiner(s): HARPER, ELIYAH STONE
• Application Number:
  US11/008,375
• Time in Patent Office: 1,965 Days
• Field of Search: 707/104.1, 707/102, 707/10, 707/203, 707/201, 707/100, 715/229
• US Class Current: 707/806
• CPC Class Codes: G06F 16/10 File systems; File servers
• CPC Class Codes: Y10S 707/99954 Version management

Abstract

A system and method manages data versions in a file system. A data version field associated with a direct block pointer in each level 1 indirect block stores a data version associated with the data contained in the direct block. When data is modified subsequent to the creation of a persistent consistency point image, the data version is incremented to a next data version value to signify that the data has been modified. Replication is aided by the ease of identifying modified data blocks by examining the data version field associated with each block.

Claims

1. A computer implemented method for managing changes of data in a storage system, comprising:

associating direct blocks and indirect blocks of the data in the storage system with a data version field;

generating a persistent consistency point image (PCPI) of the storage system;

storing the PCPI at a source device of the storage system;

marking the data version field of the direct and indirect blocks of the data which have not been modified since the PCPI was generated with a first value;

marking the data version field of the direct and indirect blocks of the data which have been modified since the PCPI was generated with a second value;

scanning one or more of the data version fields of the direct and indirect blocks to determine which direct and indirect blocks have been modified since the PCPI was generated without comparing two or more versions of the data;

transmitting to a destination device the direct and indirect blocks of the data having the data version field marked with the second value and not transmitting to the destination device the direct and indirect blocks of the data having the data version field marked with the first value; and

storing at the destination device the direct and indirect blocks of the data having the data version field marked with the second value.

2. The method of claim 1 further comprising:

associating the data version field with a direct block pointer.

3. The method of claim 1 further comprising:

associating an inode version field in a root inode of the data with the data version field.

4. The method of claim 3 further comprising indicating, by the inode version field, if any blocks within a data container defined by the root inode has been modified since the PCPI was generated.

5. The method of claim 1 wherein the first value indicates a current data version and the second value indicates a next data version.

6. The method of claim 5 further comprising:

incrementing the next data version when the PCPI is generated, whereby the current data version then equals the next data version.

7. A computer implemented system for managing changes of data versions in a storage system, comprising:

the storage system configured to store a persistent consistency point image (PCPI) at a source device of the storage system;

means for associating direct blocks and indirect blocks of the data in the storage system with a data version field;

means for generating a persistent consistency point image (PCPI) of the storage system;

means for storing the PCPI at the source device of the storage system;

means for marking the data version field of the direct and indirect blocks of the data which have not been modified since the PCPI was generated with a first value;

means for marking the data version field of the direct and indirect blocks of the data which have been modified since the PCPI was generated with a second value;

means for scanning one or more of the data version fields of the direct and indirect blocks to determine which direct and indirect blocks have been modified since the PCPI was generated without comparing two or more versions of the data;

means for transmitting to a destination device the direct and indirect blocks of the data having the data version field marked with the second value, and not transmitting to the destination device the direct and indirect blocks of the data having the data version field marked with the first value; and

means for storing at the destination device the direct and indirect blocks of the data having the data version field marked with the second value.

8. The computer implemented system of claim 7 further comprising:

means for associating the data version field with a direct block pointer.

9. The computer implemented system of claim 7 further comprising:

means for associating an inode version field in a root inode of the data with the data version field.

10. The computer implemented system of claim 9 further comprising means for indicating, by the inode version field, if any blocks within a data container defined by the root inode have been modified since the PCPI was generated.

11. The computer implemented system of claim 7 wherein the first value indicates a current data version and the second value indicates a next data version.

12. The computer implemented system of claim 11 further comprising:

means for incrementing the next data version when the PCPI is generated, whereby the current data version then equals the next data version.

13. A computer implemented system for managing changes of data versions in a computer storage system, comprising:

a module configured to mark data version fields associated with direct blocks and indirect blocks of data with a current data version and further configured to modify respective data version fields associated with the direct blocks and indirect blocks of the data to a next data version in response to an operation modifying the respective direct blocks and indirect blocks of data subsequent to generation of a persistent consistency point image (PCPI), the computer storage system configured to determine which of the respective direct and indirect blocks have been modified since the PCPI was generated without comparing two or more versions of the data;

the computer system further configured to transmit to a destination device the respective direct and indirect blocks of the data having the data version fields marked with the next data version and not transmit to the destination device the direct and indirect blocks of the data having the data version field marked with the current data version; and

the computer system further configured to store at the destination device the direct and indirect blocks of the data having the data version field marked with the next data version.

14. The computer implemented system of claim 13 wherein the module comprises a bmap module.

15. The computer implemented system of claim 13 further comprising an inode version field in a root inode configured to indicate if any blocks within a data container defined by the root inode have been modified since the PCPI was generated.

16. The computer implemented system of claim 13 wherein the computer storage system is further configured to scan one or more of the data version fields of the direct and indirect blocks to determine which of the respective direct and indirect blocks have been modified since the PCPI was generated.

17. The computer implemented system of claim 16 wherein the PCPI is configured to inherit the current data version and the next data version of an active file system of the computer storage system when the PCPI is generated.

18. The computer implemented system of claim 13 wherein the data version field is maintained in a direct block data version field of an inode.

19. The computer implemented system of claim 13 wherein when the PCPI is generated, the PCPI maintains the current data version.

20. The computer implemented system of claim 19 wherein the computer storage system is further configured to increment the next data version when the PCPI is generated, whereby the current data version then equals the next data version.

21. A physical computer readable medium containing program instructions executed by a processor, comprising:

program instructions that associate direct blocks and indirect blocks of data in a storage system with a data version field;

program instructions that generate a persistent consistency point image (PCPI) of the storage system;

program instructions that store the PCPI at a source device of the storage system;

program instructions that mark the data version field of the direct and indirect blocks of the data which have not been modified since the PCPI was generated with a first value;

program instructions that mark the data version field of the direct and indirect blocks of the data which have been modified since the PCPI was generated with a second value;

program instructions that scan one or more of the data version fields of the direct and indirect blocks to determine which direct and indirect blocks have been modified since the PCPI was generated without comparing two or more versions of the data;

program instructions that transmit to a destination device the direct and indirect blocks of the data having the data version field marked with the second value and not transmit to the destination device the direct and indirect blocks of the data having the data version field marked with the first; and

program instructions that store at the destination device the direct and indirect blocks of the data having the data version field marked with the second value.

22. A computer implemented method, comprising:

associating a data version field of direct and indirect blocks with data on a source device;

marking the data version field of the direct and indirect blocks as a current data version;

generating a persistent consistency point image (PCPI) of the data;

storing the PCPI at the source device;

modifying one or more direct and indirect blocks of the data subsequent to the PCPI, and in response to the modification occurring subsequent to the PCPI, marking the data version field of the modified direct and indirect blocks of the data as a next data version;

scanning one or more of the data version fields of the direct and indirect blocks to determine which direct and indirect blocks have been modified since the PCPI was generated;

transmitting to a destination device the direct and indirect blocks of the data marked as the next data version, wherein that transmission does not include the direct and indirect blocks of the data marked with the current data version; and

storing at the destination device the direct and indirect blocks of the data marked as the next data version.

23. The computer implemented method of claim 22 further comprising:

associating the data version field with a direct block pointer.

24. The computer implemented method of claim 22 further comprising:

associating an inode version field in a root inode of the data with the data version field.

25. The computer implemented method of claim 24 further comprising indicating, by the inode version field, if any blocks within a data container defined by the root inode have been modified since the PCPI was generated.

26. The computer implemented method of claim 22 wherein the scanning is completed without comparing two or more versions of the data.

27. The computer implemented method of claim 26 further comprising:

incrementing the next data version when the PCPI is generated, whereby the current data version then equals the next data version.

28. A computer implemented method, comprising:

generating a persistent consistency point image (PCPI) of a source device;

storing the PCPI at the source device;

marking a data version field of each individually modified direct and indirect block of data at the source device, the marking associated with a new data version value only for the direct and indirect blocks of the data having been modified subsequent to the generation of the PCPI;

maintaining a data version field of each direct and indirect block of data at the source device that has not been modified subsequent to the generation of the PCPI, the maintained data version field associated with a current data version value only for the direct and indirect blocks of the data not having been modified subsequent to the generation of the PCPI;

scanning one or more of the data version fields of the direct and indirect blocks to determine which direct and indirect blocks have been modified subsequent to the generation of the PCPI;

replicating the PCPI of the source device on a destination device by transmitting to the destination device the modified direct and indirect blocks of the data having the direct and indirect blocks of the data version field marked with the new data version value and not transmitting to the destination device the unmodified direct and indirect blocks of the data having the direct and indirect blocks of the data version field maintained with the current data version value; and

storing at the destination device the modified direct and indirect blocks of the data having the direct and indirect blocks of the data version field marked with the new data version value.

29. The computer implemented method of claim 28 further comprising:

associating the data version field with a direct block pointer.

30. The computer implemented method of claim 28 further comprising:

associating an inode version field in a root inode of the data with the data version field.

31. The computer implemented method of claim 30 further comprising indicating, by the inode version field, if any blocks within a data container defined by the root inode have been modified since the PCPI was generated.

32. The computer implemented method of claim 28 further comprising:

maintaining the current data version value by the PCPI when the PCPI is generated.

35. The computer implemented method of claim 32 further comprising:

associating an inode version field in a root inode of the data with the data version field.

36. The computer implemented method of claim 35 further comprising indicating, by the inode version field, if any blocks within a data container defined by the root inode have been modified since the first PCPI was generated.

33. A computer implemented method, comprising:

generating a first persistent consistency point image (PCPI) of data on a source device;

modifying at least some direct and indirect blocks of the data subsequent to the generation of the first PCPI;

marking a data version field associated with the direct and indirect blocks of modified data with a next data version value;

generating a second PCPI of the data on the source device, the second PCPI comprising the direct and indirect blocks of data unmodified since the first PCPI was generated and the direct and indirect blocks of the data modified since the first PCPI was generated, the direct and indirect blocks of the modified data marked with the next data version value;

scanning one or more of the data version fields of the direct and indirect blocks of the modified and unmodified data to determine which direct and indirect blocks have been modified since the first PCPI was generated;

replicating the second PCPI on a destination device by transmitting to the destination device only the data marked with the next data version value on the second PCPI; and

storing at the destination device only the data marked with the next data version value on the second PCPI.

34. The computer implemented method of claim 33 further comprising:

associating the data version field with a direct block pointer.

37. A computer implemented method for operating a data storage system, comprising:

indicating in a source computer system comprising direct and indirect blocks of data, whether the direct and indirect blocks of the data have been changed since a persistent consistency point image (PCPI) was generated, the direct and indirect blocks of the data comprising a field to indicate with a first value that a first group of direct and indirect blocks of the data has not been changed since the PCPI was generated, and having a second value indicating that a second group of direct and indirect blocks of the data has been changed since the PCPI was generated;

scanning one or more of the data version fields of the first group and second group of direct and indirect blocks to determine which direct and indirect blocks have been changed since the PCPI was generated without comparing two or more versions of the data;

transmitting to a destination computer system, in response to the field having the second value, one or more of the second group of direct and indirect blocks of the data changed since the PCPI was generated and not transmitting to the destination computer system, in response to the field having the first value, one or more of the first group of direct and indirect data blocks which have not been changed since the PCPI was generated; and

storing at the destination device the one or more of the second group of direct and indirect blocks of the data changed since the PCPI was generated.

38. The computer implemented method of claim 37, further comprising:

associating an inode version field with a root inode of the data, wherein the inode version field indicates if any blocks within a data container defined by the root inode have been changed since the PCPI was generated.

39. A computer data storage system, comprising:

a source computer system comprising at least one indirect block of data, the indirect block of data configured to have a field to indicate with a first value that the indirect block of data has not been changed since a particular persistent consistency point image (PCPI) was generated, and a field configured to indicate with a second value that the indirect block of data has been changed since the particular PCPI was generated;

an operating system of the source computer system configured to scan at least one field of the indirect block to determine if the indirect block has been changed since the particular PCPI was generated, the operating system further configured to transmit one or more data blocks having the field of the indirect block of data with the second value to a destination computer system, wherein the operating system is further configured not to transmit one or more data blocks having the field of the indirect block of data with the first value to the destination computer system; and

the destination computer system configured to store the one or more data blocks having the field of the indirect block of data with the second value.

40. The computer data storage system as in claim 39, further comprising:

associating an inode version field with a root inode of the data, wherein the inode version field indicates if any blocks within a data container defined by the root inode have been changed since the particular PCPI was generated.

41. A computer implemented method for managing changes of data in a storage system, comprising:

associating a plurality of indirect blocks of data in the storage system with a data version field;

generating a persistent consistency point image (PCPI) of the storage system;

associating the plurality of indirect blocks of the data with a current version value;

modifying a first portion of indirect blocks of the plurality of indirect blocks after generating the PCPI, and not modifying a second portion of indirect blocks of the plurality of indirect blocks after generating the PCPI;

changing, in response to modifying the first portion of indirect blocks after the PCPI was generated, the association of the data version field of the first portion of indirect blocks from the current version value to a next version value;

maintaining, in response to the second portion of indirect blocks not being modified after the PCPI was generated, the association of the data version field of the second portion of indirect blocks with the current version value;

scanning the data version field of the plurality of indirect blocks to determine which of the indirect blocks have been modified after the PCPI was generated; and

transmitting to a destination device, in response to the scan determining that the first portion of indirect blocks have been modified after the PCPI was generated, the first portion of indirect blocks which had the data version field association changed to the next version value and not transmitting to the destination device the second portion of indirect blocks having maintained the data version field association with the current version value; and

storing at the destination device the first portion of indirect blocks which had the data version field association changed to the next version value.

42. A computer implemented method for managing changes of data in a storage system, comprising:

associating direct blocks and indirect blocks of data in the storage system with a data version field;

generating a persistent consistency point image (PCPI) of the storage system;

marking the data version field of the direct and indirect blocks which have not been modified since the PCPI was generated with a current version value;

marking the data version field of the direct and indirect blocks which have been modified since the PCPI was generated with a next version value;

scanning the data version field of the direct and indirect blocks to determine the direct and indirect blocks which have been modified since the PCPI was generated without comparing two or more versions of the data;

transmitting to a destination device the direct and indirect blocks having the data version field marked with the next version value, the direct and indirect blocks having the data version field marked with the current version value not being transmitted to the destination device; and

storing at the destination device the direct and indirect blocks having the data version field marked with the next version value.

43. A computer implemented method for managing changes of data in a storage system, comprising:

associating an inode version field with data blocks and indirect blocks within an inode, the indirect blocks having a data version field for at least one data block pointer contained in the indirect blocks;

associating a direct version field with the data blocks;

generating a persistent consistency point image (PCPI) of the storage system;

storing the PCPI at a source device of the storage system;

associating an inode version field value of the inode version field with a current data version value if no blocks within the inode have been modified since the PCPI was generated, and associating the inode version field value with a next data version value if any blocks within the inode have been modified since the PCPI was generated;

associating a data version field value of the data version field of the indirect blocks which have not been modified since the PCPI was generated with the current data version value and associating the data version field value of the indirect blocks which have been modified since the PCPI was generated with the next data version value;

marking a data version field value of the direct version field with the current data version value for each data block which has not been modified since the PCPI was generated and associating the direct version field value with the next data version value for each data block which has been modified since the PCPI was generated;

scanning the inode version field to determine if any blocks within the inode have been modified since the PCPI was generated, and scanning, in response to the inode version field being associated with the next data version value, the data version field and the direct version field to determine which indirect blocks and data blocks have been

modified since the PCPI was generated without comparing two or more versions of the data;

transmitting to a destination device the indirect blocks and data blocks having the data version field associated with the next data version value, the indirect blocks and data blocks having the data version field associated with the current data version not being transmitted to the destination device; and

storing at the destination device the indirect blocks and data blocks having the data version field associated with the next data version value.