EFFICIENT HIERARCHICAL STORAGE MANAGEMENT OF A FILE SYSTEM WITH SNAPSHOTS

• US 20090043978A1
• Filed: 08/06/2007
• Published: 02/12/2009
• Est. Priority Date: 08/06/2007
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Sawdon, Wayne A., Schmuck, Frank B.
• Granted Patent: US 8,055,864 B2
• Time in Patent Office: Days
• US Class Current: 711/162
• CPC Class Codes: G06F 16/174 Redundancy elimination performed by the file system management of the data involved in backup or backup restore using de-duplication of the data G06F11/14
• CPC Class Codes: G06F 16/185 Hierarchical storage management [HSM] systems, e.g. file migration or policies thereof details of archiving G06F16/11
• CPC Class Codes: G06F 3/0607 by facilitating the process of upgrading existing storage systems, e.g. for improving compatibility between host and storage device
• CPC Class Codes: G06F 3/0647 Migration mechanisms
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

A method is provided for managing the storage of a file that has been selected for migration from a first storage level within a file system to a second storage level. The method comprises copying each of one or more data blocks of the selected file that are stored in the first storage level at a physical disk address maintained in a corresponding reference of an inode for the selected file from the first storage level to the second storage level; maintaining a logical ditto address in each reference of an inode for each snapshot file in the file system that refers to one of the one or more data blocks of the selected file that were copied to the second storage level; and updating the file system to include a managed region for controlling access to the one or more data blocks through the inode for the selected file.

Claims

1. A method for managing the storage of a file that has been selected for migration from a first storage level within a file system to a second storage level, the selected file being selected from a plurality of source files each organized within the file system into one or more data blocks, the file system having an inode for each source file of the plurality of source files, each inode having a reference to each of the one or more data blocks of the corresponding source file, the file system being configured to regularly generate snapshot files of the source files of the plurality of source files, the file system having an inode for each snapshot file, the inode for each snapshot file having a reference to each of the one or more data blocks of the corresponding source file at the time the snapshot file was generated, the method comprising:

copying each of the one or more data blocks of the selected file that are stored in the first storage level at a physical disk address maintained in the corresponding reference of the inode for the selected file from the first storage level to the second storage level;

maintaining a logical ditto address in each reference of the inode for each snapshot file in the file system that refers to one of the one or more data blocks of the selected file that were copied to the second storage level, each logical ditto address indicating that additional processing is required to obtain the physical disk address of the data block referred to by the reference; and

updating the file system to include a managed region for controlling access to the one or more data blocks through the inode for the selected file.

2. The method of claim 1, further comprising:

deallocating the one or more data blocks of the selected file that were copied to the second storage level in the first storage level.

3. The method of claim 1, further comprising:

deallocating each of the one or more data blocks of the selected file that were copied to the second storage level in the first storage level after each snapshot file having a reference to that data block has also been selected for migration.

4. The method of claim 2, wherein the file system is configured to assign a logical ditto address in each reference of the inode for each snapshot file that refers to one of the one or more data blocks of the corresponding source file when the file system generates the snapshot file.

5. The method of claim 4, wherein the file system is configured, for each reference of the inode for each snapshot file that refers to a data block of the corresponding source file, to convert the logical ditto address in that reference to a physical disk address when the data block is modified in the source file.

6. The method of claim 2, further comprising:

assigning the logical ditto address in each reference of the inode for each snapshot file in the file system that refers to any of the one or more data blocks of the selected file that were copied to the second storage level, and

updating the file system to include a managed region for each snapshot file having a reference to which a logical ditto address was assigned, each managed region controlling access to the one or more data blocks that have been copied to the second storage level through the inode for the corresponding snapshot file.

7. The method of claim 5, further comprising:

receiving a request to read or modify the selected file after the one or more data blocks of the selected file that were copied to the second storage level have been deallocated in the first storage level;

copying each of the one or more data blocks of the selected file for which a physical disk address is maintained in the corresponding reference of the inode for the selected file from the second storage level to the first storage level; and

satisfying the request to read or modify the selected file,

wherein, if a logical ditto address is maintained in the corresponding reference of the inode for the selected file for any of the one or more data blocks that are directly accessed by the request to read or modify the selected file, copying each of the one or more data blocks of each source file of the plurality of source files for which a physical disk address is maintained in the corresponding reference of the inode to the one or more data blocks that are directly accessed by the request and for which a logical ditto address is maintained in the corresponding reference of the inode for the selected file from the second storage level to the first storage level.

8. The method of claim 5, further comprising:

receiving a request to read or modify a first snapshot file within the file system having an inode with a logical ditto address in a reference that refers to one of the one or more data blocks of the selected file that was copied to the second storage level and deallocated in the first storage level;

copying each of the one or more data blocks of the selected file for which a physical disk address is maintained in the corresponding reference of the inode for the selected file from the second storage level to the first storage level; and

satisfying the request to read or modify the selected file.

9. The method of claim 6, further comprising:

receiving a request to read or modify the selected file after the one or more data blocks of the selected file that were copied to the second storage level have been deallocated in the first storage level;

copying each of the one or more data blocks of the selected file for which a physical disk address is maintained in the corresponding reference of the inode for the selected file from the second storage level to the first storage level; and

satisfying the request to read or modify the selected file,

wherein, if a logical ditto address is maintained in the corresponding reference of the inode for the selected file for any of the one or more data blocks that are directly accessed by the request to read or modify the selected file, copying each of the one or more data blocks of each source file of the plurality of source files for which a physical disk address is maintained in the corresponding reference of the inode to the one or more data blocks that are directly accessed by the request and for which a logical ditto address is maintained in the corresponding reference of the inode for the selected file from the second storage level to the first storage level.

10. The method of claim 9, further comprising:

converting each logical ditto address in each reference of the inode for each snapshot file in the file system that refers to one of the one or more data blocks that were copied to the first storage level to the physical disk address at which the corresponding data block has been copied in the first storage level.

11. The method of claim 9, further comprising:

for each snapshot file in the file system that refers to one of the one or more data blocks that were copied to the first storage level for which a subsequent request to access the snapshot file is received, converting each logical ditto address in each reference of the inode for the snapshot file to the physical disk address at which the corresponding data block has been copied in the first storage level.

12. The method of claim 6, further comprising:

receiving a request to read or modify a first snapshot file within the file system having an inode with a logical ditto address in a reference that refers to one of the one or more data blocks of the selected file that was copied to the second storage level and deallocated in the first storage level;

copying each of the one or more data blocks of the selected file for which a physical disk address is maintained in the corresponding reference of the inode for the selected file from the second storage level to the first storage level; and

satisfying the request to read or modify the selected file.

13. The method of claim 12, further comprising:

converting each logical ditto address in each reference of the inode for each snapshot file in the file system that refers to one of the one or more data blocks that were copied to the first storage level to the physical disk address to which the corresponding data block is has been copied in the first storage level.

14. The method of claim 12, further comprising:

for each snapshot file in the file system that refers to one of the one or more data blocks that were copied to the first storage level for which a subsequent request to access the snapshot file is received, converting each logical ditto address in each reference of the inode for the snapshot file to the physical disk address to which the corresponding data block has been copied in the first storage level.

15. The method of claim 5, further comprising:

receiving a request to delete or truncate the selected file after the one or more data blocks of the selected file that were copied to the second storage level have been deallocated in the first storage level; and

copying each of the one or more data blocks of the selected file for which a logical ditto address is maintained in the corresponding reference of the inode for each snapshot file in the file system that refers to one of the one or more data blocks of the selected file from the second storage level to the first storage level.

16. The method of claim 6, further comprising:

receiving a request to delete or truncate the selected file after the one or more data blocks of the selected file that were copied to the second storage level have been deallocated in the first storage level;

copying each of the one or more data blocks of the selected file for which a logical ditto address is maintained in the corresponding reference of the inode for each snapshot file in the file system that refers to one of the one or more data blocks of the selected file from the second storage level to the first storage level; and

converting at least one logical ditto address in the references of the inodes for the snapshot files in the file system that refers to each of the one or more data blocks that were copied to the first storage level to the physical disk address to which the corresponding data block has been copied in the first storage level.

17. The method of claim 1, wherein the inode for each source file of the plurality of source files has a disk address pointer to one or more indirect data blocks that contain the references to the one or more data blocks of the source file.

18. The method of claim 1, wherein the first storage level is an online memory device under the control of a processing unit upon which the file system is executed.

19. The method of claim 18, wherein the online memory device comprises a hard disk drive array.

20. The method of claim 18, wherein the second storage level comprises an offline memory device that communicates with the file system via a network.

21. The method of claim 20, wherein the offline memory device comprises storage medium selected from floppy disks, optical disks, and magnetic tape.

22. The method of claim 1, wherein the selected file is determined based on criteria specified by a file system administrator.

23. The method of claim 1, wherein the file system conforms to the Xopen file system standard.

24. A computer-usable medium having computer readable instructions stored thereon for execution by a processor to perform a method for managing the storage of a file that has been selected for migration from a first storage level within a file system to a second storage level, the selected file being selected from a plurality of source files each organized within the file system into one or more data blocks, the file system having an inode for each source file of the plurality of source files, each inode having a reference to each of the one or more data blocks of the corresponding source file, the file system being configured to regularly generate snapshot files of the source files of the plurality of source files, the file system having an inode for each snapshot file, the inode for each snapshot file having a reference to each of the one or more data blocks of the corresponding source file at the time the snapshot file was generated, the method comprising:

copying each of the one or more data blocks of the selected file that are stored in the first storage level at a physical disk address maintained in the corresponding reference of the inode for the selected file from the first storage level to the second storage level;

maintaining a logical ditto address in each reference of the inode for each snapshot file in the file system that refers to one of the one or more data blocks of the selected file that were copied to the second storage level, each logical ditto address indicating that additional processing is required to obtain the physical disk address of the data block referred to by the reference; and

updating the file system to include a managed region for controlling access to the one or more data blocks through the inode for the selected file.

25. A data processing system comprising:

a central processing unit;

a random access memory for storing data and programs for execution by the central processing unit;

a first storage level comprising a nonvolatile storage device; and

computer readable instructions stored in the random access memory for execution by central processing unit to perform a program for managing the storage of a file that has been selected for migration from the first storage level within a file system to a second storage level, the selected file being selected from a plurality of source files each organized within the file system into one or more data blocks, the file system having an inode for each source file of the plurality of source files, each inode having a reference to each of the one or more data blocks of the corresponding source file, the file system being configured to regularly generate snapshot files of the source files of the plurality of source files, the file system having an inode for each snapshot file, the inode for each snapshot file having a reference to each of the one or more data blocks of the corresponding source file at the time the snapshot file was generated, wherein the program causes operations to be performed, the operations comprising;

copying each of the one or more data blocks of the selected file that are stored in the first storage level at a physical disk address maintained in the corresponding reference of the inode for the selected file from the first storage level to the second storage level;

maintaining a logical ditto address in each reference of the inode for each snapshot file in the file system that refers to one of the one or more data blocks of the selected file that were copied to the second storage level, each logical ditto address indicating that additional processing is required to obtain the physical disk address of the data block referred to by the reference; and

updating the file system to include a managed region for controlling access to the one or more data blocks through the inode for the selected file.