Selection of migration methods including partial read restore in distributed storage management

• US 20060212746A1
• Filed: 03/21/2005
• Published: 09/21/2006
• Est. Priority Date: 03/21/2005
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: Compton, James T., Stacey, Christopher H., Gross, Jerald W., Amegadzie, Augustine, Zimran, Eyal, Rago, Stephen A.
• Granted Patent: US 8,055,724 B2
• Time in Patent Office: Days
• US Class Current: 714/6
• CPC Class Codes: G06F 16/185 Hierarchical storage management [HSM] systems, e.g. file migration or policies thereof details of archiving G06F16/11

Abstract

A hierarchical storage system is constructed from file servers and a policy engine server by building upon a file migration service. Offline attributes are added to file system inodes in a primary file server, file system parameters are added in the primary server, an offline read access method field is added to a connection database, and the primary file server is programmed to use these attributes and parameters for selecting a particular method (such as a partial read migration or a pass through read method) for read access to an offline file. In this fashion, the primary file server is provided with flexibility for selecting a particular one of a number of migration methods depending on various conditions, in order to improve the trade-off between the cost of storage and delay in file access.

Claims

1. A method of operating a file server in a data network for access to file data that is migrated between primary storage and secondary storage, the file server responding to a client request for read access to a specified file by checking whether file data of the specified file has been migrated from the primary storage to the secondary storage, and upon finding that file data of the specified file has been migrated from the primary storage to the secondary storage, selecting a particular one of a number of offline file read access methods that are selectable by the file server under various conditions, the offline file read access methods including a partial read migration method for migrating only a portion of the data of an offline file from the secondary storage to the primary storage for retention in the primary storage after client read access, and a pass-through method in which the file server obtains file data from the secondary storage and passes on to a client the file data obtained from the secondary storage without retention in the primary storage of the file data obtained from the secondary storage.

2. The method as claimed in claim 1, wherein a plurality of identical copies of the specified file are migrated to different locations in the secondary storage, and the file server maintains a list of the different locations of the specified file in the secondary storage, and the file server accesses the list for selecting one of the different locations of the specified file in the secondary storage for obtaining file data of the specified file from the selected one of the different locations of the specified file in the secondary storage.

3. The method as claimed in claim 1, wherein a plurality of different versions of the specified file are stored in different locations in the secondary storage, each of the different versions of the specified file is a different state of the specified file that existed at a respective point in time, and the file server maintains a list of the different locations in the secondary storage in which the different versions of the specified file are stored.

4. The method as claimed in claim 1, wherein the partial read migration method migrates a certain number of additional file system data blocks for the specified file in addition to at least one data block for the specified file including specific data requested by the client.

5. The method as claimed in claim 1, wherein the partial read migration method migrates file system data blocks for the specified file up to a file system data block including data for a specified offset in the specified file and a certain number of additional file system data blocks for the specified file past the file system data block including data for the specified offset in the specified file.

6. The method as claimed in claim 1, which further includes the file server responding to a request to truncate a partially migrated offline file to a new offset by checking whether data of the partially migrated offline file up to the new offset is presently resident in the primary storage of the file server, and upon finding that data of the partially migrated offline file up to the new offset is presently resident in the primary storage of the file server, truncating the partially migrated offline file and setting the partially migrated offline file to online status.

7. The method as claimed in claim 1, which includes the file server returning an error indicating that access to the specified file has been denied because the specified file is offline and offline access to a file system including the specified file has not been enabled.

8. The method as claimed in claim 1, which includes the file server selecting the pass-through read method upon finding that the specified file is in a read-only file system.

9. The method as claimed in claim 1, which includes the file server checking whether a particular one of the number of offline file read access methods that are selectable by the file server under various conditions has been set as a run-time option for a file system containing the specified file, and upon finding that a particular one of the number of offline file read access methods that are selectable by the file server under various conditions has been set as a run-time option for a file system containing the specified file, selecting the particular one of the number of offline file read access methods that are selectable by the file server under various conditions that has been set as a run-time option for a file system containing the specified file.

10. The method as claimed in claim 1, which includes the file server checking whether the client request for read access was made by a particular kind of client for which a particular one of the number of offline file read access methods that are selectable by the file server under various conditions that has been set, and upon finding that the client request for read access was made by a particular kind of client for which a particular one of the number of offline file read access methods that are selectable by the file server under various conditions has been set, selecting the particular one of the number of offline file read access methods that are selectable by the file server under various conditions that has been set for the particular kind of client.

11. The method as claimed in claim 1, which includes the file server responding to read requests from an archiving client by checking whether runtime options have been set for reading or not reading offline attributes of offline files, and for reading or not reading offline data of offline files.

12. The method as claimed in claim 1, which includes the file server checking whether a particular one of the number of offline file read access methods that are selectable by the file server under various conditions has been set in an offline attribute of the specified file, and upon finding that a particular one of the number of offline file read access methods that are selectable by the file server under various conditions has been set in an offline attribute of the specified file, selecting the particular one of the number of offline file read access methods that are selectable by the file server under various conditions that has been set in an offline attribute of the specified file.

13. The method as claimed in claim 1, which includes the file server checking whether a particular one of the number of offline file read access methods that are selectable by the file server under various conditions has been set up for a connection to be used for migration of data of the specified file, and upon finding that a particular one of the number of offline file read access methods that are selectable by the file server under various conditions has been set up for a connection to be used for migration of data of the specified file, selecting the particular one of the number of offline file read access methods that are selectable by the file server under various conditions that has been set up for the connection to be used for migration of data of the specified file.

14. The method as claimed in claim 1, which includes the file server scheduling recall of the specified file from the secondary storage at a certain time in the future, and recalling the specified file from the secondary storage at the certain time in the future upon finding that the specified file is offline at the certain time in the future.

15. Apparatus comprising a file server for access to file data that is migrated between primary storage and secondary storage, the file server being programmed for responding to a client request for read access to a specified file by checking whether file data of the specified file has been migrated from the primary storage to the secondary storage, and upon finding that file data of the specified file has been migrated from the primary storage to the secondary storage, selecting a particular one of a number of offline file read access methods that are selectable by the file server under various conditions, the offline file read access methods including a partial read migration method for migrating only a portion of the data of an offline file from the secondary storage to the primary storage for retention in the primary storage after client read access, and a pass-through method in which the file server obtains file data from the secondary storage and passes on to a client the file data obtained from the secondary storage without retention in the primary storage of the file data obtained from the secondary storage.

16. The apparatus as claimed in claim 15, wherein the file server is programmed so that the partial read migration method migrates file system data blocks for the specified file up to a file system data block including data for a specified offset in the specified file and a certain number of additional file system data blocks for the specified file following the file system data block including data for the specified offset in the specified file.

17. The apparatus as claimed in claim 15, wherein:

the file server is programmed for checking whether a particular one of the number of offline file read access methods that are selectable by the file server under various conditions has been set as a run-time option for a file system containing the specified file, and upon finding that a particular one of the number of offline file read access methods that are selectable by the file server under various conditions has been set as a run-time option for a file system containing the specified file, selecting the particular one of the number of offline file read access methods that are selectable by the file server under various conditions has been set as a run-time option for a file system containing the specified file; and

wherein the file server is programmed for checking whether a particular one of the number of offline file read access methods that are selectable by the file server under various conditions has been set in an offline attribute of the specified file, and upon finding that a particular one of the number of offline file read access methods that are selectable by the file server under various conditions has been set in an offline attribute of the specified file, selecting the particular one of the number of offline file read access methods that are selectable by the file server under various conditions that has been set in an offline attribute of the specified file.

18. The apparatus as claimed in claim 15, wherein:

the file server is programmed for checking whether a particular one of the number of offline file read access methods that are selectable by the file server under various conditions has been set in an offline attribute of the specified file, and upon finding that a particular one of the number of offline file read access methods that are selectable by the file server under various conditions has been set in an offline attribute of the specified file, selecting the particular one of the number of offline file read access methods that are selectable by the file server under various conditions that has been set in an offline attribute of the specified file; and

wherein the file server is programmed for checking whether a particular one of the number of offline file read access methods that are selectable by the file server under various conditions has been set up for a connection to be used for migration of data of the specified file, and upon finding that a particular one of the number of offline file read access methods that are selectable by the file server under various conditions has been set up for a connection to be used for migration of data of the specified file, selecting the particular one of the number of offline file read access methods that are selectable by the file server under various conditions that has been set up for the connection to be used for migration of data of the specified file.

19. Apparatus comprising a primary file server for access to file data that is migrated between primary storage of the primary file server and secondary storage of at least one secondary file server, the primary file server being programmed for responding to a client request for read access to a specified file by checking whether file data of the specified file has been migrated from the primary storage to the secondary storage, and upon finding that file data of the specified file has been migrated from the primary storage to the secondary storage, selecting a particular one of a number of offline file read access methods that are selectable by the primary file server under various conditions;

wherein the primary file server is programmed for checking whether a particular one of the number of offline file read access methods that are selectable by the primary file server under various conditions has been set as a run-time option for a file system containing the specified file, and upon finding that a particular one of the number of offline file read access methods that are selectable by the primary file server under various conditions has been set as a run-time option for a file system containing the specified file, selecting the particular one of the number of offline file read access methods that are selectable by the primary file server under various conditions that has been set as a run-time option for a file system containing the specified file;

wherein the primary file server is programmed for checking whether a particular one of the number of offline file read access methods that are selectable by the primary file server under various conditions has been set up for a connection to be used for migration of data of the specified file, and upon finding that a particular one of the number of offline file read access methods that are selectable by the primary file server under various conditions has been set up for a connection to be used for migration of data of the specified file, selecting the particular one of the number of offline file read access methods that are selectable by the primary file server under various conditions that has been set up for the connection to be used for migration of data of the specified file; and

wherein the primary file server is programmed for checking whether a particular one of the number of offline file read access methods that are selectable by the primary file server under various conditions has been set in an offline attribute of the specified file, and upon finding that a particular one of the number of offline file read access methods that are selectable by the primary file server under various conditions has been set in an offline attribute of the specified file, selecting the particular one of the number of offline file read access methods that are selectable by the primary file server under various conditions that has been set in an offline attribute of the specified file.

20. The apparatus as claimed in claim 19, which further includes a policy engine server, and a data network coupling the policy engine server and said at least one secondary file server to the primary file server, the policy engine server being programmed for enforcing a storage management policy by instructing the primary file server to perform migration of the file data from the primary storage of the primary file server to the secondary storage.