Emulation of transparent recall in a hierarchical storage management system

• US 7,853,667 B1
• Filed: 08/05/2005
• Issued: 12/14/2010
• Est. Priority Date: 08/05/2005
• Status: Active Grant

• Current Assignee: Network Appliance Incorporated (NetApp, Inc.)
• Original Assignee: Network Appliance Incorporated (NetApp, Inc.)
• Inventors: Yoder, Alan G.
• Primary Examiner(s): Vaughn, Jr.; William C
• Assistant Examiner(s): Gupta; Muktesh G
• Application Number:
  US11/197,931
• Time in Patent Office: 1,957 Days
• Field of Search: 709/217, 707/200, 707/201, 707/604, 711/161
• US Class Current: 709/217
• CPC Class Codes: G06F 16/9574 of access to content, e.g. by caching
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/0649 Lifecycle management
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

In at least one embodiment of the invention, a primary storage facility is managed in an HSM system. Data is relocated from the primary storage facility to a secondary storage facility. A request is received from a client for only a portion of the relocated data. In response to the request, the requested portion of the data is obtained from the secondary storage facility and stored in the primary storage facility as a sparse file. The requested portion of the data is then provided to the client from the sparse file.

Claims

1. A method comprising:

managing a primary storage facility in a hierarchical storage management (HSM) system;

relocating data from the primary storage facility to a secondary storage facility in the HSM system;

receiving, by the primary storage facility, a request from a client for only a portion of the relocated data, the portion comprising less than the entire relocated data; and

responding to the request by

obtaining, by the primary storage facility, only the requested portion of the relocated data from the secondary storage facility, wherein a non-requested portion of the relocated data remains in the secondary storage facility,

allocating a sparse file on disk in the primary storage facility in response to obtaining the requested portion of the relocated data,

storing the requested portion of the relocated data in the primary storage facility in the sparse file, and

providing the requested portion of the relocated data to the client from the sparse file.

2. A method as recited in claim 1, wherein the primary storage facility comprises a plurality of mass storage devices.

3. A method as recited in claim 1, wherein the primary storage facility comprises a storage server, and wherein the secondary storage facility is not directly managed by the storage server.

4. A method as recited in claim 1, wherein the secondary storage facility comprises a NearLine storage facility.

5. A method as recited in claim 1, wherein:

the primary storage facility comprises a storage server;

the primary storage facility comprises a plurality of mass storage devices;

the secondary storage facility comprises a NearLine storage facility which is not directly managed by the storage server.

6. A method as recited in claim 1, further comprising:

maintaining at the primary storage facility a stub representing the data when the data has been relocated to the secondary storage facility;

wherein storing the requested portion of the relocated data in the primary storage facility as a sparse file comprises replacing the stub with the sparse file.

7. A method as recited in claim 6, further comprising:

replacing the sparse file in the primary storage facility with a stub representing the relocated data after providing the requested portion to said one of the clients.

8. A method as recited in claim 1, wherein the storage server includes a main memory and the primary storage facility includes a plurality of non-volatile storage devices, and wherein replacing the stub in the primary storage facility with the requested portion of the relocated data as a sparse file comprises storing the sparse file in at least one of the non-volatile storage devices.

9. A method as recited in claim 8, wherein servicing the request does not include storing the requested portion of the relocated data in the main memory of the storage server.

10. A method of operating a storage server in hierarchical storage management (HSM) system, the method comprising:

operating the storage server to store data on behalf of a plurality of clients;

using an HSM server to relocate the data from a primary storage facility managed by the storage server to a NearLine storage facility, wherein the primary storage facility includes a plurality of mass storage devices;

maintaining at the storage server a stub representing the relocated data;

receiving from one of the clients a request for only a portion of the relocated data, the portion comprising less than the entire relocated data; and

servicing the request by

reading only the requested portion of the relocated data from the NearLine storage facility, wherein a non-requested portion of the relocated data remains in the NearLine storage facility,

allocating a sparse file on disk in the storage server in response to reading the requested portion of the relocated data,

replacing the stub with the requested portion of the relocated data as a in the sparse file,

providing the requested portion of the relocated data from the storage server to said one of the clients from the sparse file, and

replacing the sparse file in the storage server with a stub representing the relocated data.

11. A method as recited in claim 10, wherein the storage server includes a main memory and uses a non-volatile storage facility to store data on behalf of the clients, and wherein replacing the stub in the storage server with the requested portion of the relocated data as a sparse file comprises storing the sparse file in the non-volatile storage facility.

12. A method as recited in claim 11, wherein servicing the request does not include storing the requested portion of the relocated data in the main memory of the storage server.

13. A storage server to operate in hierarchical storage management (HSM) system, the storage server comprising:

a processor;

a network interface through which to receive data access requests from a plurality of clients and through to provide requested data to the clients;

a storage interface through which to access a plurality of mass storage devices in a primary storage facility of the HSM system, in response to requests from the clients; and

a memory storing a file system to manage files stored in the primary storage facility, the memory further storing instructions for execution by the processor to cause the storage server to;

relocate a file from the primary storage facility to a secondary storage facility in the HSM system, wherein the secondary storage facility is not directly managed by the storage server; and

respond to a request for a portion of the relocated file from a client by allocating a sparse file on disk in the primary storage facility in response to obtaining the requested portion of the relocated data, storing only the requested portion of the relocated file in the primary storage facility in the sparse file and providing the requested portion of the relocated file from the storage server to the client from the sparse file, wherein the portion comprises less than the entire relocated file and a non-requested portion of the relocated file remains in secondary storage facility.

14. A storage server as recited in claim 13, wherein the secondary storage facility comprises a NearLine storage device.

15. A storage server as recited in claim 13, further comprising:

maintaining at the storage server a stub representing the file when the file has been relocated to the secondary storage facility;

wherein storing the requested portion of the relocated file in the primary storage facility as a sparse file comprises replacing the stub with the sparse file.

16. A storage server as recited in claim 15, further comprising:

replacing the sparse file with a stub representing the relocated file after providing the requested portion of the relocated file to the client.

17. A storage server as recited in claim 13, wherein the storage server comprises a main memory, and wherein said storing the requested portion of the relocated file in the primary storage facility as a sparse file does not include storing the requested portion of the relocated file in the main memory.