Sparse recall and writes for archived and transformed data objects

• US 9,946,729 B1
• Filed: 10/31/2008
• Issued: 04/17/2018
• Est. Priority Date: 03/21/2005
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Inventors: Stacey, Christopher H. E., Gross, Jerald W., Batchu, Ravi Venkata, Chuba, Christian J., Forrester, Walter
• Primary Examiner(s): Torgrimson, Tyler
• Application Number:
  US12/262,316
• Time in Patent Office: 3,455 Days
• Field of Search: None
• CPC Class Codes: G06F 16/185 Hierarchical storage management [HSM] systems, e.g. file migration or policies thereof details of archiving G06F16/11

Abstract

This invention is a system and a method for operating a storage server in a data network using a new architecture. The system of operating a storage server includes a secondary storage that archives or stores a data object in transformed form such that the clients of the storage server can no longer access the data directly. The method of accessing the data object that is archived or transformed in a different form to the secondary storage includes recalling only those data blocks that are required for read or write operation.

Claims

1. A method comprising:

receiving a request from a client of a storage system to access a data object from a primary storage of the storage system;

determining whether the requested data object is stored in a first form at the primary storage or is stored in a second form at a secondary storage of the storage system;

identifying that the requested data object is stored in the second form at the secondary storage according to a stub file, including attributes of the data object, stored in the primary storage, replacing the data object, and linking to the data object stored in the second form at the secondary storage;

determining which data blocks of the data object requested by the client that are not stored in the primary storage, if any, are stored in the secondary storage and are required in order to satisfy the request according to a migration inode associated with the stub file stored in the primary storage;

recalling the data blocks of the data object, that are determined to be stored in the secondary storage and are required to satisfy the request, from the secondary storage; and

returning data, including the data blocks recalled from the secondary storage, to the client to satisfy the request.

2. The method of claim 1, wherein the data object is represented at the primary storage by a file or a logical unit number comprised of one or more data blocks.

3. The method of claim 1, wherein the storage system is represented by a file server configured to present a logical view of a file to the client, wherein at least a portion of the file is stored in the secondary form at the secondary storage.

4. The method of claim 1

wherein the receiving the request from the client of the storage system to access the data object from the primary storage of the storage system comprises receiving a read operation from the client;

and

wherein the recalling the data blocks of the data object, that are determined to be stored in the secondary storage and are required to satisfy the request, from the secondary storage comprises recalling the data blocks required to satisfy the read operation.

5. The method of claim 4

wherein the receiving the read operation from the client comprises receiving a specified offset in a data object;

wherein the recalling the data blocks required to satisfy the read operation comprises recalling the data block that includes data for the specified offset in the data object; and

wherein the data object is archived to the secondary storage such that the client of the storage system can no longer access data of the data object directly from the primary storage.

6. The method of claim 4

wherein the receiving a read operation from the client comprises receiving a specified offset in a data object;

wherein the recalling the data blocks required to satisfy the read operation comprises recalling the data block that includes data for the specified offset in the data object; and

wherein the data object is transformed from the primary storage and relocated to the secondary storage such that the client of the storage system can no longer access data of the data object directly from the primary storage.

7. The method of claim 1

wherein the receiving the request from the client of the storage system to access a data object from the primary storage of the storage system comprises receiving a write operation from the client;

and

wherein the recalling the data blocks of the data object, that are determined to be stored in the secondary storage and are required to satisfy the request, from the secondary storage comprises;

recalling the data blocks required for the write operation; and

modifying the recalled data blocks according to the write operation.

8. The method of claim 7

wherein the receiving the write operation from the client comprises receiving a specified offset in a data object;

wherein the recalling the data blocks required to satisfy the write operation comprises recalling the data block that includes data for the specified offset in the data object;

wherein the modifying the recalled data blocks according to the write operation comprises modifying the recalled data block that includes the data modified at the specified offset by the write operation in the data object; and

wherein the data object is archived from the primary storage to the secondary storage such that the client of the storage system can no longer access data of the data object directly from the primary storage.

9. The method of claim 7

wherein the receiving the write operation from the client comprises receiving the specified offset in the data object;

wherein the recalling the data blocks required to satisfy the write operation comprises recalling the data block that includes data for the specified offset in the data object;

wherein the modifying the recalled data blocks according to the write operation comprises modifying the data block that includes the data modified at the specified offset by the write operation in the data object; and

wherein the data object is transformed from the primary storage and relocated to the secondary storage such that the client of the storage system can no longer access data of the data object directly from the primary storage.

10. The method of claim 7 further comprising:

receiving the write operation from the client including a specified offset in a data object;

determining whether a first size of data to be written according to the write operation from the client is less than a second size of a data block size of the storage system;

upon determining that the first size of the data to be written according to the write operation from the client is less than the second size of the data block size of the storage system, recalling the data block that includes data for the specified offset in the data object; and

modifying the recalled data block that includes the data modified at the specified offset by the write operation in the data object.

11. A system comprising:

a storage system including a primary storage and a secondary storage; and

a program logic in communication with the storage system for carrying out the steps of;

receiving a request from a client of the storage system to access a data object from the primary storage of the storage system;

determining whether the requested data object is stored in a first form at the primary storage or is stored in a second form at the secondary storage;

identifying that the requested data object is stored in the second form at the secondary storage according to a stub file, including attributes of the data object, stored in the primary storage, replacing the data object, and linking to the data object stored in the second form at the secondary storage;

determining which data blocks of the data object requested by the client that are not stored in the primary storage, if any, are stored in the secondary storage and are required in order to satisfy the request according to a migration inode associated with the stub file stored in the primary storage;

recalling the data blocks of the data object, that are determined to be stored in the secondary storage and are required to satisfy the request, from the secondary storage; and

returning data, including the data blocks recalled from the secondary storage, to the client to satisfy the request.

12. The system of claim 11, wherein the data object is represented at the primary storage by a file or a logical unit number comprised of one or more data blocks.

13. The system of claim 11, wherein the storage system is represented by a file server configured to present a logical view of a file to the client, wherein at least a portion of the file is stored in the secondary form at the secondary storage.

14. The system of claim 11

wherein the receiving the request from the client of the storage system to access the data object from the primary storage of the storage system comprises receiving a read operation from the client;

and

wherein the recalling the data blocks of the data object, that are determined to be stored in the secondary storage and are required to satisfy the request, from the secondary storage comprises recalling the data blocks required to satisfy the read operation.

15. The system of claim 14

wherein the receiving the read operation from the client comprises receiving a specified offset in the data object;

wherein recalling the data blocks required to satisfy the read operation comprises recalling the data block that includes data for the specified offset in the data object; and

wherein the data object is archived to the secondary storage such that the client of the storage system can no longer access data of the data object directly from the primary storage.

16. The system of claim 14

wherein receiving the read operation from the client comprises receiving a specified offset in a data object;

wherein recalling the data blocks required to satisfy the read operation comprises recalling the data block that includes data for the specified offset in the data object; and

wherein the data object is transformed from the primary storage and relocated to the secondary storage such that the client of the storage system can no longer access data of the data object directly from the primary storage.

17. The system of claim 11

wherein the receiving the request from the client of the storage system to access the data object from the primary storage of the storage system comprises receiving a write operation from the client;

and

wherein the recalling the data blocks of the data object, that are determined to be stored in the secondary storage and are required to satisfy the request, from the secondary storage comprises;

recalling the data blocks required for the write operation; and

modifying the recalled data blocks according to the write operation.

18. The system of claim 17

wherein receiving the write operation from the client comprises receiving a specified offset in a data object;

wherein the recalling the data blocks required to satisfy the write operation comprises recalling the data block that includes data for the specified offset in the data object;

wherein the modifying the recalled data blocks according to the write operation comprises modifying the recalled data block that includes the data modified at the specified offset by the write operation in the data object; and

wherein the data object is archived from the primary storage to the secondary storage such that the client of the storage system can no longer access data of the data object directly from the primary storage.

19. The system of claim 17

wherein the receiving the write operation from the client comprises receiving a specified offset in a data object;

wherein the recalling the data blocks required to satisfy the write operation comprises recalling the data block that includes data for the specified offset in the data object;

wherein the modifying the recalled data blocks according to the write operation comprises modifying the data block that includes the data modified at the specified offset by the write operation in the data object, and

wherein the data object is transformed from the primary storage and relocated to the secondary storage such that the client of the storage system can no longer access data of the data object directly from the primary storage.

20. A computer program product including a non-transitory computer readable storage medium having computer program code encoded thereon that, when executed on a processor of a computer, causes the computer to perform the operations of:

receiving a request from a client of a storage system to access a data object from a primary storage of the storage system;

determining whether the requested data object is stored in a first form at the primary storage or is stored in a second form at a secondary storage of the storage system;

identifying that the requested data object is stored in the second form at the secondary storage according to a stub file, including attributes of the data object, stored in the primary storage, replacing the data object, and linking to the data object stored in the second form at the secondary storage;

determining which data blocks of the data object requested by the client that are not stored in the primary storage, if any, are stored in the secondary storage and are required in order to satisfy the request according to a migration inode associated with the stub file stored in the primary storage;

recalling the data blocks of the data object, that are determined to be stored in the secondary storage and are required to satisfy the request, from the secondary storage; and

returning data, including the data blocks recalled from the secondary storage, to the client to satisfy the request.