Architecture for supporting sparse volumes

• US 7,689,609 B2
• Filed: 04/24/2006
• Issued: 03/30/2010
• Est. Priority Date: 04/25/2005
• Status: Active Grant

• Current Assignee: Network Appliance Incorporated (NetApp, Inc.)
• Original Assignee: NetApp, Inc.
• Inventors: English, Robert M., Zheng, Ling, Amdur, Matthew Benjamin, Grunwald, David, Lango, Jason Ansel, Ayyar, Kartik, Tsai, Robert Lieh-Yuan, Quirion, Brian Mederic, Prakash, Ashish, Ackaouy, Emmanuel, Kesavan, Ram, Wagner, J. Christopher, Eastham, Paul
• Primary Examiner(s): Woo; Isaac M
• Assistant Examiner(s): Ng; Amy
• Application Number:
  US11/409,624
• Publication Number:
  US 20070250551A1
• Time in Patent Office: 1,436 Days
• Field of Search: 707/1, 707/204, 707/201, 707/202, 711/161, 711/162, 714/15
• US Class Current: 707/679
• CPC Class Codes: G06F 16/10 File systems; File servers

Abstract

An architecture, including a file-level protocol, for supporting sparse volumes on a storage system is provided. The file-level protocol provides coherency checking for use in retrieving data stored on a backing store remote from a storage system.

Claims

1. A first storage system that supports remote retrieval of data, comprising:

a storage operating system executed by a processor, the storage operating system configured to generate a sparse volume, wherein the sparse volume comprises a space reserved data container with an appearance that a full restore was completed, the storage operating system further configured to mark a block pointer of the sparse volume with a value to identify that a block of the sparse volume is not stored on the first storage system and further configured to retrieve the data of the block at a later time than initialization of the sparse volume using a protocol configured for remote retrieval of the data of the block from a backing store, wherein the backing store is a second storage system that stores backup data and is separate from the first storage system; and

a protocol module operatively interconnected with the storage operating system and implementing the protocol configured for remote retrieval of the data of the block from the backing store in response to a determination that the block pointer of the sparse volume has the value, the protocol module further configured to return the data of the block to the sparse volume once the data of the block has been retrieved from the backing store.

2. The first storage system of claim 1 further comprising a function configured to load a block of the storage operating system.

3. The first storage system of claim 1 wherein the protocol utilizes a transport control protocol/internet protocol for a transport layer.

4. The first storage system of claim 1 wherein the protocol comprises a read request and the read request comprises a file handle field, a file block number field and a number of blocks to be read field.

5. The first storage system of claim 1 wherein the protocol comprises an authentication request and the authentication request comprises an application field, a type field and a data field.

6. The first storage system of claim 1 wherein the protocol comprises an authentication response and the authentication response comprises a status field and a data field.

7. The first storage system of claim 1 wherein the protocol comprises a request to lock a persistent consistency point image (PCPI) and the request comprises a storage operating system identifier field and a PCPI name field.

8. The first storage system of claim 1 wherein the protocol comprises an initialize request, wherein the initialize request comprises a protocol version field, an application field, and a byte order field.

9. The first storage system of claim 1 wherein the protocol comprises an initialize response, wherein the initialize response comprises a storage operating system version field and a byte order field.

10. The first storage system of claim 1 wherein the protocol comprises a set of operations that return attributes of a data container prior to an operation of the set of operations and following the operation of a set of operations, wherein the set of operations comprises a remove operation, a rename operation, and a create operation.

11. The first storage system of claim 1 wherein the protocol comprises a read request.

12. The first storage system of claim 1 wherein the protocol comprises an authentication request.

13. The first storage system of claim 1 wherein the protocol comprises an authentication response.

14. The first storage system of claim 1 wherein the protocol comprises a request to lock a persistent consistency point image (PCPI).

15. The first storage system of claim 1 wherein the protocol comprises an initialize request.

16. The first storage system of claim 1 wherein the protocol comprises an initialize response.

17. The first storage system of claim 1 wherein the backing store comprises one or more storage systems.

18. The first storage system of claim 1 wherein the protocol comprises a set of operations that return attributes of a data container prior to an operation of the set of operations and following the operation of the set of operation.

19. The first storage system of claim 18 wherein the set of operations comprises a remove operation.

20. The first storage system of claim 18 wherein the set of operations comprises a rename operation.

21. The first storage system of claim 18 wherein the set of operations comprises a create operation.

23. The system of claim 1 wherein the protocol module is further configured to provide information to permit caching of data modified on an origin server.

24. The system of claim 1 wherein the protocol module is further configured to couple an open protocol to the protocol module.

22. The system of clam 1 wherein the storage operating system is further configured to assign pointer values to indirect blocks of data retrieved from the backing store and stored locally.

25. A method for remotely retrieving data by a first storage system from a backing store using a protocol, comprising:

generating a sparse volume within the first storage system at a first time, wherein the sparse volume comprises a space reserved data container with an appearance that a full restore was completed;

initializing the sparse volume to accept data access requests at a second time, wherein the first time is before the second time;

initializing, at a third time, a connection between the first storage system and the backing store, wherein the backing store is a second storage system that stores a backup copy of the data and is separate from the first storage system and the second time is before the third time;

marking a block pointer of the sparse volume with a value to identify that a block of the sparse volume is not stored on the first storage system;

sending, in response to a determination that the block pointer of the sparse volume has the value, one or more data access requests to the backing store to retrieve data of the block not stored on the first storage system;

in response to the one or more data access requests, retrieving the data of the block from the backing store;

sending one or more data access responses to the first storage system, where the one or more data access responses comprise the data of the block; and

storing the data of the block in the sparse volume within the first storage system.

26. The method of claim 25 further comprising sending a lock PCPI command to the first storage system.

27. The method of claim 25 wherein initializing the connection between the first storage system and the backing store further comprises:

(a) sending an authentication request to the backing store;

(b) sending an authentication response to the first storage system by the backing store;

(c) repeating steps (a)-(b) until a connection is authenticated;

(d) sending an initialization request to the backing store; and

(e) sending an initialization response to the first storage system.

28. A computer readable storage medium storing executable program instructions executed by a processor, comprising:

program instructions that generate a sparse volume within a first storage system, wherein the sparse volume comprises a space reserved data container with an appearance that a full restore was completed;

program instructions that initialize the sparse volume to accept data access requests at a second time, wherein the first time is before the second time;

program instructions that initialize at a third time, a connection between the first storage system and a backing store, wherein the backing store is a second storage system that stores a backup copy of the data and is separate from the first storage system and the second time is before the third time;

program instructions that mark a block pointer of the sparse volume with a value to identify that a block of the sparse volume is not stored on the first storage system;

program instructions that send, in response to a determination that the block pointer of the sparse volume has the value, one or more data access requests to the backing store to retrieve data of the block not stored on the first storage system;

in response to the one or more data access requests, program instructions that retrieve the data of the block from the backing store;

program instructions that send one or more data access responses to the first storage system, wherein the one or more data access responses comprise the data of the block; and

program instructions that store the data of the block in the sparse volume within the first storage system.

29. The computer readable storage medium of claim 28 wherein initializing the connection between the first storage system and the backing store further comprises:

(a) sending an authentication request to the backing store;

(b) sending an authentication response to the first storage system by the backing store;

(c) repeating steps (a)-(b) until a connection is authenticated;

(d) sending an initialization request to the backing store; and

(e) sending an initialization response to the first storage system.

30. A method, comprising:

generating a sparse volume within a first storage system at a first time, wherein the sparse volume comprises a space reserved data container with an appearance that a full restore was completed;

marking a block pointer of the sparse volume with a value to identify that one or more blocks of the sparse volume are not stored on the first storage system;

initializing the sparse volume to accept data access requests at a second time, wherein the first time is before the second time;

identifying, at a third time, the one or more data blocks to retrieve from the second storage system, wherein the second time is before the third time;

initializing a connection between the first storage system and the second storage system, wherein the second storage system stores a backup copy of data stored on the first storage system and is separate from the first storage system;

retrieving the one or more data blocks not stored on the first storage system from the second storage system in response to a determination that the block pointer of the sparse volume has the value; and

storing the one or more data blocks in the sparse volume within the first storage system.

31. A computer data storage system, comprising:

a first storage system configured with a file system executed by a processor, the file system configured to create a sparse volume, wherein the sparse volume comprises a space reserved data container with an appearance that a full restore was completed;

a backing store configured to store a backup copy of data stored on the first storage system, wherein the backing store is a second separate storage system located in a different location than the first storage system;

the file system further configured to mark a block pointer of the sparse volume with a value to identify, at a later time than creating the sparse volume, one or more blocks of data that are not stored on the first storage system, the file system further configured to retrieve the data using a protocol, wherein the protocol is configured to remotely retrieve the one or more blocks of data from the backing store; and

a protocol module operatively interconnected with the file system and implementing the protocol configured for remote retrieval of the one or more blocks of data from the backing store in response to a determination that the block pointer of the sparse volume has the value, the protocol module further configured to return the one or more blocks of data to the sparse volume once the one or more blocks of data have been retrieved from the backing store.

32. A computer data storage system, comprising:

a first storage system configured with a plurality of storage devices organized to form an aggregate and a plurality of virtual volumes stored within the aggregate;

a backing store configured to store a backup copy of data stored on the first storage system;

an operating system executed by a processor of the first storage system, the operating system configured to generate a sparse volume and allow data access requests to the sparse volume, wherein the sparse volume comprises a space reserved data container with an appearance that a full restore was completed, wherein the operating system is further configured to mark a block pointer of the sparse volume with a value to identify that a block of the sparse volume is not stored on the first storage system; and

a protocol module operatively interconnected with the operating system and implementing the protocol configured for remote retrieval of block data from the backing store at a later time than generating the sparse volume, in response to a determination that the block pointer of the sparse volume has the value, the protocol module further configured to return the block data to the sparse volume once the block data has been retrieved from the backing store.