System and method for restoring data on demand for instant volume restoration

• US 7,809,693 B2
• Filed: 04/24/2006
• Issued: 10/05/2010
• Est. Priority Date: 02/10/2003
• Status: Expired due to Term

• Current Assignee: NetApp, Inc.
• Original Assignee: NetApp, Inc.
• Inventors: Cho, Yong Eun, English, Robert M., Manley, Stephen L., Zheng, Ling, Edwards, John K., Lango, Jason Ansel, Ackaouy, Emmanuel, Eastham, Paul Christopher
• Primary Examiner(s): Mahmoudi; Tony
• Assistant Examiner(s): ALLEN, BRITTANY N
• Application Number:
  US11/409,626
• Publication Number:
  US 20070124341A1
• Time in Patent Office: 1,625 Days
• Field of Search: 707/202, 707/679, 707/686
• US Class Current: 707/679
• CPC Class Codes: G06F 16/10 File systems; File servers

Abstract

A technique is disclosed for restoring data of sparse volumes, where one or more block pointers within the file system structure are marked as ABSENT, and fetching the appropriate data from an alternate location on demand. Client data access requests to the local storage system initiate a restoration of the data from a backing store as required. A demand generator can also be used to restore the data as a background process by walking through the sparse volume and restoring the data of absent blocks. A pump module is also disclosed to regulate the access of the demand generator. Once all the data has been restored, the volume contains all data locally, and is no longer a sparse volume.

Claims

1. A system, comprising:

a storage operating system executed by a processor on a first storage system, the storage operating system configured to detect a failed volume served by the first storage system;

means for detecting the failed volume served by the first storage system, the first storage system executing the storage operating system;

means for replacing the failed volume with a sparse volume served by the first storage system, wherein the sparse volume comprises at least one pointer of a buffer tree referencing data that is not stored locally on a volume served by the first storage system;

means for storing volume infrastructure metadata within the sparse volume, wherein the volume infrastructure metadata is copied from a second storage system;

means for marking the at least one pointer of the buffer tree within the sparse volume with a pointer value, wherein the pointer value identifies that the data referenced by the at least one pointer of the buffer tree is stored on a volume served by the second storage system;

means for waiting until the data referenced by the at least one pointer of the buffer tree within the sparse volume is requested by scanning the at least one pointer of the buffer tree; and

means for receiving a request for the data referenced by the at least one pointer of the buffer tree marked with the pointer value and copying, in response to receiving the request, the data referenced by the at least one pointer from the second storage system to the sparse volume.

2. The system of claim 1, further comprising means for initiating a backup operation of the sparse volume to a remote backup volume.

3. A method, comprising:

detecting a failed volume served by a first storage system, the first storage system executing a storage operating system;

replacing the failed volume with a sparse volume served by the first storage system, wherein the sparse volume comprises at least one pointer of a buffer tree referencing data that is not stored locally on a volume served by the first storage system;

storing volume infrastructure metadata within the sparse volume, wherein the volume infrastructure metadata is copied from a second storage system;

marking the at least one pointer of the buffer tree within the sparse volume with a pointer value, wherein the pointer value identifies that the data referenced by the at least one pointer of the buffer tree is stored on a volume served by the second storage system;

waiting until the data referenced by the at least one pointer of the buffer tree within the sparse volume is requested by scanning the at least one pointer of the buffer tree; and

receiving a request for the data referenced by the at least one pointer of the buffer tree marked with the pointer value and copying, in response to receiving the request, the data referenced by the at least one pointer from the second storage system to the sparse volume.

4. The method of claim 3, wherein the pointer value is an ABSENT value.

5. The method of claim 3, wherein the request is a client access request.

6. The method of claim 3, further comprising:

scanning, by a demand generator, the sparse volume; and

initiating, by the demand generator in response to locating the pointer value, the request as a remote fetch operation.

7. The method of claim 3, further comprising processing data access operations directed to the sparse volume.

8. The method of claim 7, wherein the data access operations comprise write operations.

9. The method of claim 3 further comprising initiating a back up operation of the sparse volume to a remote backup volume.

10. The method of claim 3, wherein the failed volume comprises a plurality of objects.

11. The method of claim 10, wherein the objects are luns, directories, qtrees, special data containers, or files.

12. A system for use with a first storage device, comprising:

a first storage system configured to execute a storage operating system, the first storage system comprising a first volume stored across the first storage device, wherein the first storage device is coupled to the first storage system;

a second storage system configured with a backup copy of the first volume, wherein the backup copy is stored on a second storage device coupled to the second storage system;

in response to the first volume failing, the first storage system configured to replace the failed volume with a sparse volume stored across the first storage system, wherein the sparse volume comprises at least one pointer of a buffer tree referencing data that is not stored locally on a volume served by the first storage system, wherein the sparse volume further comprises a plurality of pointer values to identify that the data referenced by the at least one pointer of the buffer tree is stored on the backup copy of the second storage system;

the first storage system further configured to wait until the data referenced by the at least one pointer of the buffer tree within the sparse volume is requested by scanning the at least one pointer of the buffer tree; and

the first storage system further configured to receive a request for the data referenced by the at least one pointer of the buffer tree marked with the pointer value and further configured to copy, in response to receiving the request, the data referenced by the at least one pointer of the buffer tree from the second storage system to the sparse volume.

13. The system of claim 12, wherein the request is a client request.

14. The system of claim 12, further comprising a demand generator configured to scan the sparse volume and further configured to initiate the request as a remote fetch operation for the data referenced by the at least one pointer in response to locating the pointer value.

15. The system of claim 14, wherein the first storage system is further configured with a pump module configured to provide flow control of the demand generator.

16. The system of claim 15, wherein the pump module is configured to implement a priority policy that grants precedence to a client issued request over a demand generator generated request.

17. The system of claim 12, wherein the failed volume comprises a plurality of objects.

18. The system of claim 17, wherein the objects are luns, directories, qtrees, special data containers, or files.

19. The system of claim 12, wherein the second storage system is further configured to perform backup operations while restoring to the first storage system.

20. A method, comprising:

configuring an aggregate from a plurality of storage devices attached to a first storage system, wherein the aggregate defines a global storage space of the first storage system;

providing a plurality of volumes stored within the aggregate;

detecting a failed volume on the first storage system;

activating a new volume on the first storage system, wherein the new volume is a sparse volume, wherein the sparse volume comprises at least one pointer of a buffer tree referencing data that is not stored on a volume served by the first storage system;

storing volume infrastructure metadata within the sparse volume, wherein the volume infrastructure metadata is copied from a second storage system;

populating the sparse volume with a pointer value of the at least one pointer of the buffer tree indicating that data referenced by the at least one pointer of the buffer tree is located on the second storage system;

waiting until the data referenced by the at least one pointer of the buffer tree within the sparse volume is requested by scanning the at least one pointer of the buffer tree; and

receiving a request for the data referenced by the at least one pointer of the buffer tree with the pointer value and copying, in response to receiving the request, the data referenced by the at least one pointer of the buffer tree from the second storage system to the sparse volume.

21. The method of claim 20, wherein the pointer value is an ABSENT value.

22. The method of claim 20, wherein the failed volume and the new volume are virtual volumes.

23. The method of claim 20, wherein the aggregate is a physical volume.

24. The method of claim 20, wherein the volume infrastructure metadata contains at least one of a current file system version, a total size of the volume, and content of a root file system directory.

25. The method of claim 20, wherein the request is a client request.

26. The method of claim 20, further comprising restoring the data from the second storage system upon receiving a request initiated by a demand generator.

27. The method of claim 26, further comprising:

scanning, by the demand generator the sparse volume to locate blocks with the pointer value.

28. The method of claim 20, further comprising processing data access operations directed to the sparse volume.

29. The method of claim 28, wherein the data access operations comprise write operations.

30. The method of claim 20, further comprising initiating a back up operation of the sparse volume to a remote backup volume.

31. A system, comprising:

an aggregate configured from a plurality of storage devices attached to a first storage system, the first storage system configured to execute a storage operating system, wherein the aggregate defines a global storage space of the first storage system;

a plurality of volumes stored within the aggregate;

in response to a first volume failing on the first storage system, the first storage system is configured to activate a new volume, wherein the new volume is a sparse volume and wherein the sparse volume comprises at least one pointer of a buffer tree referencing data that is not stored on a volume served by the first storage system;

the first storage system is further configured to store volume infrastructure metadata within the sparse volume, wherein the volume infrastructure metadata is copied from a second storage system;

the first storage system is further configured to populate the sparse volume with a pointer value of the at least one pointer of the buffer tree indicating that data referenced by the at least one pointer of the buffer tree is located on the second storage system;

the first storage system is further configured to wait until the data referenced by the at least one pointer of the buffer tree within the sparse volume is requested by scanning the at least one pointer of the buffer tree; and

the first storage system is further configured to receive a request for the data referenced by the at least one pointer of the buffer tree with the pointer value and is further configured to copy, in response to receiving the request, the data referenced by the at least one pointer of the buffer tree from the second storage system to the sparse volume.

32. The system of claim 31, wherein the failed volume and the sparse volume are virtual volumes.

33. The system of claim 31, wherein the aggregate is a physical volume.

34. The system of claim 31, wherein the volume infrastructure metadata contains at least one of a current file system version, a total size of the volume, and content of a root file system directory.

35. A computer readable storage medium containing executable program instructions executed by a processor, comprising:

program instructions that configure an aggregate from a plurality of storage devices attached to a first storage system, wherein the aggregate defines a global storage space of the first storage system;

program instructions that provide a plurality of volume stored within the aggregate;

program instructions that detect a failed volume on the first storage system;

program instructions that activate a new volume on the first storage system, wherein the new volume is a sparse volume, wherein the sparse volume comprises at least one pointer of a buffer tree referencing data that is not stored on a volume served by the first storage system;

program instructions that store volume infrastructure metadata within the sparse volume, wherein the volume infrastructure metadata is copied from a second storage system;

program instructions that populate the sparse volume with a pointer value of the at least one pointer of the buffer tree indicating that data referenced by the at least one pointer of the buffer tree is located on the second storage system;

program instructions that wait until the data referenced by the at least one pointer of the buffer tree within the sparse volume is requested by scanning the at least one pointer of the buffer tree; and

program instructions that receive a request for the data referenced by the at least one pointer of the buffer tree with the pointer value and copy, in response to receiving the request, the data referenced by the at least one pointer of the buffer tree from the second storage system to the sparse volume.

36. The computer readable storage medium of claim 35, wherein the failed volume and the sparse volume are virtual volumes.

37. The computer readable storage medium of claim 35, wherein the request is a client request.

38. The computer readable storage medium of claim 35, wherein the volume infrastructure metadata contains at least one of a current file system version, a total size of the volume, and content of a root file system directory.