Method and system for providing substantially constant-time execution of a copy operation

• US 8,484,164 B1
• Filed: 10/23/2009
• Issued: 07/09/2013
• Est. Priority Date: 10/23/2009
• Status: Active Grant

• Current Assignee: NetApp, Inc.
• Original Assignee: NetApp, Inc.
• Inventors: Pawar, Dnyaneshwar, Raj, Bipul, Sivakumar, Naresh, Yong, Albert
• Primary Examiner(s): Nguyen, Cindy
• Application Number:
  US12/605,263
• Time in Patent Office: 1,355 Days
• Field of Search: 707/637, 707/640, 707/649, 707/657
• US Class Current: 707/657
• CPC Class Codes: G06F 16/11 File system administration, e.g. details of archiving or snapshots file system backup G06F11/14
• CPC Class Codes: G06F 3/0611 in relation to response time
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

A system and method for providing a substantially constant-time copy operation for file system objects managed by a network storage server begins by generating a new file based on metadata in a source file. The system then generates a snapshot of the source file or the logical volume in which the source file resides. The system then copies each of the indirect blocks of the source file to a corresponding location in the destination file. During the copy process, a modified set of file system operations can be executed on the destination metadata container. In response to a read request directed to a target block in the destination file, the system determines if target block has been copied. If the block has not been copied, the system provides a data segment from the corresponding data block in the snapshot. Write requests are processed by generating a new data block and adding a reference to the new data block to the target location in the destination metadata container. During the copy process, the system checks each block reference before copying to determine if the block reference has been modified by a write operation. If the block was modified, the system skips copying that particular block reference. The system provides additional operations to execute truncate and append operations during the copy process.

Claims

1. A method comprising:

generating a snapshot of at least a portion of a data set managed by a storage server;

performing a copy operation in the storage server to generate a copy of the data set separate from the snapshot, wherein performing the copy operation comprises defining a set of block locations containing null references, and replacing the contents of individual block locations with references to individual data segments;

during the copy operation to generate the copy of the data set separate from the snapshot, receiving from a requester a first read request directed to the copy of the data set that the copy operation is to generate; and

providing data from the snapshot to the requester by the storage server, in response to the first read request.

2. The method of claim 1, wherein the data set comprises a plurality of references to data blocks and wherein performing the copy operation comprises copying at least a portion of the plurality of references to the copy of the data set.

3. The method of claim 1, wherein the snapshot is a read-only copy of the portion of the data set.

4. The method of claim 1, further comprising:

making a determination, in response to the first read request, that data targeted by the first read request has not yet been copied from the data set to the copy of the data set, wherein said providing data from the snapshot to the requester is done in response to said determination.

5. The method of claim 1, further comprising:

during the copy operation, receiving a second read request directed to the copy of the data set;

making a determination, in response to the second read request, that data targeted by the second read request has already been copied from the source version of the data set to the copy of the data set; and

providing data from the copy of the data set in response to the second read request and the determination that data targeted by the second read request has already been copied from the source version of the data set to the copy of the data set.

6. The method of claim 1, further comprising:

during the copy operation, receiving a write request directed to a target write location in the copy of the data set; and

modifying the target write location in the copy of the data set based on the write request;

wherein the copy operation includes skipping the copying of the portion of the data set corresponding to the target write location in response to receiving the write request.

7. The method of claim 1, wherein metadata in the data set includes a plurality of references to data blocks, the method further comprising:

determining a range of references to copy from the source metadata container to the destination metadata container; and

stopping the copy operation after the range of references has been copied.

8. The method of claim 7, further comprising:

during the copying, receiving a truncate request; and

reducing the size of the range of references to copy based on the truncate request.

9. The method of claim 7, further comprising:

during the copy operation, receiving an append request; and

based on the append request, appending a reference to an appended data segment to the copy of the data set beginning at a location in the copy of the data set determined based on the range of references to copy.

10. The method of claim 1, further comprising:

during the copy operation, receiving a write request directed to a target location in the copy of the data set;

determining that the target location is a hole;

generating a data segment based on the write request; and

storing a reference to the generated data segment in the target location.

11. A system comprising:

a memory;

a processor coupled to the memory to execute instructions stored in the memory;

a copy component configured to execute a copy operation based on the instructions stored in the memory to copy a plurality of blocks from a source metadata container to a destination metadata container, wherein the copy operation comprises defining a set of block locations containing null references, and replacing the contents of individual block locations with references to individual data segments; and

a write component configured to execute a modification operation based on the instructions stored in the memory to modify a target block of the destination metadata container during the copy operation in response to a write request to modify the target block, wherein the modification operation comprises generating a new data segment based on the write request; and

replacing an individual null reference at the block location with a reference to the new data segment,

wherein the copy component is configured to skip copying an individual block of the plurality of blocks that corresponds to the target block of the destination metadata container in response to the write request to modify the target block.

12. The system of claim 11, further comprising:

a snapshot generator component configured to generate a snapshot of a file system object corresponding to the source metadata container; and

a read component configured to respond to a read request directed to a block of the destination metadata container by providing a data segment from a corresponding block in the snapshot of the source metadata container.

13. The system of claim 11, wherein the copy component is configured to copy the blocks by copying a plurality of references to direct blocks.

14. The system of claim 11, wherein the write component is configured to generate the destination metadata container by generating a plurality of holes in the destination metadata container corresponding to the plurality of block of the source metadata container and to copy blocks by replacing an individual hole of the plurality of holes with a block copied from the source metadata container.

15. The system of claim 11, wherein the copy component is further configured to:

determine a first range of blocks to copy from the source metadata container to the destination metadata container based on the contents of the source metadata container;

during the copying, determine a second range of blocks to copy based on a received truncate request; and

stop the copying after copying the blocks in the second range.

16. The system of claim 11, wherein the copy component is further configured to:

determine a range of blocks to copy from the source metadata container to the destination metadata container based on the contents of the source metadata container; and

in response to a received append request, append a block to the destination metadata container at a location after the range of blocks.

17. A method comprising:

performing a copy operation in a storage server to generate a copy of a data set managed by the storage server;

wherein performing the copy operation comprises defining a set of block locations containing null references, and replacing the contents of individual block locations with references to individual data segments; and

during the copy operation, modifying a portion of the copy of the data set based on a write request to modify the portion of the copy of the data set;

wherein modifying the portion of the copy of the data set comprises generating a new data segment based on the write request; and

replacing an individual null reference at the block location with a reference to the new data segment;

wherein copying the data set includes;

for a portion of the data set, determining whether a corresponding location in the copy of the data set was modified by the write request to modify the portion of the copy of the data set; and

based on the determination, selectively skipping the copying of the portion of the data set in response to the write request to modify the portion of the copy of the data set.

18. The method of claim 17, further comprising:

generating a snapshot of the data set; and

during the copy operation, responding to a read request directed to a location in the copy of the data set by providing a data segment based on a corresponding location in the snapshot of the data set.

19. The method of claim 17, further comprising:

determining a range of data block references to copy from the data set to the copy of the data set; and

stopping the copying after the range of data block references has been copied.

20. The method of claim 19, further comprising:

during the copying, receiving a truncate request; and

reducing the range of data block references to copy based on the truncate request.

21. The method of claim 19, further comprising:

during the copying, receiving an append request; and

based on the append request, appending data block references to the copy of the data set beginning at a location in the copy of the data set determined based on the range of data block references to copy.

22. The method of claim 17, wherein determining whether the corresponding location was modified by the write request comprises:

determining if the corresponding location is a hole; and

in response to determining that the corresponding location is not a hole, determining that the corresponding location was modified.

23. A system comprising:

a memory;

a processor coupled to the memory to execute instructions stored in the memory;

a snapshot generator configured to execute a snapshot operation based on the instructions stored in the memory to generate a snapshot of a file system object having a source inode;

a copy component configured to execute a copy operation based on the instructions stored in the memory to copy a plurality of data block references separate from the snapshot from the source inode to a destination inode;

wherein the copy operation comprises defining a set of block locations containing null references, and replacing the contents of individual block locations with references to individual data segments; and

a read component configured to execute a read operation based on the instructions stored in the memory to respond to a read request directed to the plurality of data block references in the destination inode during the copy by providing a data segment from the snapshot corresponding to a target read location of the read request in the destination inode.

24. The system of claim 23, further comprising a write component configured to modify a target data block reference of the destination inode based on a received write request, wherein copying the data block references includes skipping the copying of the target data block reference.

25. The system of claim 23, wherein the snapshot is a read-only copy of the file system object and wherein the destination inode is writeable and can be modified independently of the source inode.

26. The system of claim 23, wherein the copy component is further configured to:

determine a first range of data block references to copy from the source inode to the destination inode based on the contents of the source inode;

during the copying, determine a second range of data block references to copy based on a received truncate request; and

stop the copying after copying the data block references in the second range.

27. The system of claim 23, wherein the copy component is further configured to:

determine a range of data block references to copy from the source metadata container to the destination metadata container; and

in response to a received append request, append a data block reference to the destination inode at a location after the range of data block references.

28. The system of claim 23, wherein the snapshot generator is further configured to lock the generated snapshot such that the snapshot can not be deleted during the copying and to unlock the generated snapshot after the copying is complete.

29. The system of claim 23, further comprising a write component configured to:

during the copying, receive a write request directed to a target location in the destination metadata container;

determine that the target location is a hole;

generate a data segment based on the write request; and

store a data block reference to the generated data segment in the target location.

30. The system of claim 23, wherein the copy component is configured to copy data block references by:

for each data block reference in the source inode, determining if the corresponding location in the destination inode is a hole; and

copying the data block reference if the corresponding location is a hole and skipping the data block reference otherwise.