DATA MIGRATION PRESERVING STORAGE EFFICIENCY

• US 20160088080A1
• Filed: 09/23/2014
• Published: 03/24/2016
• Est. Priority Date: 09/23/2014
• Status: Active Grant

• Current Assignee: NetApp, Inc.
• Original Assignee: NetApp, Inc.
• Inventors: Curley, Jonathan, Bare, Keith II
• Granted Patent: US 9,832,260 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 3/0613 in relation to throughput
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0647 Migration mechanisms
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 3/0685 Hybrid storage combining heterogeneous device types, e.g. hierarchical storage, hybrid arrays
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

Technology is disclosed for a data migration process for a storage server that preserves storage efficiency information. The storage server receives an indication of a group of two or more files and selects among the two or more files a reference file and one or more selected files. The storage server initiates a first migration operation to copy or move the reference file from a source storage server to a destination storage server. The storage server initiates one or more additional migration operations to copy or move the selected files from the source storage server to the destination storage server. At least one of the additional migration operations include a step of transmitting to the destination storage server data blocks of the selected files that are not shared between the reference file and the selected files, but avoid transmitting to the destination storage server the blocks shared with the reference file.

Claims

1. A method, comprising:

receiving an indication of a group of two or more files;

selecting among the two or more files a reference file and one or more selected files;

initiating a first migration operation to migrate the reference file from a source storage server to a destination storage server; and

initiating one or more additional migration operations to migrate each of the one or more selected files from the source storage server to the destination storage server, wherein at least one of the additional migration operations includes;

transmitting to the destination storage server data blocks of the selected file that are not shared between the reference file and the selected file; and

avoiding transmitting to the destination storage server the shared data blocks.

2. The method of claim 1, wherein at least one of the additional migration operations further includes:

transmitting to the destination storage server a data structure indicating data blocks that are shared between the reference file and the selected file.

3. The method of claim 2, wherein transmitting the data structure comprises:

transmitting to the destination storage server the data structure comprising a bit map including bits that correspond to data blocks of the selected file, the bits indicating whether the corresponding data blocks are shared between the selected file and the reference file.

4. The method of claim 2, further comprising:

generating the data structure by comparing block locations on the source storage server for data blocks of the reference file with block locations on the source storage server for data blocks of the selected file.

5. The method of claim 2, further comprising:

comparing a checksum, a cryptographic hash, or data content of a data block of the reference file with a checksum, a cryptographic hash, or data content of a data block of the selected file.

6. The method of claim 1, further comprising:

during at least one of the additional migration operations, transmitting to the destination storage server offset numbers within the reference file for the shared data blocks.

7. The method of claim 1, wherein the first migration operation is to copy or move the reference file from the source storage server to the destination storage server;

and at least one of the additional migration operations is to copy or move the selected file from the source storage server to the destination storage server.

8. The method of claim 1, further comprising:

receiving a user instruction determining the files within the group.

9. The method of claim 1, further comprising:

maintaining a metadata structure recording relationships between parent files and child files that derive from the parent files; and

determining the files within the group based on the metadata structure.

10. The method claim 1, further comprising:

determining the files within the group by comparing checksums, cryptographic hashes, or data contents of multiple files stored in the source storage server.

11. The method of claim 1, wherein the selecting comprises:

selecting among the two or more files reference files and selected files, wherein the reference files share common data blocks with at least one of the selected files.

12. A method, comprising:

receiving an indication of a group of two or more files including a reference file and one or more selected files;

performing a first migration operation to migrate the reference file from a source storage server to a destination storage server; and

performing one or more additional migration operations to migrate each of the one or more selected files from the source storage server to the destination storage server, wherein at least one of the additional migration operations includes;

receiving from the source storage server a data structure indicating shared data blocks that are shared between the reference file and the selected file;

receiving from the source storage server unshared data blocks of the selected file that are not shared between the reference file and the selected file; and

establishing the selected file in the destination storage server including the shared data blocks and unshared data blocks.

13. The method of claim 12, further comprising:

during at least one of the additional migration operations,

determining that at least one individual data block of a copied reference file has been modified at the destination storage server and is no longer identical to an initially copied state of the individual data block of the copied reference file;

determining that the individual data block is one of the shared data blocks; and

in response to the determination that the individual data block of the copied reference file has been modified and the determination that the individual data block is one of the shared data blocks, retrieving a copy of the individual data block from the source storage server and establishing the selected file in the destination storage server including the copy of the individual data block.

14. The method of claim 12, further comprising:

during at least one of the additional migration operations,

determining that an individual data block of the reference file has not been allocated in the destination storage server;

determining that the individual data block is one of the shared data blocks; and

delaying establishing the selected file until the individual data block of the reference file has been allocated in the destination storage server.

15. The method of claim 12, wherein the first migration operation causes an absent allocation operation of the reference file at the destination storage server.

16. The method of claim 12, wherein at least one of the additional migration operations causes an absent allocation operation of the selected file at the destination storage server.

17. The method of claim 12, wherein the receiving comprises:

receiving an indication that a group of two or more files includes a reference file and a selected file and that the reference file and the selected file share common data blocks stored at the source storage server.

18. A storage server, comprising:

a storage efficiency component configured to;

determine a group of two or more data objects;

select among the two or more data objects a reference data object and one or more selected data objects; and

generate data structures identifying shared data blocks that are shared between the reference data object and each of the selected data objects; and

a data migration component configured to;

migrate the reference data object by transmitting to a destination storage server data blocks of the reference data object; and

migrate the selected data objects by transmitting to the destination storage server the data structures and data blocks of the selected data objects that are not shared between the reference data object and the selected data objects.

19. The storage server of claim 18, wherein the data migration component is further configured to avoid transmitting to the destination storage server the shared data blocks when migrating the selected data objects.

20. The storage server of claim 18, further comprising:

a data object relationship component configured to;

monitor creation of data objects in the storage server;

maintain a metadata structure recording derivation relationships between parent data objects and child data objects that derive from the parent data objects; and

determine two or more data objects that have derivation relationships based on the metadata structure.