Method to perform parallel data migration in a clustered storage environment

• US 7,913,056 B2
• Filed: 01/28/2008
• Issued: 03/22/2011
• Est. Priority Date: 12/20/2004
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: Brown, Michael F., Madnani, Kiran P., DesRoches, David W.
• Primary Examiner(s): Elmore; Stephen C
• Assistant Examiner(s): Ayash; Marwan
• Application Number:
  US12/020,609
• Publication Number:
  US 20080147942A1
• Time in Patent Office: 1,149 Days
• Field of Search: None
• US Class Current: 711/203
• CPC Class Codes: G06F 11/1666 where the redundant component is memory or memory area
• CPC Class Codes: G06F 11/20 using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0617 in relation to availability
• CPC Class Codes: G06F 3/0647 Migration mechanisms
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

A clustered storage array consists of multiple nodes coupled to one or more storage systems. The nodes provide a LUN-device for access by a client. The LUN-device maps to a source logical unit corresponding to areas of storage on the one or more storage systems. A target logical unit corresponds to different areas of storage on the one or more storage systems. The source logical unit is migrated in parallel by the multiple nodes to the target logical unit. Data to be copied from the source logical unit to the target logical unit are grouped into data chunks. Two or more of the plurality of nodes concurrently attempt to acquire an exclusive lock for a set of data chunks. The node acquiring the exclusive lock migrates the set of data chunks from the source logical unit to the target logical unit, while the exclusive lock is used to prevent other nodes from migrating the set of data chunks.

Claims

1. A method of migrating data from a source logical unit to a target logical unit, the source and target logical units corresponding to different areas of storage on one or more storage systems, the method comprising the steps of:

providing a clustered storage array comprised of a plurality of nodes interconnected with each other and in communication with the one or more storage systems by a network;

running, by each node of the plurality of nodes, shared file system software to enable multiple clients to concurrently access shared data in the one or more storage systems through any of the nodes, and clustered system software to ensure coherency of the shared data;

providing by the nodes a LUN-device for shared access by the multiple clients, the LUN-device mapping to the source logical unit;

grouping data that are to be copied from the source logical unit to the target logical unit into data chunks;

concurrently attempting, by two or more of the plurality of nodes, to acquire an exclusive lock for a set of data chunks;

providing a bit-mask having one bit for each data chunk;

dividing the bit-mask into splices of multiple bits;

uniquely associating each splice with one set of data chunks;

wherein the step of concurrently attempting to acquire the exclusive lock for a set of data chunks includes the step of attempting to acquire an exclusive lock on the splice associated with said set of data chunks, and

migrating, by the node that acquires the exclusive lock, the set of data chunks from the source logical unit to the target logical unit, while the exclusive lock is used to prevent other nodes from migrating the set of data chunks.

2. The method of claim 1, wherein the step of migrating includes:

copying, for each bit in the splice, the data chunk corresponding to that bit to the target logical unit;

setting each bit in the splice corresponding to a copied data chunk; and

unlocking the splice when all data chunks corresponding to the bits in the splice have been copied.

3. The method of claim 1, further comprising the steps of:

storing multiple copies of the bit-mask, each bit-mask copy having an associated checksum; and

if any one node of the plurality of nodes fails, checking by one or more of the other node the checksums to identify a valid copy of the bit-mask.

4. The method of claim 3, further comprising the step of performing synchronous sequential updates to the copies of the bit-mask to ensure only one bit-mask copy is being updated at any given time.

5. The method of claim 1, further comprising the steps of:

performing an I/O access by one of the nodes from a client to the LUN-device during the step of migrating, the step of performing comprising the steps of;

if the I/O access is a read access, returning data from the source logical unit to the client;

if the I/O access is a write access including data to be written;

ascertaining the data chunk to which data is to be written;

locking the splice corresponding to the data chunk;

if all the bits in the splice are set, then writing the data to be written to the source logical unit and the target logical unit;

if less than all of the bits in the splice are set, then reading the data chunks in the splice from the source logical unit, combining the data to be written with the data chunks read, and writing the combined data to the target logical unit and source logical unit;

setting the bits in the bit-mask corresponding to the copied data chunks; and

unlocking the splice.

6. The method of claim 1, wherein the step of grouping the data includes sequentially ordering the splices;

and wherein the step of attempting to acquire an exclusive lock on the splice includes accessing each splice in sequential order to find the next splice that is currently unlocked.

7. The method of claim 1, further comprising the step of canceling the migrating at any time during the migrating by coordinating the cancel request amongst the plurality of nodes, preserving the source logical unit and its corresponding LUN-device, and deleting the target logical unit.

8. The method of claim 1, wherein the plurality of nodes comprises a clustered system, and wherein the source logical unit and target logical unit are logical entities utilizing the clustered system to access the corresponding areas of storage, and wherein the migrating of data chunks operates over the clustered system.

9. The method of claim 1, wherein the set of data chunks is a first set and the node that acquires the exclusive lock for the first set is a first node, and further comprising the steps of:

acquiring by a second node of the plurality of nodes an exclusive lock for a second set of data chunks; and

migrating, by the second node, the second set of data chunks from the source logical unit to the target logical unit in parallel to the migrating of the first set of data chunks by the first node.

10. A clustered system comprising:

at least one hardware storage system including a source logical unit and a target logical unit, the source and target logical units corresponding to different areas of storage on the one or more storage systems; and

a clustered storage array comprised of a plurality of nodes interconnected with each other and in communication with the one or more storage systems by a network, each node running shared file system software to enable multiple clients to concurrently access shared data in the one or more storage systems through any of the nodes and clustered system software to ensure coherency of the shared data, the nodes providing, for shared access by the multiple clients, a LUN-device mapped to the source logical unit, each node including logic for grouping data that are to be copied from the source logical unit to the target logical unit into data chunks and logic for attempting to acquire an exclusive lock for a set of said data chunks, the system further comprising a bit-mask having one bit for each data chunk;

logic for dividing the bit-mask into splices of multiple bits;

logic for uniquely associating each splice with one set of data chunks; and

wherein the logic for attempting to acquire the exclusive lock for a set of data chunks includes logic for attempting to acquire an exclusive lock on the splice associated with said set of data chunks, and

wherein two or more of the plurality of nodes concurrently attempt to acquire an exclusive lock for the set of data chunks, with the node that acquires the exclusive lock migrating the set of data chunks from the source logical unit to the target logical unit, while the exclusive lock is used to prevent other nodes from migrating the set of data chunks.

11. The clustered system of claim 10, wherein the node that migrates the set of data chunks includes:

logic for copying, for each bit in the splice, the data chunk corresponding to that bit to the target logical unit;

logic for setting each bit in the splice corresponding to a copied data chunk; and

logic for unlocking the splice when all data chunks corresponding to the bits in the splice have been copied.

12. The clustered system of claim 10, further comprising:

memory storing multiple copies of the bit-mask, each bit-mask copy having an associated checksum; and

wherein each node includes logic, operational if any one node of the plurality of nodes fails, for checking the checksums of the bit-mask copies to identify a valid copy of the bit-mask.

13. The clustered system of claim 12, further comprising logic for performing synchronous sequential updates to the copies of the bit-mask to ensure only one bit-mask copy is being updated at any given time.

14. The clustered system of claim 10, wherein each node further comprises logic for performing an I/O access from a client to the LUN-device during the migrating of the set of data chunks, the logic for performing comprising:

logic for returning data from the source logical unit to the client if the I/O access is a read access; and

if the I/O access is a write access including data to be written;

logic for ascertaining the data chunk to which data is to be written, logic for locking the splice corresponding to the data chunk, logic for writing the data to be written to the source logical unit and the target logical unit if all the bits in the splice are set, logic for reading the data chunks in the splice from the source logical unit, combining the data to be written with the data chunks read, and writing the combined data to the target logical unit and source logical unit if less than all of the bits in the splice are set;

logic for setting the bits in the bit-mask corresponding to the copied data chunks; and

logic for unlocking the splice.

15. The clustered system of claim 10, wherein the logic for grouping the data includes logic for sequentially ordering the splices;

and wherein the logic for attempting to acquire an exclusive lock on the splice includes logic for accessing splices in sequential order to find the next splice that is currently unlocked.

16. The clustered system of claim 10, further comprising the logic for canceling the migrating at any time during the migrating by coordinating the cancel request amongst the plurality of nodes, preserving the source logical unit and its corresponding LUN-device, and deleting the target logical unit.

17. The clustered system of claim 10, wherein the source logical unit and target logical unit are logical entities utilizing the clustered system to access the corresponding areas of storage.

18. The clustered system of claim 10, wherein the set of data chunks is a first set and the node that acquires the exclusive lock for the first set is a first node, and wherein a second node of the plurality of nodes includes logic for acquiring an exclusive lock for a second set of data chunks;

and further comprising logic for migrating, by the second node, the second set of data chunks from the source logical unit to the target logical unit in parallel to the migrating of the first set of data chunks by the first node.