Method and apparatus for managing migration of data in a clustered computer system environment

• US 7,080,221 B1
• Filed: 04/23/2003
• Issued: 07/18/2006
• Est. Priority Date: 04/23/2003
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: Buckley, Donagh, Todd, Stephen J., Blumenau, Steven M., Bell, John Philip II, Cakeljic, Zoran, Fox, Conor
• Primary Examiner(s): CHACE, CHRISTIAN
• Application Number:
  US10/421,401
• Time in Patent Office: 1,182 Days
• Field of Search: 711/161, 711/162, 711/165, 707/201, 707/202, 707/203, 707/204, 714/6
• US Class Current: 711/161
• CPC Class Codes: G06F 11/1464 for networked environments
• CPC Class Codes: G06F 11/1466 to make the backup process non-disruptive
• CPC Class Codes: G06F 11/1471 involving logging of persistent data for recovery
• CPC Class Codes: G06F 11/2071 using a plurality of controllers
• CPC Class Codes: G06F 3/0605 by facilitating the interaction with a user or administrator
• CPC Class Codes: G06F 3/0647 Migration mechanisms
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: Y10S 707/99952 Coherency, e.g. same view to multiple users
• CPC Class Codes: Y10S 707/99953 Recoverability

Abstract

Methods and apparatus for performing a data migration in a clustered computer system. In one aspect, the availability of the data being migrated is maintained during the migration. In another aspect, a data set is migrated by a transfer utility in one of the hosts in the cluster, and in response to a failure of the host, the migration is resumed without restarting from the beginning point of the data set. In a further embodiment, the hosts in the cluster have agents associated therewith to manage the data migration, and prior to taking at least one action associated with the data migration, the agents coordinate to ensure that the hosts in the cluster are prepared to have the at least one action taken.

Claims

1. A method for performing a data migration in a clustered computer system including at least two hosts and at least one storage system that stores data shared by the hosts, the at least one storage system providing a plurality of storage volumes to the hosts to store data shared by the at least two hosts, the plurality of storage volumes including at least one first storage volume and at least one second storage volume, the method comprising acts of:

(A) migrating data from the at least one first storage volume to the at least one second storage volume under the control of a transfer utility that is external to the at least one storage system, the at least one first storage volume being allocated to at least one application program installed on each of the at least two hosts in the clustered computer system to store data written by the at least one application program wherein the transfer utility reads the data from the at least one storage system that provides the first storage volume to at least one intermediate component and writes the data from the at least one intermediate component to the at least one storage system that provides the second storage volume; and

(B) maintaining the at least one first storage volume available to the at least one application program so that write operations from the at least one application program can be performed to the at least one first storage volume during the act of migrating.

2. The method of claim 1, wherein the at least one intermediate component comprises at least one of the at least two hosts so that the act (A) comprises acts of reading the data from the at least one first storage volume to the at least one of the at least two hosts, and writing the data from the at least one of the at least two hosts to the at least one second storage volume.

3. The method of claim 1, wherein the act (A) comprises, at at least one point in time during the migration, an act of storing, in a persistent manner, state information indicating a portion of the at least one first storage volume successfully copied to the at least one second storage volume at the at least one point in time.

4. The method of claim 1, further comprising acts of:

(C) after the act of migrating is complete, intercepting an access request from at least one instance of the at least one application program to the at least one first storage volume; and

(D) redirecting the access request to the at least one second storage volume, so that the access request can be serviced by the at least one second storage volume.

5. The method of claim 4, wherein the acts of intercepting and redirecting are performed within at least one of the at least two hosts.

6. The method of claim 1, wherein the act (A) includes migrating a data set, and wherein the method further comprises an act (C) of resuming, after an interruption, the migrating act without restarting from the beginning of the data set.

7. The method of claim 6, wherein the act (A) comprises, at at least one point in time during the migration, an act of storing, in a persistent manner, state information indicating a portion of the at least one first storage volume successfully copied to the at least one second storage volume at the at least one point in time, and wherein the act (C) of resuming further comprises resuming the act of migrating after the portion of the data identified by the state information as successfully copied.

8. The method of claim 6, wherein the at least one intermediate component comprises a first host of the at least two hosts, and wherein the act (A) comprises acts, performed by the first host, of reading the data from the at least one first storage volume to the first host, and writing the data from the first host to the at least one second storage volume;

wherein the interruption results from a failure of the first host; and

wherein the act (C) comprises an act of resuming the migration by having a second host of the at least two hosts read the data from the at least one first storage volume to the second host, and write the data from the second host to the at least one second storage volume.

9. The method of claim 1, wherein the clustered computer system is an active/passive clustered environment.

10. The method of claim 1, wherein the clustered computer system is an active/active clustered environment.

11. The method of claim 1, wherein the transfer utility is not implemented by any of the at least two hosts.

12. The method of claim 11, wherein the at least one storage system is coupled to the at least two hosts via at least one interconnection device, wherein the at least one storage system and the at least one interconnection device together form a storage area network (SAN), wherein the transfer utility is implemented by the SAN, and wherein the at least one intermediate component comprises the at least one interconnection device.

13. The method of claim 1, wherein each of the at least two hosts has an associated agent to manage the migration, and wherein the act (A) comprises an act of coordinating among the agents to ensure that each is prepared to begin the migration before executing the act (A).

14. The method of claim 4, wherein each of the at least two hosts has an associated agent to manage the migration, and wherein the method comprises an act (E) of, prior to performing acts (C) and (D), coordinating among the agents to ensure that each is prepared to have an access request issued therefrom redirected to the at least one second storage volume.

15. The method of claim 14, wherein the act (D) is performed by each of the at least two hosts, and wherein the act (E) comprises an act of coordinating among the agents to ensure that each is prepared to redirect an access request issued therefrom to the at least one second storage volume prior to performing the acts (C) and (D).

69. The method of claim 1, wherein the at least one storage system is coupled to the at least two hosts via at least one interconnection device, and wherein the at least one intermediate component comprises the at least one interconnection device.

70. The method of claim 69, wherein the at least one interconnection device comprises a plurality of interconnection devices, and wherein the at least one intermediate component comprises a subset of the plurality of interconnection devices.

16. At least one computer readable medium encoded with instructions for execution on a clustered computer system including at least two hosts and at least one storage system that stores data shared by the hosts, the at least one storage system providing a plurality of storage volumes to the hosts to store data shared by the at least two hosts, the plurality of storage volumes including at least one first storage volume and at least one second storage volume, the instructions, when executed, perform a method of performing a data migration, the method comprising acts of:

(A) migrating data from the at least one first storage volume to the at least one second storage volume under the control of a transfer utility that is external to the at least one storage system, the at least one first storage volume being allocated to at least one application program installed on each of the at least two hosts in the clustered computer system to store data written by the at least one application program, wherein the transfer utility reads the data from the at least one storage system that provides the first storage volume to at least one intermediate component and writes the data from the at least one intermediate component to the at least one storage system that provides the second storage volume; and

(B) maintaining the at least one first storage volume available to the at least one application program so that write operations from the at least one application program can be performed to the at least one first storage volume during the act of migrating.

17. The at least one computer readable medium of claim 16, wherein the at least one intermediate component comprises at least one of the at least two hosts so that the act (A) comprises acts of reading the data from the at least one first storage volume to the at least one of the at least two hosts, and writing the data from the at least one of the at least two hosts to the at least one second storage volume.

18. The at least one computer readable medium of claim 16, wherein the act (A) comprises, at at least one point in time during the migration, an act of storing, in a persistent manner, state information indicating a portion of the at least one first storage volume successfully copied to the at least one second storage volume at the at least one point in time.

19. The at least one computer readable medium of claim 16, wherein the method further comprises acts of:

(C) after the act of migrating is complete, intercepting an access request from at least one instance of the at least one application program to the at least one first storage volume; and

(D) redirecting the access request to the at least one second storage volume, so that the access request can be serviced by the at least one second storage volume.

20. The at least one computer readable medium of claim 19, wherein the acts of intercepting and redirecting are performed within at least one of the at least two hosts.

21. The at least one computer readable medium of claim 16, wherein the act (A) includes migrating a data set, and wherein the method further comprises an act (C) of resuming, after an interruption, the migrating act without restarting from the beginning of the data set.

22. The at least one computer readable medium of claim 21, wherein the act (A) comprises, at at least one point in time during the migration, an act of storing, in a persistent manner, state information indicating a portion of the at least one first storage volume successfully copied to the at least one second storage volume at the at least one point in time, and wherein the act (C) of resuming further comprises resuming the act of migrating after the portion of the data identified by the state information as successfully copied.

23. The at least one computer readable medium of claim 21, wherein the at least one intermediate component comprises a first host of the at least two hosts, and wherein the act (A) comprises acts, performed by the first host, of reading the data from the at least one first storage volume to the first host, and writing the data from the first host to the at least one second storage volume;

wherein the interruption results from a failure of the first host; and

wherein the act (C) comprises an act of resuming the migration by having a second host of the at least two hosts read the data from the at least one first storage volume to the second host, and write the data from the second host to the at least one second storage volume.

24. The at least one computer readable medium of claim 16, wherein the clustered computer system is an active/passive clustered environment.

25. The at least one computer readable medium of claim 16, wherein the clustered computer system is an active/active clustered environment.

26. The at least one computer readable medium of claim 16, wherein the transfer utility is not implemented by any of the at least two hosts.

27. The at least one computer readable medium of claim 26, wherein the at least one storage system is coupled to the at least two hosts via at least one interconnection device, wherein the at least one storage system and the at least one interconnection device together form a storage area network (SAN), wherein the transfer utility is implemented by the SAN, and wherein the at least one intermediate component comprises the at least one interconnection device.

28. The at least one computer readable medium of claim 16 wherein each of the at least two hosts has an associated agent to manage the migration, and wherein the act (A) comprises an act of coordinating among the agents to ensure that each is prepared to begin the migration before executing the act (A).

29. The at least one computer readable medium of claim 19, wherein each of the at least two hosts has an associated agent to manage the migration, and wherein the method comprises an act (E) of, prior to performing acts (C) and (D), coordinating among the agents to ensure that each is prepared to have an access request issued therefrom redirected to the at least one second storage volume.

30. The at least one computer readable medium of claim 29, wherein the act (D) is performed by each of the at least two hosts, and wherein the act (E) comprises an act of coordinating among the agents to ensure that each is prepared to redirect an access request issued therefrom to the at least one second storage volume prior to performing the acts (C) and (D).

71. The at least one computer readable medium of claim 16, wherein the at least one storage system is coupled to the at least two hosts via at least one interconnection device, and wherein the at least one intermediate component comprises the at least one interconnection device.

31. A first host computer for managing a migration of data in a clustered computer system including a plurality of host computers and at least one storage system that stores the data, the plurality of host computers including the first host computer, wherein the data is shared by instances of at least one application executing on the plurality of host computers, the at least one storage system providing a plurality of storage volumes to the hosts to store the data shared by the plurality of host computers, the plurality of storage volumes including at least one first storage volume and at least one second storage volume, the first host computer comprising:

a processor to execute at least one instance of the at least one application program; and

a controller adapted to take over management of the migration of the data set from the at least one first storage volume to the at least one second storage volume in the event that while maintaining the at least one first storage volume available to the at least one application program so that write operations from the at least one application program can be performed to the at least one first storage volume during the act of migrating.

32. The host computer of claim 31, further comprising means for reading the data from the at least one first storage volume to the first host computer, and writing the data from the first host computer to the at least one second storage volume.

33. The first host computer of claim 31, further comprising means for storing, in a persistent manner and at at least one point in time during the migration, state information indicating a portion of the at least one first storage volume successfully copied to the at least one second storage volume at the at least one point in time.

34. The first host computer of claim 31, further comprising:

means for intercepting, after the migration is complete, an access request from the at least one application program to the at least one first storage volume; and

means for redirecting the access request to the at least one second storage volume, so that the access request can be serviced by the at least one second storage volume.

35. The first host computer of claim 31, wherein the data being migrated is a data set, and wherein the at least one controller is capable of, after an interruption, resuming the migration without restarting from the beginning of the data set.

36. The first host computer of claim 35, wherein the at least one controller, at at least one point in time during the migration, stores, in a persistent manner, state information indicating a portion of the at least one first storage volume successfully copied to the at least one second storage volume at the at least one point in time, and after the interruption, resumes the migration after the portion of the data identified by the state information as successfully copied.

37. The first host computer of claim 31, wherein the clustered computer system is an active/passive clustered environment.

38. The first host computer of claim 31, wherein the clustered computer system is an active/active clustered environment.

39. The first host computer of claim 31, wherein the at least one controller instructs a transfer utility to copy the data from the at least one first storage volume to the at least one second storage volume, wherein the transfer utility is not implemented by any of the plurality of host computers.

40. The first host computer of claim 39, wherein the at least one storage system is coupled to the plurality of host computers via at least one interconnection device, wherein the at least one storage system and the at least one interconnection device together form a storage area network (SAN), and wherein the transfer utility is implemented by the SAN.

41. The first host computer of claim 31, wherein the at least one controller coordinates with each of the other of the plurality of host computers to ensure that each is prepared to begin the migration before beginning the migration.

42. The first host computer of claim 34, wherein the at least one controller coordinates with each of the other of the plurality of host computers to ensure that each is prepared to have access requests issued therefrom redirected to the at least one second storage volume before switching to the second storage volume by instructing the means for intercepting to intercept access requests to the at least one first storage volume and the means for redirecting to redirect access requests to the at least one firs storage volume.

43. The first host computer of claim 37, wherein the first host computer is an active host in the active/passive clustered environment.

44. A method for performing a data migration in a clustered computer system including first and second host computers and at least one storage system that stores data shared by the first and second host computers, the at least one storage system providing a plurality of storage volumes to the first and second host computers to store data shared by the first and second host computers, the plurality of storage volumes including at least one first storage volume and at least one second storage volume, the method comprising acts of:

(A) migrating a data set from the at least one first storage volume to the at least one second storage volume using a transfer utility implemented by the first host computer, the at least one first storage volume being allocated to at least one application program installed on each of the first and second host computers to store data written by the at least one application program, wherein the act of migrating starts at a beginning point of the data set and progresses to an end point of the data set; and

(B) in response to a failure of the first host computer, resuming the migration of the data set without restarting from the beginning point of the data set.

45. The method of claim 44, wherein the act (B) comprises an act of resuming the migration using a transfer utility implemented by the second host computer.

46. The method of claim 44, wherein the act (A) comprises acts of reading portions of the data set from the at least one first storage volume to the first host computer and writing the portions of the data set from the first host computer to the at least one second storage volume.

47. The method of claim 44, wherein the act (A) comprises, at at least one point in time during the migration, an act of storing, in a persistent manner, state information indicating portions of the data set successfully copied to the at least one second storage volume at the at least one point in time.

48. The method of claim 44, wherein the act (A) comprises, at at least one point in time during the migration, an act of storing, in a persistent manner, state information indicating portions of the data set successfully copied to the at least one second storage volume at the at least one point in time, and wherein the act (B) of resuming further comprises resuming the migration after the portions of the data set identified by the state information as successfully copied.

49. The method of claim 44, wherein the clustered computer system is an active/passive clustered environment.

50. The method of claim 49, wherein before the failure of the first host computer, the first host computer is an active host computer and the second host computer is a passive host computer.

51. The method of claim 44, wherein the clustered computer system is an active/active clustered environment.

52. The method according to claim 44, wherein the act (B) includes an act of not re-copying at least some of the data set copied before the failure of the first host computer.

53. At least one computer readable medium encoded with instructions for execution on a clustered computer system including first and second host computers and at least one storage system that stores data shared by the first and second host computers, the at least one storage system providing a plurality of storage volumes to the first and second host computers to store data shared by the first and second host computers, the plurality of storage volumes including at least one first storage volume and at least one second storage volume, the instructions, when executed, implement a method of performing a data migration, the method comprising acts of:

(A) migrating a data set from the at least one first storage volume to the at least one second storage volume using a transfer utility implemented by the first host computer, the at least one first storage volume being allocated to at least one application program installed on each of the first and second host computers to store data written by the at least one application program, wherein the act of migrating starts at a beginning point of the data set and progresses to an end point of the data set; and

(B) in response to a failure of the first host computer, resuming the migration of the data set without restarting from the beginning point of the data set.

54. The at least one computer readable medium of claim 53, wherein the act (B) comprises an act of resuming the migration using a transfer utility implemented by the second host computer.

55. The at least one computer readable medium of claim 53, wherein the act (A) comprises acts of reading portions of the data set from the at least one first storage volume to the first host computer and writing the portions of the data set from the first host computer to the at least one second storage volume.

56. The at least one computer readable medium of claim 53, wherein the act (A) comprises, at at least one point in time during the migration, an act of storing, in a persistent manner, state information indicating portions of the data set successfully copied to the at least one second storage volume at the at least one point in time.

57. The at least one computer readable medium of claim 53, wherein the act (A) comprises, at at least one point in time during the migration, an act of storing, in a persistent manner, state information indicating portions of the data set successfully copied to the at least one second storage volume at the at least one point in time, and wherein the act (B) of resuming further comprises resuming the migration after the portions of the data set identified by the state information as successfully copied.

58. The at least one computer readable medium of claim 53, wherein the clustered computer system is an active/passive clustered environment.

59. The at least one computer readable medium of claim 58, wherein before the failure of the first host computer, the first host computer is an active host computer and the second host computer is a passive host computer.

60. The at least one computer readable medium of claim 53, wherein the clustered computer system is an active/active clustered environment.

61. The method according to claim 53, wherein the act (B) includes an act of not re-copying at least some of the data set copied before the failure of the first host computer.

62. A clustered computer system comprising:

a first host computer comprising a first transfer utility;

a second host computer comprising a second transfer utility; and

at least one storage system that is coupled to the first and second host computers and stores data shared by the first and second host computers, the at least one storage system providing a plurality of storage volumes to the first and second host computers to store the data shared by the first and second host computers, the plurality of storage volumes including at least one first storage volume and at least one second storage volume;

wherein the first host computer comprises a first agent to begin a migration of a data set from the at least one first storage volume to the at least one second storage volume using the first transfer utility, wherein the first agent begins the migration at a beginning point of the data set and proceeds toward an end point of the data set; and

wherein the second host computer comprises a second agent that, in response to a failure of the first host computer, resumes the migration of the data set without restarting from the beginning point of the data set.

63. The clustered computer system of claim 62, wherein the second agent resumes the migration using the second transfer utility.

64. The clustered computer system of claim 62, wherein the first agent, at at least one point in time during the migration, stores, in a persistent manner, state information indicating portions of the data set successfully copied to the at least one second storage volume at the at least one point in time, and wherein the second agent resumes the migration after the portions of the data set identified by the state information as successfully copied.

65. The clustered computer system of claim 62, wherein the clustered computer system is an active/passive clustered environment.

66. The clustered computer system of claim 65, wherein before the failure of the first host computer, the first host computer is an active host computer and the second host computer is a passive host computer.

67. The clustered computer system of claim 62, wherein the clustered computer system is an active/active clustered environment.

68. The clustered computer system of claim 62, wherein the second agent does not re-copy at least some of the data set copied before the failure of the first host computer.