Method and apparatus for migrating data

• US 7,707,151 B1
• Filed: 01/29/2003
• Issued: 04/27/2010
• Est. Priority Date: 08/02/2002
• Status: Expired due to Term

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: Todd, Stephen J., Blumenau, Steven M.
• Primary Examiner(s): Thai; Tuan V
• Assistant Examiner(s): Rojas; Midys
• Application Number:
  US10/353,322
• Time in Patent Office: 2,645 Days
• Field of Search: 711/165
• US Class Current: 711/165
• CPC Class Codes: G06F 11/1438 Restarting or rejuvenating
• CPC Class Codes: G06F 3/0617 in relation to availability
• CPC Class Codes: G06F 3/0647 Migration mechanisms
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

One aspect is directed to a method for performing data migration from a first volume to a second volume while allowing a write operation to be performed on the first volume during the act of migrating. Another aspect is a method and apparatus that stores, in a persistent manner, state information indicating a portion of the first volume successfully copied to the second volume. Another aspect is a method and apparatus for migrating data from a first volume to a second volume, and resuming, after an interruption of the migration, copying data from the first volume to the second volume without starting from the beginning of the data. Another aspect is a method and apparatus for migrating to data from a first to a second volume, receiving an access request directed to the first volume from an application that stores data on the first volume, and redirecting the access request to the second volume without having to reconfigure the application that accesses data on the first volume.

Claims

1. A method for performing data migration in a computer system including at least one host and at least one storage system that stores data written by the host, the at least one storage system providing a plurality of storage volumes to the host to store data written by the host, the plurality of storage volumes including at least one first storage volume and at least one second storage volume, the method comprising acts of:

(A) migrating data from the at least one first storage volume to the at least one second storage volume, the at least one first storage volume being allocated to at least one application program in the computer system to store data;

(B) maintaining the at least one first storage volume available to the at least one application program so that write operations from the at least one application program can be performed to the at least one first storage volume during the act of migrating,

wherein the act (A) comprises, at least one point in time during the migration, an act of storing, in a non-volatile storage medium, state information indicating a portion of a data set of the at least one first storage volume successfully copied to the at least one second storage volume at the at least one point in time, and

wherein the act (A) comprises acts of reading the data from the at least one first storage volume to the at least one host, and writing the data from the at least one host to the at least one second storage volume; and

(C) resuming, after an interruption that results in a loss of at least some state information stored in at least one volatile storage medium in the computer system, the act of migrating without restarting from the beginning of the data set, the act of resuming being performed using the state information stored in the non-volatile storage medium.

2. The method according to claim 1, wherein the act of storing further comprises an act of storing, on at least one of the at least one first storage volume and the at least one second storage volume, a checkpoint that indicates the portion of the data that was successfully copied to the at least one second storage volume.

3. The method according to claim 1, wherein the act of storing further comprises an act of storing, on at least one of the at least one first storage volume and the at least one second storage volume, an indicator that indicates the portion of the data that was successfully copied to the at least one second storage volume.

4. The method according to claim 1, further comprising an act of processing a write operation, the act of processing comprising acts of:

receiving at least one write operation from the at least one application program; and

writing data associated with the at least one write operation to both the at least one first storage volume and the at least one second storage volume.

5. The method according to claim 4, wherein the act of processing a write request further comprises an act of sending an acknowledgement to the at least one application only after data associated with the at least one write operation has been written to both the at least one first storage volume and the at least one second storage volume.

6. The method according to claim 1, wherein the acts of reading data from the at least one first storage volume to the at least one host and writing the data from the at least one host to the at least one second storage volume are performed by a driver in an input/output (I/O) stack of the at least one host.

7. The method according to claim 1, further comprising acts of:

after the act of migrating is complete, intercepting an access request from the at least one application program to the at least one first storage volume; and

redirecting the access request to the at least one second storage volume, so that the access request can be serviced by the at least one second storage volume.

8. The method according to claim 7, wherein the acts of intercepting and redirecting are performed within the at least one host.

9. The method according to claim 8, wherein the at least one host includes an input/output (I/O) stack, and wherein the acts of intercepting and redirecting are performed in the I/O stack.

10. The method according to claim 1, wherein the act of resuming further comprises resuming the act of migrating after the portion of the data identified by the state information as successfully copied.

11. The method of claim 1, wherein the act (A) further comprises (A) migrating data from the at least one first storage volume to the at least one second storage volume under the control of at least one migration layer in the computer system, wherein the at least one migration layer is separate from the at least one application program;

and

wherein the act of storing the state information further comprises an act of storing, on at least one of the at least one first storage volume and the at least one second storage volume, a checkpoint that indicates the portion of the data that was successfully copied to the at least one second storage volume; and

further comprising an act of, in response to a read operation from the at least one application program that reads data from the at least one first storage volume, writing to the at least one second storage volume the data read by the read operation.

12. The method according to claim 11, wherein the act of writing to the at least one second storage volume the data read by the read operation is performed if the data read by the read operation is located before the checkpoint.

13. The method according to claim 11, wherein the at least one migration layer is disposed on the at least one host.

14. The method according to claim 13, wherein the acts of reading data from the at least one first storage volume to the at least one host and writing the data from the at least one host to the at least one second storage volume are performed by a driver in an input/output (I/O) stack of the at least one host.

15. The method according to claim 11, wherein the act of resuming further comprises resuming the act of migrating after the portion of the data identified by the state information as successfully copied.

16. A method for performing data migration in a computer system including at least one host, at least one storage system that stores data written by the host, and at least one controller which is external to the at least one storage system and which is operable to facilitate the data migration, the at least one storage system providing a plurality of storage volumes to the host to store data written by the host, the plurality of storage volumes including at least one first storage volume and at least one second storage volume, the method comprising acts of:

(A) migrating, via the at least one controller which is external to the at least one storage system, a data set from the at least one first storage volume to the at least one second storage volume, the at least one first storage volume being allocated to at least one application program in the computer system to store data wherein the act of migrating starts at a beginning of the data set and progresses to an end of the data set,

wherein the act (A) comprises, at least one point in time during the migration, an act of storing, in a non-volatile storage medium, state information indicating a portion of the data set of the at least one first storage volume successfully copied to the at least one second storage volume at the at least one point in time; and

(B) resuming, after an interruption that results in a loss of at least some state information stored in at least one volatile storage medium in the computer system, the act of migrating without restarting from the beginning of the data set, the act of resuming being performed using the state information stored in the non-volatile storage medium,

wherein the act (A) comprises acts of reading the data from the at least one first storage volume to the at least one host, and writing the data from the at least one host to the at least one second storage volume.

17. The method according to claim 16, further comprising an act of maintaining the at least one first storage volume available to the at least one application program so that write operations from the at least one application program can be performed to the at least one first storage volume during the act of migrating.

18. The method according to claim 16, wherein the interruption results in a loss of volatile state information for the at least one host.

19. The method according to claim 18, wherein the acts of reading data from the at least one first storage volume to the at least one host and writing the data from the at least one host to the at least one second storage volume are performed by a driver in an input/output (I/O) stack of the at least one host.

20. The method according to claim 16, wherein the act of storing further comprises an act of storing, on at least one of the at least one first storage volume and the at least one second storage volume, a checkpoint that indicates the portion of the data that was successfully copied to the at least one second storage volume.

21. The method according to claim 16, wherein the act of storing further comprises an act of storing, on at least one of the at least one first storage volume and the at least one second storage volume, an indicator that indicates the portion of the data that was successfully copied to the at least one second storage volume.

22. The method according to claim 16, further comprising acts of performing a read operation on the at least one first storage volume and performing a write operation on the at least one second storage volume with data read by the act of reading.

23. The method according to claim 16, further comprising an act of processing a write operation, the act of processing comprising acts of:

receiving at least one write operation from the at least one application program; and

writing data associated with the at least one write operation to the at least one first storage volume and the at least one second storage volume.

24. The method according to claim 23, wherein the act of processing a write operation further comprises an act of sending an acknowledgement to the at least one application only after writing data associated with the at least one write operation has been written to both the at least one first storage volume and at least one second storage volume.

25. The method according to claim 16, further comprising acts of:

after the act of migrating is complete, intercepting an access request from the at least one application program to the at least one first storage volume; and

redirecting the access request to the at least one second storage volume, so that the access request can be serviced by the at least one second storage volume.

26. The method according to claim 25, wherein the acts of intercepting and redirecting are performed within the at least one host.

27. The method according to claim 26, wherein the at least one host includes an input/output (I/O) stack, and wherein the acts of intercepting and redirecting are performed in the I/O stack.

28. The method according to claim 16, wherein the act of resuming further comprises resuming the act of migrating after the portion of the data identified by the state information as successfully copied.

29. The method according to claim 16, wherein act (B) includes not re-copying at least some of the data set copied before the interruption.

30. The method according to claim 16, wherein the at least one controller is disposed on the at least one host.

31. The method according to claim 30, wherein the at least one host comprises a server or a client.

32. The method according to claim 16, wherein the at least one controller is disposed in a logical volume manager.

33. The method according to claim 16, wherein the at least one controller is disposed in a file system.

34. A computer-readable medium encoded with instructions for execution on a computer system, the instructions when executed, perform a method comprising acts of:

(A) migrating data from at least one first storage volume to at least one second storage volume, the at least one first storage volume being allocated to at least one application program in the computer system to store data;

(B) maintaining the at least one first storage volume available to the at least one application program so that write operations from the at least one application program can be performed to the at least one first storage volume during the act of migrating,

wherein the act (A) comprises, at least one point in time during the migration, an act of storing, in a non-volatile storage medium, state information indicating a portion of a data set of the at least one first storage volume successfully copied to the at least one second storage volume at the at least one point in time,

wherein the act (A) comprises acts of reading the data from the at least one first storage volume to at least one host, and writing the data from the at least one host to the at least one second storage volume; and

(C) resuming, after an interruption that results in a loss of at least some state information stored in at least one volatile storage medium in the computer system, the act of migrating without restarting from the beginning of the data set, the act of resuming being performed using the state information stored in the non-volatile storage medium.

35. The computer-readable medium according to claim 34, wherein the act of storing further comprises an act of storing, on at least one of the at least one first storage volume and at least one second storage volume, a checkpoint that indicates the portion of the data that was successfully copied to the at least one second storage volume.

36. The computer-readable medium according to claim 34, wherein the act of storing further comprises an act of storing, on at least one of the at least one first storage volume and at least one second storage volume, an indicator that indicates the portion of the data that was successfully copied to the at least one second storage volume.

37. The computer-readable medium according to claim 34, wherein the method further comprises an act of processing a write operation, the act of processing comprising acts of:

receiving at least one write operation from the at least one application program; and

writing data associated with the at least one write operation to both the at least one first storage volume and at least one second storage volume.

38. The computer-readable medium according to claim 37, wherein the act of processing a write operation further comprises an act of sending an acknowledgement to the at least one application only after data associated with the at least one write operation has been written to both the at least one first storage volume and at least one second storage volume.

39. The computer-readable medium according to claim 34, wherein the acts of reading data from the at least one first storage volume and writing the data from the at least one host to the at least one second storage volume are performed by a driver in an I/O stack of the at least one host.

40. The computer-readable medium according to claim 34, wherein the method further comprises acts of:

after the act of migrating is complete, intercepting an access request from the at least one application program to the at least one first storage volume; and

redirecting the access request to the at least one second storage volume, so that the access request can be serviced by the at least one second storage volume.

41. The computer-readable medium according to claim 40, wherein the acts of intercepting and redirecting are performed within the at least one host.

42. The computer-readable medium according to claim 41, wherein the at least one host includes an I/O stack, and wherein the acts of intercepting and redirecting are performed in the I/O stack.

43. The computer-readable medium according to claim 34, wherein the act of resuming further comprises resuming the act of migrating after the portion of the data identified by the state information as successfully copied.

44. The computer-readable medium of claim 34, wherein the act (A) further comprises migrating data from the at least one first storage volume to the at least one second storage volume under the control of at least one a migration layer in the computer system, wherein the at least one migration layer is separate from the at least one application program;

and

wherein the act of storing the state information further comprises an act of storing, on at least one of the at least one first storage volume and at least one second storage volume, a checkpoint that indicates the portion of the data that was successfully copied to the at least one second storage volume; and

wherein the method further comprises an act of, in response to a read operation from the at least one application program that reads data from the at least one first storage volume, writing to the at least one second storage volume the data read by the read operation.

45. The computer-readable medium according to claim 44, wherein the act of writing to the at least one second volume the data read by the read operation is performed if the data read by the read operation is located before the checkpoint.

46. The computer-readable medium according to claim 44, wherein the acts of reading data from the at least one first storage volume and writing the data from the at least one host to the at least one second storage volume are performed by a driver in an I/O stack of the at least one host.

47. The computer-readable medium according to claim 44, wherein the act of resuming further comprises resuming the act of migrating after the portion of the data identified by the state information as successfully copied.

48. A computer-readable medium encoded with instructions for execution on a computer system which includes at least one host, at least one storage system that stores data written by the host, and at least one controller which is external to the at least one storage system, the at least one storage system providing a plurality of storage volumes to the host to store data written by the host, the plurality of storage volumes including at least one first storage volume and at least one second storage volume, the instructions, when executed, causing the at least one controller to perform a method comprising acts of:

(A) migrating a data set from at least one first storage volume to at least one second storage volume, the at least one first storage volume being allocated to at least one application program in the computer system to store data wherein the act of migrating starts at a beginning of the data set and progresses to an end of the data set,

wherein the act (A) comprises, at least one point in time during the migration, an act of storing, in a non-volatile storage medium, state information indicating a portion of the data set of the at least one first storage volume successfully copied to the at least one second storage volume at the at least one point in time; and

(B) resuming, after an interruption that results in a loss of at least some state information stored in at least one volatile storage medium in the computer system, the act of migrating without restarting from the beginning of the data set, the act of resuming being performed using the state information stored in the non-volatile storage medium,

wherein the act of migrating comprises acts of reading the at least one data set from the at least one first storage volume to at least one host, and writing the data from the at least one host to the at least one second storage volume.

49. The computer-readable medium according to claim 48, wherein the method further comprises an act of maintaining the at least one first storage volume available to the at least one application program so that write operations from the at least one application program can be performed to the at least one first storage volume during the act of migrating.

50. The computer readable medium according to claim 48, wherein the interruption results in a loss of volatile state information for the at least one host.

51. The computer-readable medium according to claim 50, wherein the acts of reading data from the at least one first storage volume to the at least one host and writing the data from the at least one host to the at least one second storage volume are performed by a driver in an I/O stack of the at least one host.

52. The computer-readable medium according to claim 48, wherein the act of storing further comprises an act of storing, on at least one of the at least one first storage volume and the at least one second storage volume, a checkpoint that indicates the portion of the data that was successfully copied to the at least one second storage volume.

53. The computer-readable medium according to claim 48, wherein the act of storing further comprises an act of storing, on at least one of the at least one first storage volume and at least one second storage volume, an indicator that indicates a portion of the data that was successfully copied to the at least one second storage volume.

54. The computer-readable medium according to claim 48, wherein the method further comprises acts of performing a read operation on the at least one first volume and performing a write operation on the at least one second storage volume with data read by the act of reading.

55. The computer-readable medium according to claim 48, wherein the method further comprises an act of processing a write operation, the act of processing comprising acts of:

receiving at least one write operation from the at least one application program; and

writing data associated with the at least one write operation to the at least one first storage volume and the at least one second storage volume.

56. The computer-readable medium according to claim 55, wherein the act of processing a write operation further comprises an act of sending an acknowledgement to the at least one application only after data associated with the at least one write operation has been written to both the at least one first storage volume and the at least one second storage volume.

57. The computer-readable medium according to claim 48, wherein the method further comprises acts of:

after the act of migrating is complete, intercepting an access request from the at least one application program to the at least one first storage volume; and

redirecting the access request to the at least one second storage volume, so that the access request can be serviced by the at least one second storage volume.

58. The computer-readable medium according to claim 57, wherein the acts of intercepting and redirecting are performed within the at least one host.

59. The computer-readable medium according to claim 58, wherein the at least one host includes an input/output (I/O) stack, and wherein the acts of intercepting and redirecting are performed in the I/O stack.

60. The computer-readable medium according to claim 48, wherein the at least one controller is disposed on the at least one host.

61. The computer-readable medium according to claim 60, wherein the at least one host comprises a server or a client.

62. The computer-readable medium according to claim 48, wherein the at least one controller is disposed in a logical volume manager.

63. The computer-readable medium according to claim 48, wherein the at least one controller is disposed in a file system.

64. A host computer for migrating data in a computer system including the host computer and at least one storage system that stores data written by the host computer, the at least one storage system providing a plurality of storage volumes to the host computer to store data written by the host computer, the plurality of storage volumes including at least one first storage volume and at least one second storage volume, the host computer comprising:

a processor to execute at least one application program; and

a controller adapted to migrate data from the at least one first storage volume to the at least one second storage volume, the at least one storage volume being allocated to at least one application program to store data, the controller being further adapted to maintain the at least one first storage volume available to the at least one application program so that write operations from the at least one application program can be performed to the at least one first storage volume during the act of migrating, wherein the data migrating includes reading the data from the at least one first storage volume to the at least one host, and writing the data from the at least one host to the at least one second storage volume; and

further comprising means for storing, in a non-volatile storage medium and at least one point in time during the migration, state information indicating a portion of a data set of the at least one first storage volume successfully copied to the at least one second storage volume at the at least one point in time, wherein the controller is further adapted to resume, after an interruption that results in a loss of at least some state information stored in at least one volatile storage medium in the computer system, the act of migrating without restarting from the beginning of the data set, the act of resuming being performed using the state information stored in the non-volatile storage medium.

65. The host computer according to claim 64, further comprising means from reading the data from the at least one first storage volume to the at least one host, and writing the data from the at least one host to the at least one second storage volume.

66. The host computer according to claim 65, wherein the means for reading data from the at least one storage volume includes a driver in an input/output (110) stack of the at least one host.

67. The host computer according to claim 64, wherein the means for storing further comprises means for storing, on the at least one first storage volume and the at least one second storage volume, a checkpoint that indicates the portion of the data that was successfully copied to the at least one second storage volume.

68. The host computer according to claim 64, wherein the means for storing further comprises means for storing, on the at least one first storage volume and the at least one second storage volume, an indicator that indicates the portion of the data that was successfully copied to the at least one second storage volume.

69. The host computer according to claim 64, further comprising means for processing a write operation comprising:

means for receiving at least one write operation from the at least one application program; and

means for writing data associated with the at least one write operation to both the at least one first storage volume and the at least one second storage volume.

70. The host computer according to claim 69, wherein the means for processing a write request further comprises means for sending an acknowledgment to the at least one application only after data associated with the at least one write operation has been written to both the at least one first storage volume and the at least one second storage volume.

71. The host computer according to claim 64, further comprising:

means for intercepting, after migrating the data is complete, an access request from the at least one application program to the at least one first storage volume; and

means for redirecting the access request to the at least one second storage volume, so that the access request can be serviced by the at least one second storage volume.

72. The host computer according to claim 64, further comprising an input/output (I/O) stack, and wherein the controller is disposed within the I/O stack.

73. The host computer of claim 64, wherein the controller is further adapted to migrate data from the at least one first storage volume to the at least one second storage volume under the control of at least one migration layer in the computer system separate from the at least one application program,

wherein the means for storing state information further comprises means for storing, on the at least one first storage volume and the at least one second storage volume, a checkpoint that indicates the portion of the data that was successfully copied to the at least one second storage volume;

and

further comprising means, responsive to a read operation from the at least one application program that reads data from the at least one first storage volume, for writing to the at least one second storage volume the data read by the read operation.

74. The host computer according to claim 73, wherein the means for writing to the at least one second storage volume the data read by the read operation if the data read by the read operation is located before the checkpoint.

75. The host computer according to claim 73, further comprising means from reading the data from the at least one first storage volume to the at least one host, and writing the data from the at least one host to the at least one second storage volume.

76. The host computer according to claim 75, wherein the means for reading data from the at least one storage volume includes a driver in an input/output (I/O) stack of the at least one host.

77. A host computer for migrating data in a computer system including the host computer and at least one storage system that stores data written by the host computer, the at least one storage system providing storage volumes to the host computer to store data written by the host computer, the plurality of storage volumes including at least one first storage volume and at least one second storage volume, the host computer comprising:

a processor to execute at least one application program; and

a controller adapted to migrate a data set from at least one first storage volume to at least one second storage volume, the at least one first storage volume being allocated to at least one application program in the computer system to store data, wherein the controller begins migrating data from a beginning of the data set and progresses to an end of the data set; and

a storing layer that stores, in a non-volatile storage medium and at least one point in time during the migration, state information indicating a portion of the date set of the at least one first storage volume successfully copied to the at least one second storage volume at the at least one point in time,

wherein the controller is further adapted to resume, after an interruption that results in a loss of at least some state information stored in at least one volatile storage medium in the host computer, migrating the data set without starting from the beginning of the data set the resuming being performed using the state information stored in the non-volatile storage medium.

78. The host computer according to claim 77, further comprising means for reading the data from the at least one first storage volume to the at least one host, and writing the data from the at least one host to the at least one second storage volume.

79. The host computer according to claim 78, wherein the means for reading data from the at least one storage volume includes a driver in an input/output (I/O) stack of the at least one host.

80. The host computer according to claim 77, wherein the storing layer further comprises means for storing, on the at least one first storage volume and the at least one second storage volume, a checkpoint that indicates the portion of the data that was successfully copied to the at least one second storage volume.

81. The host computer according to claim 80, further comprising means, responsive to a read operation that reads data from the at least one first storage volume, for writing to the at least one second storage volume the data read by the read operation.

82. The host computer according to claim 81, wherein the means for writing to the at least one second storage volume the data read by the read operation if the data read by the read operation is located before the checkpoint.

83. The host computer according to claim 77, wherein the storing layer further comprises means for storing, on the at least one first storage volume and the at least one second storage volume, an indicator that indicates the portion of the data that was successfully copied to the at least one second storage volume.

84. The host computer according to claim 77, further comprising means for processing a write operation comprising:

means for receiving at least one write operation from the at least one application program; and

means for writing data associated with the at least one write operation to both the at least one first storage volume and the at least one second storage volume.

85. The host computer according to claim 84, wherein the means for processing a write request further comprises means for sending an acknowledgment to the at least one application only after data associated with the at least one write operation has been written to both the at least one first storage volume and the at least one second storage volume.

86. The host computer according to claim 77, further comprising:

means for intercepting, after migrating the data is complete, an access request from the at least one application program to the at least one first storage volume; and

means for redirecting the access request to the at least one second storage volume, so that the access request can be serviced by the at least one second storage volume.

87. The host computer according to claim 77, further comprising an input/output (I/O) stack, and wherein the controller is disposed within the 110 stack.

88. The host computer according to claim 77, wherein the at least one host comprises a server or a client.

89. The host computer according to claim 77, wherein the at least one controller is disposed in a logical volume manager.

90. The host computer according to claim 77, wherein the at least one controller is disposed in a file system.

91. The host computer of claim 77, further comprising:

means for receiving at least one write operation from the at least one application program;

means for writing data associated with the at least one write operation to both the at least one first storage volume and the at least one second storage volume; and

means for sending an acknowledgment to the at least one application only after data associated with the at least one write operation has been written to both the at least one first storage volume and the at least one second storage volume.

92. The host computer according to claim 91, further comprising means for reading the data from the at least one first storage volume to the apparatus, and writing the data from the apparatus to the at least one second storage volume.

93. The host computer according to claim 92, further comprising means for storing, in a non-volatile storage medium and at least one point in time during the migration, state information indicating a portion of the at least one first storage volume successfully copied to the at least one second storage volume at the at least one point in time.

94. The host computer according to claim 93, wherein the means for storing further comprises means for storing, on the at least one first storage volume and the at least one second storage volume, a checkpoint that indicates the portion of the data that was successfully copied to the at least one second storage volume.

95. The host computer according to claim 93, wherein the means for storing further comprises means for storing, on the at least one first storage volume and the at least one second storage volume, an indicator that indicates the portion of the data that was successfully copied to the at least one second storage volume.

96. The host computer according to claim 91, further comprising:

means for intercepting, after migrating the data is complete, an access request from the at least one application program to the at least one first storage volume; and

means for redirecting the access request to the at least one second storage volume, so that the access request can be serviced by the at least one second storage volume.

97. The host computer according to claim 91, further comprising a logical volume manager.

98. The host computer according to claim 91, further comprising a file system.