System and method for on-line, real time, data migration

• US 6,356,977 B2
• Filed: 12/12/2000
• Issued: 03/12/2002
• Est. Priority Date: 09/01/1995
• Status: Expired due to Term

• Current Assignee: EMC Corporation (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: Ofek, Yuval, Yanai, Moshe
• Primary Examiner(s): Kim, Matthew
• Assistant Examiner(s): BATAILLE, PIERRE MICHE
• Application Number:
  US09/735,023
• Publication Number:
  US 20010001870A1
• Time in Patent Office: 455 Days
• Field of Search: 711/112, 711/114, 711/161, 711/162, 711/165, 711/111, 711/163, 714/6, 714/7, 714/8, 712/20, 709/217, 709/219, 709/226
• US Class Current: 711/112
• CPC Class Codes: G06F 12/0638 Combination of memories, e.g. ROM and RAM such as to permit replacement or supplementing of words in one module by words in another module address formation of the next microinstruction G06F9/26; masking faults in memories by using spares or by reconfiguring G11C29/70
• CPC Class Codes: G06F 12/08 in hierarchically structured memory systems, e.g. virtual memory systems
• CPC Class Codes: G06F 12/0866 for peripheral storage systems, e.g. disk cache
• CPC Class Codes: G06F 3/0607 by facilitating the process of upgrading existing storage systems, e.g. for improving compatibility between host and storage device
• CPC Class Codes: G06F 3/0647 Migration mechanisms
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 3/0689 Disk arrays, e.g. RAID, JBOD
• CPC Class Codes: G11B 19/02 Control of operating function, e.g. switching from recording to reproducing
• CPC Class Codes: G11B 5/86 Re-recording, i.e. transcribing information from one magnetisable record carrier on to one or more similar or dissimilar record carriers by varying the order of the information G11B27/029, G11B27/036

Abstract

A system and method for providing on-line, real-time, transparent data migration from an existing storage device to a replacement storage device. The existing and replacement storage devices are connected as a composite storage device that is coupled to a host, network or other data processing system. The replacement storage device includes a table which identifies data elements that have migrated to the replacement storage device. When a host system makes a data transfer request for one or more data elements, the replacement storage device determines whether the data elements have been migrated. If the data elements have migrated, the replacement storage device responds to the data transfer request independently of any interaction with the existing storage device. If the data elements have not migrated, the replacement storage device migrates the requested data elements and then responds to the data request and updates the data element map or table. When not busy servicing other requests, the replacement storage device operates in a background mode to migrate data elements so the data migration can occur concurrently with and transparently to system operations.

Claims

1. A system for on-line replacement of an existing first data storage device connected to said data processing device over a pre-existing data communication path by a replacement second data storage device, said pre-existing path being disconnected from the existing first data storage device and connected to the replacement second data storage device, and a second replacement data communication path being established between the existing and replacement storage devices, the replacement storage device including:

A) first mode means for migrating data elements from the existing to the replacement storage device over the second path in a first mode independently of and concurrently with processing of data by the data processing device and,

B) second mode means operable in a second mode in response to commands received from the data processing device over the preexisting first path for accessing data elements yet to be migrated from the existing to the replacement storage device over the second path, whereby said first and second mode means perform transfers over the second path until all the data elements required for access by the data processing device have migrated to the replacement storage device whereupon thereafter all commands from the data processing device are processed in the replacement storage device and interaction with the existing first data storage device ceases.

2. The system of claim 1 wherein the existing first storage device stores data elements in sequentially addressed locations defining a data block, wherein each of said first and second mode means transfers a data block from the existing storage device to the replacement storage device and wherein the replacement storage device additionally includes:

i) a data element map which is initialized to indicate that all data blocks in the existing storage device require data migration, and

ii) map updating means for updating said data element map in response to each data block migration thereby to terminate the data migration when all data elements required for access by the data processing device have migrated to the replacement storage device.

3. The system of claim 2 wherein said replacement second data storage device includes means for copying data from a first location to a second location in response to control parameters including an identification of the first and second locations, each of said first and second mode means identifying data blocks in the existing first storage device as first locations and locations in the replacement storage device as second locations, said first mode means including a background mode control for establishing first values of others of the control parameters and said second mode means including foreground mode control for establishing second values for the others of the control parameters in response to commands received from the data processing device to access data blocks located only in the existing storage device.

4. A method for providing on-line, transparent data migration from a first data storage device that is an existing data storage device connected to said data processing device over a pre-existing data communication path to a second data storage device that is a replacement for said first data storage device, the method comprising the steps of:

A) connecting said existing first and replacement second data storage devices with a first data communication path between said data processing device and said replacement second data storage device and with a second data communication path between said existing first and replacement second data storage devices;

B) establishing a first mode of operation of the replacement second data storage device for migrating data elements from addressed storage locations in said existing first data storage device over said second path to addressed storage locations in said replacement second data storage device independently of operations of said data processing device, and

C) establishing a second mode of operation of the replacement second data storage device in response to a read or write command from said data processing device over said first path determining whether a data element in an addressed storage location identified in the command has already been migrated to said replacement second data storage device and if it has yet to be migrated, causing a migration of the data element in the addressed storage location in said existing first data storage device to a corresponding addressed storage location in said replacement second data storage device over said second path and recording the migration of each data element.

5. The method of claim 4 including the step of, prior to coupling said second data storage device to said data processing device by said first path performing the step of uncoupling said pre-existing path between said first data storage device and said data processing device.

6. The method of claim 4 wherein said second data storage device includes a data element map providing an indication of whether a data element having a predetermined addressed storage location identified in the command is stored in said second data storage device, said method including in the second mode the steps of:

i) receiving in said second data storage device over said first path from said data processing device a data transfer command comprising a data element read or write request regarding at least one data element stored at a predetermined addressed storage location in said first data storage device,

ii) accessing said data element map by said second data storage device to determine if said at least one data element is stored in said second data storage device; and

iii) responding to said accessing by causing migration of said at least one data element from said first to said second data storage device over said second path and recording the migration of said at least one data element by updating the data element map.

7. The method of claim 6 wherein the response to the receipt of a data read request by the storage device includes the steps of:

i) determining whether a data element identified by the data read request is stored in said second data storage device;

ii) if the requested data element is stored in said second data storage device providing the requested data element to the data processing device over said first path from the second data storage device; and

iii) if the requested one data element is not stored in said second data storage device transferring the data element from the first data storage device to a corresponding location in said second data storage device over said second path providing the requested data element to the data processing device over the first path from the second data storage device.

8. The method of claim 6 wherein the response to the receipt of a data write request by the second storage device includes the steps of:

i) determining whether a data element has previously been transferred from the first to the second data storage device;

ii) if the transfer has occurred, writing the data element from said data processing device over said first path;

iii) if the transfer has yet to occur, transferring the data element from the first data storage device to the second data storage device over the second path and thereafter writing the data element to the second data storage device over the first path.

9. The method of claim 6 further including the steps of determining that the second data storage device is not completely busy responding to data element read or write requests from the data processing device, determining from the data element map which data elements stored in the first data storage device have yet to be migrated to the second data storage device, and transferring the yet to be migrated data elements to the second data storage device from the first data storage device and updating the data element map to indicate that the data element migration has occurred.

10. The method of claim 4 further including the steps of performing the first and second mode migrations over the second path until all the data elements required for access by the data processing device have migrated to the second data storage device whereupon thereafter all commands from the data processing device are processed by the second data storage device.

11. The method of claim 10 wherein the first data storage device stores data elements in sequentially addressed locations defining a data block, wherein each of said first and second mode migrations transfers a data block from the first data storage device to the second data storage device and wherein the second data storage device includes a data map which is initialized to indicate that all data blocks in the first data storage device require data migration, and is updated in response to each data block migration thereby to terminate the data migration when all data elements required for access by the data processing device have migrated to the second data storage device.

12. The method of claim 11 wherein said first and second mode migrations are effected by controlling the operation of a copy subroutine for transferring data blocks from the first to the second data storage device in response to control parameters, said first mode migration including background mode control for establishing first values of the control parameters, and said second mode migration including foreground mode control for establishing second values for the control parameters in response to commands issued by the data processing device to access data blocks located only in the first storage device.

13. In a data processing system including a first data storage device holding data comprising a plurality of data elements in a plurality of addressed storage locations and a data processing device for issuing read and write data transfer commands to said first data storage device and other system devices, said commands for said first data storage device identifying specific ones of said addressed storage locations, apparatus for providing an on-line transparent data migration to a second data storage device for holding data elements in a plurality of addressed storage locations, said apparatus additionally comprises:

A) means for interposing said second data storage device between the data processing system and the first data storage device whereby data transfer commands previously issued to the first data storage device are issued to said second data storage device,

B) means for transferring data elements from the first data storage device to said second data storage device independently of operations of the data processing system,

C) command responsive means responsive to a read or write command issued from the data processing device for determining whether a predetermined data element in an addressed storage location identified in the command is stored in said second data storage device and, if it has yet to be stored, migrating the data element in the corresponding addressed storage location in the first data storage device to said second data storage device,

D) means for recording the migration of each data element by said transfer means and said command responsive means.

14. The system of claim 13 wherein said second data storage device includes a data element map for recording the migration of each data element for each of the first and second data storage devices.

15. The system of claim 14 wherein said second data storage device is responsive to a read command for a predetermined data element received from said data processing device and to an indication in said data element map that said predetermined data element is yet to be stored in said second data storage device for causing the said predetermined data element in said first data storage device to be migrated to said second data storage device and for updating the recording of the migration in said data element map.

16. The system of claim 15 wherein said second data storage device is operable to cause the transfer of a plurality of predetermined data elements from said first data storage device to said second data storage device.

17. The system of any of claim 14 wherein said second data storage device is responsive to an indication that said data write command received from said data processing device is for data to be stored in a portion of the addressed storage location for a data element, for causing a data migration of the data element in the addressed storage location in said first data storage device to the addressed storage location in the second data storage device and thereafter transferring the data to the addressed storage location in the second, data storage device.

18. The system of any of claim 14 wherein said second data storage device is responsive to an indication that said data write command received from said data processing device is for a predetermined data element for being stored in the addressed storage location, for writing said received data element to said second data storage device and updating the migration recording in the data element map.