System and method for on-line, real time, data migration

• US 6,240,486 B1
• Filed: 07/29/1999
• Issued: 05/29/2001
• Est. Priority Date: 09/01/1995
• Status: Expired due to Term

• Current Assignee: EMC Corporation (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: Ofek, Yuval, Yanai, Moshe
• Primary Examiner(s): Kim, Matthew
• Assistant Examiner(s): BATAILLE, PIERRE MICHE
• Application Number:
  US09/363,482
• Time in Patent Office: 670 Days
• Field of Search: 711/112, 711/100, 711/111, 711/151, 711/156, 711/162, 711/165, 711/202, 711/203, 711/205, 714/7, 710/20
• US Class Current: 711/112
• CPC Class Codes: G06F 12/0638 Combination of memories, e.g. ROM and RAM such as to permit replacement or supplementing of words in one module by words in another module address formation of the next microinstruction G06F9/26; masking faults in memories by using spares or by reconfiguring G11C29/70
• CPC Class Codes: G06F 12/08 in hierarchically structured memory systems, e.g. virtual memory systems
• CPC Class Codes: G06F 12/0866 for peripheral storage systems, e.g. disk cache
• CPC Class Codes: G06F 3/0607 by facilitating the process of upgrading existing storage systems, e.g. for improving compatibility between host and storage device
• CPC Class Codes: G06F 3/0647 Migration mechanisms
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 3/0689 Disk arrays, e.g. RAID, JBOD
• CPC Class Codes: G11B 19/02 Control of operating function, e.g. switching from recording to reproducing
• CPC Class Codes: G11B 5/86 Re-recording, i.e. transcribing information from one magnetisable record carrier on to one or more similar or dissimilar record carriers by varying the order of the information G11B27/029, G11B27/036

Abstract

A system and method for providing on-line, real-time, transparent data migration from an existing storage device to a replacement storage device. The existing and replacement storage devices are connected as a composite storage device that is coupled to a host, network or other data processing system. The replacement storage device includes a table which identifies data elements that have migrated to the replacement storage device. When a host system makes a data transfer request for one or more data elements, the replacement storage device determines whether the data elements have been migrated. If the data elements have migrated, the replacement storage device responds to the data transfer request independently of any interaction with the existing storage device. If the data elements have not migrated, the replacement storage device migrates the requested data elements and then responds to the data request and updates the data element map or table. When not busy servicing other requests, the replacement storage device operates in a background mode to migrate data elements so the data migration can occur concurrently with and transparently to system operations.

Claims

1. In a data processing system with a host system that processes data stored in blocks of sequentially addressed storage locations of an existing magnetic disk array storage device connected to a host system over a preexisting path, data migration apparatus for transferring the data to a storage locations in a replacement magnetic disk array storage device comprising:

A. connection means for establishing first and second paths, the first path being established between the host system and the replacement storage device as a replacement for the preexisting path and the second path being established between the replacement storage device and the existing storage device, and

B. data migration means for transferring the data stored in the existing storage device to the replacement storage device over the second path independently of and concurrently with the host system processing of the data, said data migration means responding to a data transfer request from the host system and including;

i. a copy subroutine means for migrating data blocks from the existing storage device to the replacement storage device in response to control parameters including the address of a data block and a mode parameter,

ii. first transfer means including a foreground mode controller for establishing first value of the mode control parameter in response to data transfer requests for data blocks located only in the existing storage device, said first transfer means being responsive to a data transfer request from the host system for transferring data between the storage locations in the replacement storage device and the host system over the first path,

iii. second transfer means for transferring data from data blocks of the sequentially addressed locations in the existing storage device to corresponding locations in the replacement storage device over the second path prior to a transfer of the data between the replacement storage device and the host system, said second transfer means including a background mode controller for establishing a second value of the mode control parameter,

iv. a table with a flag corresponding to each data block that is initialized to a first value indicating that the corresponding data block is only located in the existing storage device and a second value indicating that the corresponding data block has migrated to the replacement storage device, said copy subroutine including means for establishing the second flag value in response to each data block migration from the existing storage device to the replacement storage device thereby to terminate the data migration when all data elements have migrated to the replacement storage device and all said flags have the second value,

v. control means in the replacement storage device for controlling said first and second transfer means until the data in all the storage locations of the existing storage device have transferred to the corresponding storage locations in the replacement storage device,

vi. means for determining the existence of a significant pattern of accesses to the existing storage device controlled by said foreground mode controller, and

vii. means for altering the control parameters from said background mode controller in response to the occurrence of the significant pattern.

2. Data migration apparatus as recited in claim 1 wherein said means for determining the existence of a significant pattern includes:

A. means for defining identifiable statistical blocks comprising a predetermined number of contiguous data blocks; and

B. means for counting successive data transfer requests initiated by said foreground controller that access a given statistical block for a predetermined number of data transfers;

C. said background mode controller includes means responsive to the interruption flag for loading the address of the statistical block as a control parameter.

3. Data migration apparatus as recited in claim 2 additionally comprising an interruption flag:

A. said background mode controller including means responsive to the interruption flag for loading the address of the statistical block; and

B. i) means for monitoring the number of data transfers by the copy subroutine in response to the operation of said background controller; and

C. means for setting the interruption flag only after a predetermined number of iterations of the copy subroutine have been performed in response to the operation of said background controller.

4. A method for migrating data elements from an existing magnetic disk array storage device, that stores data in blocks of sequentially addressed storage locations and that is connected to a host system over a preexisting path, to a replacement magnetic disk array storage device comprising the steps of:

A. connecting the existing and replacement storage devices with a first path between the host system and the replacement storage device as a replacement for the preexisting path and with a second path between the existing and replacement storage devices,

B. transferring data elements from the existing storage device over the second path to the replacement storage device independently of and concurrently with the host system operations, including the steps of;

i. defining a copy subroutine for migrating data blocks from the existing storage device to the replacement storage device in response to control parameters including the address of a data block and a mode parameter,

ii. initializing to a first value each flag in a table that corresponds to each data block that is only located in the existing storage device to indicate that all data blocks in the existing storage device require data migration and changing each flag to a second value in response to the migration of a corresponding data block thereby to terminate the data migration when all data elements have migrated to the replacement storage device and all the flags have the second value,

iii. performing first transfers with the host system over the first path in response to data transfer requests from the host system, said first transfer establishing the first value for the mode parameter in response to data transfer requests to data blocks located only in the existing storage device,

iv. performing second transfers from the existing storage device to the replacement storage device over the second path and including the establishment of the second value of the mode parameter,

v. determining the existence of a significant pattern of accesses to the existing storage device controlled by said foreground mode controller, and

vi. altering the control parameters from said background mode controller in response to the occurrence of the significant pattern, and

vii. controlling said first and second transfers until all the data elements have migrated to the replacement storage device whereupon thereafter all host system transfer requests are processed in the replacement storage device, and

C. in response to a predetermined data transfer request for a data element received from the host system over the first path, effecting a transfer of the selected data element between the replacement storage device and the host system over the first path.

5. A method as recited in claim 4 wherein said step of determining the existence of a significant pattern includes:

A. defining identifiable statistical blocks comprising a predetermined number of contiguous data blocks; and

B. counting successive data transfer requests initiated by said foreground controller that access a given statistical block for a predetermined number of data transfers;

C. said background mode control includes responding to the interruption flag being set by loading the address of the statistical block as a control parameter.

6. A method as recited in claim 5 wherein:

said step of determining the existence of a significant pattern additionally includes setting the interruption flag; and

said step of background mode control includes responding to the interruption flag by loading the address of the statistical block; and

said step of altering comprises;

i) monitoring the number of data transfers by the copy subroutine in response to said background mode control; and

ii) setting the interruption flag only after a predetermined number of iterations of the copy subroutine have been performed in response to said background mode control.