Data migration using parallel, distributed table driven I/O mapping

• US 20020019922A1
• Filed: 06/01/2001
• Published: 02/14/2002
• Est. Priority Date: 06/02/2000
• Status: Active Grant

• Current Assignee: Hewlett Packard Enterprise Development LP (Hewlett-Packard Enterprise Company)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Bean, Robert G., Wrenn, Richard F., Thiel, David W., Reuter, James M.
• Granted Patent: US 6,718,404 B2
• Time in Patent Office: Days
• US Class Current: 711/206
• CPC Class Codes: G06F 11/0727 in a storage system, e.g. in a DASD or network based storage system drivers for digital recording or reproducing units G06F3/06; circuits for error detection or correction within digital recording or reproducing units G11B20/18; for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS], H04L67/1097
• CPC Class Codes: G06F 11/0772 Means for error signaling, e.g. using interrupts, exception flags, dedicated error registers
• CPC Class Codes: G06F 11/0793 Remedial or corrective actions recovery from an exception in an instruction pipeline G06F9/3861; by retry G06F11/1402; for recovering from a failure of a protocol instance or entity H04L69/40
• CPC Class Codes: G06F 11/1435 using file system or storage system metadata
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 3/0605 by facilitating the interaction with a user or administrator
• CPC Class Codes: G06F 3/0608 Saving storage space on storage systems
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0611 in relation to response time
• CPC Class Codes: G06F 3/0614 Improving the reliability of storage systems
• CPC Class Codes: G06F 3/0617 in relation to availability
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/0647 Migration mechanisms
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/0665 at area level, e.g. provisioning of virtual or logical volumes
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 3/0683 Plurality of storage devices

Abstract

A system for moving physically stored data in a distributed, virtualized storage network is disclosed. A group of data sets is written to a first storage device as part of a write operation such as migration. A plurality of storage devices partially filled with data are designated as substitutes. The write operation to the first storage device is suspended upon receiving a request to read a data set stored in the first storage device, such as occurs in a recall operation. A second storage device is then selected from the plurality of substitute storage devices. The write operation is continued by writing data sets from the group of data sets included in the write operation that were not written to the first storage device to the selected second storage device. The requested data is then read from the first storage device. After data has been read from the first storage device, the first storage device may be designated as a substitute storage device so that the partially filled first storage device may be selected for continuing write operations. Data sets from substitute storage devices may be transferred or merged into a lesser number of storage devices during recycle operations to prevent the number of substitute storage devices from increasing beyond a predetermined limit or goal. Recycling operations in which data sets from different storage devices are transferred or merged may be performed by building a first queue including a list of filled tapes ordered according to the least amount of valid data and a second queue including all unassociated partially filled storage devices ordered by the amount of available storage space, and merging.

Claims

1. A method for moving information from a first physical storage location to a second physical storage location in a virtual storage system for linking a host to the storage locations, the system having an agent connected to the host, the agent storing a table, the table having entries that map to virtual storage disk positions to the locations on the storage devices, the table entries including an indication of whether a no-write (Nw) state is activated, whereby activation of the Nw state prevents writing operations to the storage location contained in that entry, and a controller connected to the agent, the controller controlling the contents of the table, the method comprising the steps of:

a) the controller activating the Nw state for the table entry corresponding to the first physical storage location;

b) copying information from the first physical storage location to the second physical storage location;

c) the controller changing the virtual mapping from the first physical location to the second physical location; and

d) the controller deactivating the Nw state for the table entry corresponding to the second physical storage location.

2. A virtual storage system for moving data accessed over a network from a first storage location to a second storage location on a storage device, the system comprising:

an agent having volatile memory for storing a first copy of a table, the table having entries to map virtual disk positions to locations on the storage device; and

a controller coupled to the agent, the controller having non-volatile memory for storing a second copy of the table and means for directing movement of data from the first storage location to the second storage location while the agent prevents any I/O operations that would change the data at the first storage location.

3. The system of claim 2, wherein said agent includes means for updating the first copy of the table in response to direction from the controller to match an updated copy of the second copy of the table.

4. The system of claim 2, wherein the table entries further include an indication of whether an invalid state is activated such that the invalid state for a table entry becomes activated when that table entry contains no useable mapping information.

5. The system of claim 4, wherein the agent does not allow the host to complete the I/O operations with one of the entries if the invalid state for that entry is activated.

6. The system of claim 2, wherein the table entries further include an indication of whether a Nw state is activated such that the Nw state for one of the entries becomes activated when data cannot be written to the storage location contained in that entry.

7. The system of claim 6, wherein the agent does not allow the host to write data to the storage location in one of the entries if the no-write state for that entry is activated.

8. The system of claim 2, further comprising a communication channel to couple the agent and the controller.

9. The system of claim 8, wherein the communication channel employs a data transfer protocol to transport messages on the communication channel.

10. The system of claim 2, wherein the entries include an offset.

11. The system of claim 10, wherein the offset includes a logic unit number identifier.

12. The system of claim 10, wherein the offset includes a block identifier.

14. The method of claim 13, further comprising setting a blocking flag until operations are completed.

15. The method of claim 13, further comprising obtaining mapping information from one of the entries in the table.

13. A method for maintaining a table for mapping virtual disk blocks to migrated storage locations on storage devices within a network, comprising:

activating states within entries of the table;

migrating data from a first storage location to a second storage location;

completing operations at the table; and

updating the table in response to the command.

16. A method for moving data from a first storage location to a second storage location on a storage device in a virtual disk storage system, comprising:

providing an agent having volatile memory for storing a first copy of a table, the table having entries to map virtual disk positions to locations on the storage device;

providing a controller coupled to the agent, the controller having non-volatile memory for storing a second copy of the table;

responsive to a request to move virtual disk data from a first location to a second location, the controller changing the second copy of the table and issuing an order to change the first copy of the table to the agent;

after completion of all prior I/O operations, the agent alerting the controller of completion;

the controller then copying the contents of the first storage location to the second storage location;

the controller then updating the second copy of the table and directing the agent to update the first copy of the table to match the updated second copy of the table; and

after completion of all prior I/O operations, the agent updating the first copy of the table to match the updated second copy of the table and responding to the controller.