Storage Device, and Data path Failover Method of Internal Network of Storage Controller

• US 20100115329A1
• Filed: 12/18/2008
• Published: 05/06/2010
• Est. Priority Date: 10/30/2008
• Status: Active Grant

• Current Assignee: Hitachi, Ltd.
• Original Assignee: Hitachi, Ltd.
• Inventors: Shimada, Kentaro, TANAKA, Katsuya
• Granted Patent: US 8,082,466 B2
• Time in Patent Office: Days
• US Class Current: 714/5
• CPC Class Codes: G06F 11/201 between storage system components

Abstract

A storage device in which the MR-IOV is applied to an internal network of a storage controller. Data path failover can be executed in the storage device. The internal network of the storage controller is configured to enable the access of a virtual function (VF) “VF 0:0, 1” of each endpoint device (ED0-ED2) from a root port RP0. Likewise, “VF 1:0, 1” of each endpoint device can be accessed from a root port RP1. In a first data path from the RP0 to ED0 in a normal state, “VF 0:0, 1” and “MVF 0, 0” are connected by VF mapping. When a failure occurs on the first data path, the MR-PCIM executes the VF migration, whereby in the second data path from the RP1 to ED0, “VF 1:0, 1” and “MVF 0, 0” are connected by VF mapping. As a result, failover to the second data path is realized.

Claims

1. A storage device having an internal network in which components located within a storage controller are connected by PCI Express, wherein

the storage controller and the internal network thereof comprise:

one or more root complexes that have a plurality of root ports as a whole, each root complex having one or more root ports;

one or more processors connected to each root complex;

a plurality of endpoint devices that are multi-root aware devices provided with predetermined interfaces and having a plurality of virtual functions that can be accessed from each of the plurality of root ports; and

a switch that is a multi-root aware switch that connects the plurality of root ports and the plurality of endpoint devices, configures data paths, and has a plurality of ports;

at least one of the plurality of endpoint devices can execute a virtual function migration of changing a mapping of the virtual function to the predetermined interface;

in the storage controller and the internal network thereof, the switch and the endpoint devices are so set that all of the plurality of root ports can access virtual functions located within all of the plurality of endpoint devices; and

when a failure occurs in one data path from one of the plurality of root ports to one of the plurality of endpoint devices via the switch, the storage controller and the internal network thereof perform a processing of switching an access source root port of the data path and a processing of switching a mapping of a virtual function of the access target endpoint device to the predetermined interfaces by the virtual function migration in switching from a first data path in a failure state to an alternative second data path, as a processing for executing a data path failover.

2. The storage device according to claim 1, wherein

a first endpoint device comprising a front-end interface for connecting a host device and capable of executing the virtual function migration, and

a second endpoint device comprising a back-end interface for connecting a group of storage devices and capable of executing the virtual function migration

are provided as the endpoint devices,

the switch connects the plurality of root ports to the first and second endpoint devices,

in the storage controller and the internal network thereof, the switch and the first and second endpoint devices are so set that all of the plurality of root ports can access virtual functions located within all of the plurality of endpoint devices;

and

when a failure occurs in one data path from one of the plurality of root ports to one device of the first and second endpoint devices, the storage controller and the internal network thereof perform a processing of switching an access source root port of the data path and a processing of switching a mapping of a virtual function of the access target endpoint device to the predetermined interfaces by the virtual function migration in switching from a first data path in a failure state to an alternative second data path, as a processing for executing a data path failover.

3. The storage device according to claim 2, wherein

a third endpoint device comprising a cache memory interface for connecting a cache memory is provided as the endpoint device,

the switch connects the plurality of root ports to the first and second and third endpoint devices,

the switch and the first and second and third endpoint devices are so set that all of the plurality of root ports can access virtual functions located within all of the plurality of endpoint devices;

and

when a failure occurs in one data path from one of the plurality of root ports to one device of the first and second and third endpoint devices, a processing of switching an access source root port of the data path and a processing of switching a mapping of a virtual function of the access target endpoint device to the predetermined interfaces by the virtual function migration in switching from a first data path in a failure state to an alternative second data path are performed as a processing for executing a data path failover.

4. The storage device according to claim 3, wherein

the third endpoint device comprises a mission virtual function that will be mapped to the virtual function, the cache memory interface is mapped to the mission virtual function, and the number of mission virtual functions provided in the third endpoint device is equal to or larger than the number of root ports that access the third endpoint device.

5. The storage device according to claim 3, wherein

in the storage controller and the internal network thereof,

the front-end interface that becomes a mission virtual function of the first endpoint device and the cache memory interface that becomes a mission virtual function of the third endpoint device are set to be accessible from the same root port, and

the back-end interface that becomes a mission virtual function of the second endpoint device and the cache memory interface that becomes a mission virtual function of the third endpoint device are set to be accessible from the same root port.

6. The storage device according to claim 3, wherein

each of the plurality of root complexes has a first root port and a second root port, and

in the storage controller and the internal network thereof,

the first root port of each root complex is set to enable the access to the front-end interface that becomes a mission virtual function of the first endpoint device and the cache memory interface that becomes a mission virtual function of the third endpoint device, and

the second root port of each root complex is set to enable the access to the back-end interface that becomes a mission virtual function of the second endpoint device and the cache memory interface that becomes a mission virtual function of the third endpoint device.

7. The storage device according to claim 1, wherein

the endpoint device is provided with a first function for data transfer relating to data reading from the storage device or data writing thereto and a second function that can control a state transition of a virtual function associated with the first function.

8. The storage device according to claim 7, wherein

a multi-root PCI manager program is provided in the storage controller or a maintenance terminal device connected thereto, and

in a configuration of virtual hierarchy from the root port to an endpoint device,

the endpoint device causes a state transition of a virtual function of the first function by inbound communication in a virtual hierarchy in which a data path failure has not occurred from the multi-root PCI manager program to the second function.

9. The storage device according to claim 7, wherein

a multi-root PCI manager program is provided in the storage controller or a maintenance terminal device connected thereto, and

the endpoint device causes a state transition of a virtual function of the first function by outbound communication from the multi-root PCI manager program to the second function.

10. The storage device according to claim 7, wherein

a single-port PCI manager program is provided in the storage controller, and

in a configuration of virtual hierarchy from the root port to an endpoint device,

the endpoint device determines whether or not communication is possible between the single-root PCI manager program in a virtual hierarchy in which a data path failure has occurred and the second function by an autonomous function and causes a state transition of a virtual function of the first function when the communication is impossible.

11. The storage device according to claim 1, wherein

the storage controller and the internal network thereof manage a plurality of tables including:

a table that manages a mapping between the root ports and virtual functions within endpoint devices;

a table that manages a mapping of the virtual function to the predetermined interface; and

a table that manages a migration of the virtual function during failover, and

the storage controller and the internal network thereof perform a processing of switching an access source root port of the data path and a processing of switching a mapping of a virtual function of the access target endpoint device to the predetermined interfaces by the virtual function migration with reference to the plurality of tables when a processing of executing a failover of the data path is executed.

12. A data path failover method for an internal network of a storage controller in which components located within a storage controller are connected by PCI Express, wherein the internal network of the storage controller comprises a plurality of root ports as a whole, a processor connected to a root complex having the root ports, a plurality of endpoint devices that are provided with predetermined interfaces and have a plurality of virtual functions that can be accessed from each of the plurality of root ports, and a switch that connects the plurality of root ports and the plurality of endpoint devices, configures data paths, and has a plurality of ports;

at least one of the plurality of endpoint devices can execute a virtual function migration of changing a mapping of the virtual function to the predetermined interface;

in the internal network of the storage controller, the switch and the endpoint devices are so set that all of the plurality of root ports can access virtual functions located within all of the plurality of endpoint devices;

the internal network of the storage controller comprising a processing step of switching an access source root port of the data path and a processing step of switching a mapping of a virtual function of the access target endpoint device to the predetermined interface by the virtual function migration in switching from a first data path in a failure state to an alternative second data path, as processing steps for executing a data path failover when a failure occurs in one data path from one of the plurality of root ports to one of the plurality of endpoint devices via the switch.

13. The data path failover method for an internal network of a storage controller according to claim 12, wherein

a single-root PCI manager program and a failover control program are provided in the internal network of the storage controller;

a multi-root PCI manager program is provided in the internal network of the storage controller or a maintenance terminal device connected to the storage controller; and

the internal network of the storage controller comprises;

a processing step in which the multi-root PCI manager program notifies the failover control program about a data path failure occurrence;

a processing step in which the failover control program refers to a failover management table located within the internal network and requests the virtual function migration to the multi-root PCI manager program on the basis of the notification information;

a processing step in which the multi-root PCI manager executes the virtual function migration in response to the request and updates a table managing a mapping of the virtual function located within the internal network to the predetermined interface; and

a processing step in which the failover control program updates the failover management table, following the execution of the virtual function migration

as a processing for executing a data path failover when a failure occurs in one data path from one of the plurality of root ports to one of the plurality of endpoint devices via the switch.

14. The data path failover method for an internal network of a storage controller according to claim 13, comprising

a processing step in which the multi-root PCI manager program executes the virtual function migration and updates the virtual function mapping management table with respect to a data path in which the failure of the data path has not occurred;

and

a processing step in which the failover control program updates the failover management table, following the execution of the virtual function migration.

15. The data path failover method for an internal network of a storage controller according to claim 12, wherein

the endpoint device is provided with a first function for data transfer relating to data reading from the storage device or data writing thereto and a second function that can control a state transition of a virtual function associated with the first function, and

the method comprises a processing step in which the end point device causes a state transition of a virtual function of the first function on the basis of communication from the single-root PCI manager program or the multi-root PCI manager program to the second function.