Storage device, and data path failover method of internal network of storage controller

• US 8,082,466 B2
• Filed: 12/18/2008
• Issued: 12/20/2011
• Est. Priority Date: 10/30/2008
• Status: Expired due to Fees

• Current Assignee: Hitachi, Ltd.
• Original Assignee: Hitachi, Ltd.
• Inventors: Tanaka, Katsuya, Shimada, Kentaro
• Primary Examiner(s): Beausoliel, Jr., Robert
• Assistant Examiner(s): Lottich, Joshua P
• Application Number:
  US12/338,173
• Publication Number:
  US 20100115329A1
• Time in Patent Office: 1,097 Days
• Field of Search: 714/3, 714/4, 714/5, 714/4.11, 714/6.1, 714/6.3
• US Class Current: 714/6.3
• CPC Class Codes: G06F 11/201 between storage system components

Abstract

A storage device in which the MR-IOV is applied to an internal network of a storage controller. Data path failover can be executed in the storage device. The internal network of the storage controller is configured to enable the access of a virtual function (VF) “VF 0:0, 1” of each endpoint device (ED0-ED2) from a root port RP0. Likewise, “VF 1:0, 1” of each endpoint device can be accessed from a root port RP1. In a first data path from the RP0 to ED0 in a normal state, “VF 0:0, 1” and “MVF 0, 0” are connected by VF mapping. When a failure occurs on the first data path, the MR-PCIM executes the VF migration, whereby in the second data path from the RP1 to ED0, “VF 1:0, 1” and “MVF 0, 0” are connected by VF mapping. As a result, failover to the second data path is realized.

Claims

1. A storage device having an internal network in which components located within a storage controller are connected by PCI Express, wherein,

the storage controller and the internal network thereof comprise:

one or more root complexes that have a plurality of root ports as a whole, each root complex having one or more root ports;

one or more processors connected to each root complex;

a plurality of endpoint devices that are multi-root aware devices provided with predetermined interfaces and having a plurality of virtual functions that can be accessed from each of the plurality of root ports; and

a switch that is a multi-root aware switch that connects the plurality of root ports and the plurality of endpoint devices, configures data paths, and has a plurality of ports;

at least one of the plurality of endpoint devices can execute a virtual function migration of changing a mapping of one of the plurality of the virtual functions to one of the predetermined interfaces;

wherein

a first endpoint device comprising a front-end interface for connecting a host device and capable of executing the virtual function migration, and

a second endpoint device comprising a back-end interface for connecting a group of storage devices and capable of executing the virtual function migration

are provided as the plurality of endpoint devices,

the switch connects the plurality of root ports to the first and second endpoint devices,

in the storage controller and the internal network thereof, the switch and the first and second endpoint devices are so set that all of the plurality of root ports can access virtual functions located within all of the plurality of endpoint devices; and

when a failure occurs in one data path from one of the plurality of root ports to one device of the first and second endpoint devices, the storage controller and the internal network thereof perform a processing of switching an access source root port of the data path and a processing of switching a mapping of a virtual function of the access target endpoint device to the predetermined interfaces by the virtual function migration in switching from a first data path in a failure state to an alternative second data path, as a processing for executing a data path failover,

wherein

the plurality of endpoint devices further comprises a third endpoint device comprising a cache memory interface for connecting a cache memory,

the switch connects the plurality of root ports to the first and second and third endpoint devices,

the switch and the first and second and third endpoint devices are so set that all of the plurality of root ports can access virtual functions located within all of the plurality of endpoint devices; and

when a failure occurs in one data path from one of the plurality of root ports to one device of the first and second and third endpoint devices via the switch, the storage controller and the internal network thereof perform a processing of switching an access source root port of the data path and a processing of switching a mapping of a virtual function of the access target endpoint device to the predetermined interfaces by the virtual function migration in switching from a first data path in a failure state to an alternative second data path are performed as a processing for executing a data path failover.

2. The storage device according to claim 1, wherein

the third endpoint device comprises a number of mission virtual functions that will be mapped to the plurality of virtual functions, the cache memory interface is mapped to one of the number of the mission virtual functions, and the number of mission virtual functions provided in the third endpoint device is equal to or larger than the number of root ports that access the third endpoint device.

3. The storage device according to claim 2, wherein

each of the first endpoint device, second endpoint device, and third endpoint device comprises a mission virtual function,

in the storage controller and the internal network thereof,

the front-end interface that becomes the mission virtual function of the first endpoint device and the cache memory interface that becomes the mission virtual function of the third endpoint device are set to be accessible from the same root port, and

the back-end interface that becomes the mission virtual function of the second endpoint device and the cache memory interface that becomes the mission virtual function of the third endpoint device are set to be accessible from the same root port.

4. The storage device according to claim 1, wherein

each of the first endpoint device, second endpoint device, and third endpoint device comprises a mission virtual function,

each of the one or more root complexes has a first root port and a second root port, and

in the storage controller and the internal network thereof,

the first root port of each root complex is set to enable the access to the front-end interface that becomes the mission virtual function of the first endpoint device and the cache memory interface that becomes the mission virtual function of the third endpoint device, and

the second root port of each root complex is set to enable the access to the back-end interface that becomes the mission virtual function of the second endpoint device and the cache memory interface that becomes the mission virtual function of the third endpoint device.

5. The storage device according to claim 1, wherein

the plurality of endpoint devices are provided with a first function for data transfer relating to data reading from the storage device or data writing thereto and a second function that can control a state transition of a virtual function associated with the first function.

6. The storage device according to claim 5, wherein

a multi-root PCI manager program is provided in the storage controller or a maintenance terminal device connected thereto, and

in a configuration of virtual hierarchy from the root port to an endpoint device, the endpoint device causes a state transition of a virtual function of the first function by in-band communication in a virtual hierarchy in which a data path failure has not occurred from the multi-root PCI manager program to the second function.

7. The storage device according to claim 5, wherein

a multi-root PCI manager program is provided in the storage controller or a maintenance terminal device connected thereto, and

the plurality of endpoint devices cause a state transition of a virtual function of the first function by out-band communication from the multi-root PCI manager program to the second function.

8. The storage device according to claim 5, wherein

a single-root PCI manager program is provided in the storage controller, and

in a configuration of virtual hierarchy from the root port to an endpoint device,

the endpoint device determines whether or not communication is possible between the single-root PCI manager program in a virtual hierarchy in which a data path failure has occurred and the second function by an autonomous function and causes a state transition of a virtual function of the first function when the communication is impossible.

9. The storage device according to claim 1, wherein

the storage controller and the internal network thereof manage a plurality of tables including:

a table that manages a mapping between the plurality of root ports and the plurality of virtual functions within endpoint devices;

a table that manages a mapping of the plurality of virtual functions to the predetermined interfaces; and

a table that manages a migration of the plurality of virtual functions during failover, and

the storage controller and the internal network thereof perform a processing of switching an access source root port of the data path and a processing of switching a mapping of a virtual function of the access target endpoint device to one of the predetermined interfaces by the virtual function migration with reference to the plurality of tables when a processing of executing a failover of the data path is executed.

10. A data path failover method for an internal network of a storage controller in which components located within a storage controller are connected by PCI Express, wherein the internal network of the storage controller comprises a plurality of root ports as a whole, a processor connected to a root complex having the root ports, a plurality of endpoint devices that are provided with predetermined interfaces and have a plurality of virtual functions that can be accessed from each of the plurality of root ports, and a switch that connects the plurality of root ports and the plurality of endpoint devices, configures data paths, and has a plurality of ports;

the plurality of endpoint devices further comprises a first endpoint device comprising a front-end interface for connecting a host device, a second endpoint device comprising a back-end interface for connecting a group of storage devices, and a third endpoint device comprising a cache memory interface for connecting a cache memory,

each of the first endpoint device and the second endpoint device is capable of executing a virtual function migration of changing a mapping of one of the plurality of virtual functions to one of the predetermined interfaces; and

the switch and the first and second and third endpoint devices are so set that each of the plurality of root ports can access virtual functions located within the first and second and third endpoint devices;

the internal network of the storage controller comprising a processing step of switching an access source root port of the data path and a processing step of switching a mapping of a virtual function of the access target endpoint device to one of the predetermined interfaces by the virtual function migration in switching from a first data path in a failure state to an alternative second data path, as processing steps for executing a data path failover when a failure occurs in one data path from one of the plurality of root ports to one of the plurality of endpoint devices via the switch.

11. The data path failover method for an internal network of a storage controller according to claim 10, wherein

a single-root PCI manager program and a failover control program are provided in the internal network of the storage controller;

a multi-root PCI manager program is provided in the internal network of the storage controller or a maintenance terminal device connected to the storage controller; and

the internal network of the storage controller comprises;

a processing step in which the multi-root PCI manager program notifies the failover control program about a data path failure occurrence;

a processing step in which the failover control program refers to a failover management table located within the internal network and requests the virtual function migration to the multi-root PCI manager program on the basis of the notification information;

a processing step in which the multi-root PCI manager executes the virtual function migration in response to the request and updates a table managing a mapping of the virtual function located within the internal network to the predetermined interface; and

a processing step in which the failover control program updates the failover management table, following the execution of the virtual function migration

as a processing for executing a data path failover when a failure occurs in one data path from one of the plurality of root ports to one of the plurality of endpoint devices via the switch.

12. The data path failover method for an internal network of a storage controller according to claim 11, comprising:

a processing step in which the multi-root PCI manager program executes the virtual function migration and updates the virtual function mapping management table with respect to a data path in which the failure of the data path has not occurred; and

a processing step in which the failover control program updates the failover management table, following the execution of the virtual function migration.

13. The data path failover method for an internal network of a storage controller according to claim 10, wherein

the plurality of endpoint devices are provided with a first function for data transfer relating to data reading from the storage device or data writing thereto and a second function that can control a state transition of a virtual function associated with the first function, and

the method comprises a processing step in which the plurality of endpoint devices cause a state transition of a virtual function of the first function on the basis of communication from the single-root PCI manager program or the multi-root PCI manager program to the second function.

14. A storage device having an internal network in which components located within a storage controller are connected by PCI Express, wherein:

the storage controller and the internal network thereof comprise;

one or more root complexes that have a plurality of root ports as a whole, each root complex having one or more root ports;

one or more processors connected to each root complex;

a plurality of endpoint devices that are multi-root aware devices provided with predetermined interfaces and having a plurality of virtual functions that can be accessed from each of the plurality of root ports,

the plurality of endpoint devices further comprises a first endpoint device comprising a front-end interface for connecting a host device, a second endpoint device comprising a back-end interface for connecting a group of storage devices, and a third endpoint device comprising a cache memory interface for connecting a cache memory,

each of the first endpoint device and the second endpoint device is capable of executing a virtual function migration of changing a mapping of one of the plurality of virtual functions to one of the predetermined interfaces; and

a switch that is a multi-root aware switch that connects the plurality of root ports to the first and second and third endpoint devices, configures data paths, and has a plurality of ports,

the switch and the first and second and third endpoint devices are so set that each of the plurality of root ports can access virtual functions located within the first and second and third endpoint devices; and

when a failure occurs in one data path from one of the plurality of root ports to one device of the first and second and third endpoint devices, a processing of switching an access source root port of the data path and a processing of switching a mapping of a virtual function of the access target endpoint device to one of the predetermined interfaces by the virtual function migration in switching from a first data path in a failure state to an alternative second data path are performed as a processing for executing a data path failover.

15. The storage device according to claim 14, wherein

the third endpoint device comprises a number of mission virtual functions that will be mapped to the plurality of virtual functions, the cache memory interface is mapped to one of the number of the mission virtual functions, and the number of mission virtual functions provided in the third endpoint device is equal to or larger than the number of root ports that access the third endpoint device.

16. The storage device according to claim 15, wherein

each of the first endpoint device, second endpoint device, and third endpoint device comprises a mission virtual function,

in the storage controller and the internal network thereof,

the front-end interface that becomes the mission virtual function of the first endpoint device and the cache memory interface that becomes the mission virtual function of the third endpoint device are set to be accessible from the same root port, and

the back-end interface that becomes the mission virtual function of the second endpoint device and the cache memory interface that becomes the mission virtual function of the third endpoint device are set to be accessible from the same root port.

17. The storage device according to claim 14, wherein

each of the first endpoint device, second endpoint device, and third endpoint device comprises a mission virtual function,

each of the one or more root complexes has a first root port and a second root port, and

in the storage controller and the internal network thereof,

the first root port of each root complex is set to enable the access to the front-end interface that becomes the mission virtual function of the first endpoint device and the cache memory interface that becomes the mission virtual function of the third endpoint device, and

the second root port of each root complex is set to enable the access to the back-end interface that becomes the mission virtual function of the second endpoint device and the cache memory interface that becomes the mission virtual function of the third endpoint device.

18. The storage device according to claim 14, wherein

the plurality of endpoint devices are provided with a first function for data transfer relating to data reading from the storage device or data writing thereto and a second function that can control a state transition of a virtual function associated with the first function.

19. The storage device according to claim 18, wherein

a multi-root PCI manager program is provided in the storage controller or a maintenance terminal device connected thereto, and

in a configuration of virtual hierarchy from the root port to an endpoint device, the endpoint device causes a state transition of a virtual function of the first function by in-band communication in a virtual hierarchy in which a data path failure has not occurred from the multi-root PCI manager program to the second function.

20. The storage device according to claim 18, wherein

a multi-root PCI manager program is provided in the storage controller or a maintenance terminal device connected thereto, and

the plurality of endpoint devices cause a state transition of a virtual function of the first function by out-band communication from the multi-root PCI manager program to the second function.

21. The storage device according to claim 18, wherein

a single-root PCI manager program is provided in the storage controller, and

in a configuration of virtual hierarchy from the root port to an endpoint device,

the endpoint device determines whether or not communication is possible between the single-root PCI manager program in a virtual hierarchy in which a data path failure has occurred and the second function by an autonomous function and causes a state transition of a virtual function of the first function when the communication is impossible.

22. The storage device according to claim 14, wherein

the storage controller and the internal network thereof manage a plurality of tables including:

a table that manages a mapping between the plurality of root ports and the plurality of virtual functions within endpoint devices;

a table that manages a mapping of the plurality of virtual functions to the predetermined interfaces; and

a table that manages a migration of the plurality of virtual functions during failover, and

the storage controller and the internal network thereof perform a processing of switching an access source root port of the data path and a processing of switching a mapping of a virtual function of the access target endpoint device to one of the predetermined interfaces by the virtual function migration with reference to the plurality of tables when a processing of executing a failover of the data path is executed.