Fabric independent PCIe cluster manager

• US 9,086,919 B2
• Filed: 08/23/2012
• Issued: 07/21/2015
• Est. Priority Date: 08/23/2012
• Status: Active Grant

• Current Assignee: Dell Products LP (Dell Technologies Inc.)
• Original Assignee: Dell Products LP (Dell Technologies Inc.)
• Inventors: Iyer, Shyamkumar, Domsch, Matthew L.
• Primary Examiner(s): Dang, Khanh
• Application Number:
  US13/593,234
• Publication Number:
  US 20140059265A1
• Time in Patent Office: 1,062 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 12/00 Accessing, addressing or allocating within memory systems or architectures digital input from, or digital output to record carriers, e.g. to disk storage units, G06F3/06
• CPC Class Codes: G06F 13/20 for access to input/output bus
• CPC Class Codes: G06F 15/00 Digital computers in general details G06F1/00 – G06F13/00; Data processing equipment in general
• CPC Class Codes: G06F 2213/0026 PCI express
• CPC Class Codes: G06F 9/46 Multiprogramming arrangements
• CPC Class Codes: G06F 9/5011 the resources being hardware resources other than CPUs, Servers and Terminals

Abstract

A cluster manager of a computer cluster determines an allocation of resources from the endpoints for running applications on the nodes of the computer cluster and configures the computer cluster to provide resources for the applications in accordance with the allocation. The cluster may include a Peripheral Component Interconnect express (PCIe) fabric. The cluster manager may configure PCIe multi-root input/output (I/O) virtualization topologies of the computer cluster. The allocations may satisfy Quality of Service requirements, including priority class and maximum latency requirements. The allocations may involve splitting I/O traffic.

Claims

1. A cluster manager of a node of a computer cluster, the computer cluster having endpoints connected to the computer cluster by a plurality of interconnects, the cluster manager comprising:

a communication module to receive specifications of resource requirements of applications to run on the computer cluster, the resource requirements including storage and network connectivity requirements; and

a configuration module to determine resources available from the endpoints of the computer cluster, to determine allocations of resources from the endpoints to satisfy the resource requirements of the applications, and to configure the computer cluster to provide resources for the applications in accordance with the allocations, the configuring including configuring the interconnects to provide the allocated resources to the applications, wherein;

the specifications of resource requirements includes a specification of Quality of Service (QoS) requirements, including priority class and a maximum latency requirements; and

the configuration module is to determine an allocation of resources to satisfy the QoS requirements, including the priority class and maximum latency requirements, the determining comprising;

selecting classes of traffic of the applications; and

mapping the classes of traffic to Data Center Bridging (DCB) priority classes based upon priorities of the classes of traffic.

2. The cluster manager of claim 1, wherein:

the computer cluster comprises a Peripheral Component Interconnect express (PCIe) fabric; and

the configuration module is to allocate PCIe lanes from the PCIe fabric for interprocess communications and cluster service traffic.

3. The cluster manager of claim 2, wherein the configuration module is to allocate resources of virtual functions (VF) from the endpoints to the applications.

4. The cluster manager of claim 1, wherein the endpoints communicate with the computer cluster via at least one of the following communications protocols:

InfiniBand;

Fibre Channel;

Internet Small Computer System Interface (iSCSI); and

Ethernet.

5. The cluster manager of claim 1, wherein:

the computer cluster comprises a plurality of nodes;

the determination of allocations of resources is to include a first allocation of resources for a first application to run on a first node of the nodes and a second allocation of resources for a second application to run on a second node of the nodes; and

the configuration is to include setting I/O configuration parameters for the second node by the configuration module.

6. A method comprising:

receiving at a cluster manager of a node of a computer cluster a specification of resource requirements of a first application to run on the computer cluster, the resource requirements including storage and network connectivity bandwidth requirements;

determining resources available from endpoints of the computer cluster, wherein the endpoints are connected to the computer cluster by a plurality of interconnects;

determining of a first allocation of resources from the endpoints to satisfy the resource requirements of the first application;

determining by the cluster manager a second allocation of resources from the endpoints for a second application, the second application to run on another node of the computer cluster;

configuring the computer cluster to provide resources for the first application in accordance with the allocation, including configuring the interconnects to provide the allocated resources to the application; and

setting I/O configuration parameters for the other node of the computer cluster by the cluster manager.

7. The method of claim 6, wherein:

the specification of resource requirements includes a specification of Quality of Service (QoS) requirements; and

the determining the allocation of resources comprises determining an allocation of resources to satisfy the QoS requirements.

8. The method of claim 7, wherein:

the QoS requirements comprise priority class and maximum latency requirements, and

the determining the allocation of resources comprises determining the allocation of resources to satisfy the priority class and maximum latency requirements.

9. The method of claim 8, wherein the determining the allocation of resources comprises:

selecting classes of traffic of the application; and

mapping the classes of traffic to Data Center Bridging (DCB) priority classes based upon priorities of the classes of traffic.

10. The method of claim 7, wherein the QoS requirements include Ethernet DCB QoS guarantees and InfiniBand latency limits.

11. The method of claim 6, wherein:

the computer cluster comprises a Peripheral Component Interconnect express (PCIe) fabric; and

the configuring comprises allocating PCIe lanes from the PCIe fabric for interprocess communications and cluster service traffic.

12. The method of claim 6, wherein the configuring comprises allocating resources from virtual functions (VF) of the endpoints to the application.

13. The method of claim 6, wherein the configuring comprises configuring multiqueue input/output (I/O) adapters with PCIe bandwidth allocated for the application.

14. The method of claim 6, wherein:

the computer cluster comprises a multi-root I/O virtualization (MR-IOV) PCIe fabric;

the endpoints comprise PCIe devices connected to multi-root aware (MRA) switches of the PCIe fabric; and

the determining resources available comprises determining the characteristics of the PCIe devices based upon their connections to the MRA switches.

15. The method of claim 14, wherein the configuring comprises configuring PCIe MR-IOV topologies of the computer cluster.

16. The method of claim 6, wherein the endpoints communicate with the computer cluster via at least one of the following communications protocols:

InfiniBand;

Fibre Channel;

Internet Small Computer System Interface (iSCSI); and

Ethernet.

17. The method of claim 6, wherein the configuring comprises allocating a portion of the capacity of an endpoint to the application and allocating another portion of the capacity of the endpoint to another application running on another node of the computer cluster.

18. A method comprising:

determining by a cluster manager of a node of a computer cluster resources available from endpoints of the computer cluster, wherein;

the computer cluster comprises a plurality of nodes connected by a fabric; and

the endpoints are connected to the computer cluster by a plurality of interconnects, wherein the endpoints communicate with the computer cluster via at least one of the following communications protocols;

InfiniBand;

Fibre Channel;

Internet Small Computer System Interface (iSCSI); and

Ethernet;

determining an allocation of resources from the endpoints for running applications on the nodes of the computer cluster; and

configuring the computer cluster to provide resources for the applications in accordance with the allocation, the configuring including;

configuring the interconnects to provide the allocated resources to the applications; and

setting input/output (I/O) configuration parameters for another node of the computer cluster.

19. The method of claim 18, wherein the determining the allocation of resources comprises determining the allocation of resources to satisfy Quality of Service (QoS) requirements on the applications, including priority class and maximum latency requirements, the determining comprising:

selecting classes of traffic of the applications; and

mapping the classes of traffic to Data Center Bridging (DCB) priority classes based upon priorities of the classes of traffic.

20. The method of claim 18, wherein:

the computer cluster comprises a multi-root IO virtualization (MR-IOV) PCIe fabric;

the endpoints comprise PCIe devices connected to multi-root aware (MRA) switches of the PCIe fabric;

the determining the resources available comprises determining the characteristics of the PCIe devices based upon their connections to the MRA switches; and

the configuring comprises;

configuring PCIe express MR-IOV topologies of the computer cluster;

allocating PCIe lanes from the MR-IOV PCIe fabric for interprocess communications and cluster service traffic; and

allocating resources of virtual functions (VF) of the endpoints to the applications.