Providing remote, reliant and high performance PCI express device in cloud computing environments

• US 10,303,645 B2
• Filed: 08/27/2015
• Issued: 05/28/2019
• Est. Priority Date: 11/21/2014
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Franke, Hubertus, Pasetto, Davide, Penner, Hartmut
• Primary Examiner(s): Meky, Moustafa M
• Assistant Examiner(s): Mendaye, Kidest
• Application Number:
  US14/837,921
• Publication Number:
  US 20160147710A1
• Time in Patent Office: 1,370 Days
• CPC Class Codes: G06F 15/17331 Distributed shared memory [DSM], e.g. remote direct memory access [RDMA]
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A system architecture, a method, and a computer program product are disclosed for attaching remote physical devices. In one embodiment, the system architecture comprises a compute server and a device server. The compute server includes a system memory, and one or more remote device drivers; and the device server includes a system memory and one or more physical devices, and each of the physical devices includes an associated device memory. The compute server and the device server are connected through an existing network fabric that provides remote direct memory access (RDMA) services. A system mapping function logically connects one or more of the physical devices on the device server to the compute server, including mapping between the system memories and the device memories and keeping the system memories and the device memories in synchronization using the RDMA.

Claims

1. A method for attaching remote physical devices of a device server to a compute server through an existing network fabric providing remote direct memory access (RDMA) services, the compute server including a compute server system memory, and running an operating system and one or more remote device drivers;

the device server including a device server system memory and one or more physical devices, each of the physical devices including an associated device memory; and

wherein the compute server and the device server are connected through the existing network fabric, the device server directly accesses the system memory of the compute server using the RDMA services of the network fabric, and the compute server directly accesses the system memory of the device server using the RDMA services of the network fabric, the method comprising;

allocating memory space in the system memory of the compute server as memory buffer space on the compute server for data transfers to and from each of the physical devices of the device server, and wherein the device server allocates memory space in the system memory of the device server for data transfers to and from each of the physical devices of the device server; and

logically connecting one or more of the physical devices on the device server to the compute server, including

mapping, using the RDMA, the device memory associated with each of the physical devices to a copy of the associated device memory in the memory space allocated in the system memory of the compute server for said each physical device, to keep the associated device memory and the copy thereof on the compute server in synchronization, and

mapping, using the RDMA, each of the memory buffer spaces on the compute server to a copy of said each memory buffer space on the device server to keep the each memory buffer space and the copy thereof on the compute server in synchronization.

2. The method according to claim 1, wherein:

the mapping each of the device memories to a main memory copy includes shadowing and remapping the device memories to the main memory copies on the compute server to keep the device memories and the main memory copies in synchronization using the RDMA, and

the mapping each memory buffer space on the compute server to a copy on the device server includes shadowing and remapping the memory buffer spaces on the compute server to copies on the device server to keep the memory buffer spaces and the copies in synchronization using the RDMA.

3. The system architecture according to claim 1, wherein the logically connecting further includes re-mapping interrupt vector addresses.

5. The system architecture according to claim 1, wherein the physical devices on the device server are PCIe devices.

4. A method for attaching remote physical devices of a device server to a compute server through an existing network fabric providing remote direct memory access (RDMA) services, the compute server including a compute server system memory, and running an operating system and one or more remote device drivers;

the device server including a device server system memory and one or more physical devices, each of the physical devices including an associated device memory; and

wherein the compute server and the device server are connected through the existing network fabric, the device server directly accesses the system memory of the compute server using the RDMA services of the network fabric, and the compute server directly accesses the system memory of the device server using the RDMA services of the network fabric, the method comprising;

allocating memory space in the system memory of the compute server for each of the physical devices of the device server, and wherein the device server allocates memory space in the system memory of the device server for each of the physical devices of the device server; and

logically connecting one or more of the physical devices on the device server to the compute server, including mapping between the system memories and the device memories and keeping the system memories and the device memories in synchronization using the RDMA;

wherein;

the logically connecting includes

shadowing and remapping the device memories to main memory copies on the compute server and keeping the device memories and the main memory copies in synchronization using the RDMA, and

shadowing and remapping memory buffer spaces on the compute server to copies on the device server and keeping the memory buffer spaces and the copies in synchronization using the RDMA;

the shadowing and remapping the device memories includes using a device mapping component on the device server to shadow and remap each device memory to a main memory copy on the compute server and to keep the each device memory and the main memory copy in synchronization using the RDMA; and

the shadowing and remapping memory buffer spaces on the compute server includes using a client mapping component on the compute server to shadow and remap each memory buffer space on the compute server to a copy on the device server and to keep the each memory buffer space and the copy in synchronization using the RDMA.