Low latency RDMA-based distributed storage
First Claim
1. A distributed storage system, comprising:
- a plurality of networked computing devices in communication via RDMA-enabled NICs;
distribution tables and metadata hosted on one or more of the networked computing devices;
a plurality of memory regions allocated across two or more of the networked computing devices;
a separate instance of an RDMA-based messaging module hosted on each of a plurality of the networked computing devices;
a separate instance of an RDMA-based memory allocator module hosted on each of a plurality of the networked computing devices, each RDMA-based memory allocator module configured to perform allocations and de-allocations of shared memory on any of the networked computing devices; and
applying the RDMA-based memory allocator module of one or more of the networked computing devices and the RDMA-based messaging module of one or more of the networked computing devices in combination with the distribution tables and metadata to concurrently perform any combination of RDMA-based lock-free memory operations on one or more of the plurality of memory regions.
1 Assignment
0 Petitions
Accused Products
Abstract
A “Distributed Storage Controller” applies an RDMA-based memory allocator to implement a distributed thread-safe and lock-free storage system in shared memory distributed across multiple networked computers. The Distributed Storage Controller enables any number of client applications, processes, subroutines, etc., on any number of networked computers to concurrently perform self-directed lock-free allocations, de-allocations, reads, writes, etc., on the shared memory via sequences of one-way RDMA messages (e.g., RDMA reads, RDMA writes, and RDMA atomic compare and swap (CAS) operations) without requiring CPU locks. As such, the CPUs of computing devices hosting the shared memory do not need to be notified of RDMA-based reads, writes or CAS operations on that memory. Consequently, CPU utilization for computing devices hosting any portion of the shared memory is minimal, thereby enabling the Distributed Storage Controller to scale to very large numbers of concurrent accesses by very large numbers of networked computing devices.
-
Citations
20 Claims
-
1. A distributed storage system, comprising:
-
a plurality of networked computing devices in communication via RDMA-enabled NICs; distribution tables and metadata hosted on one or more of the networked computing devices; a plurality of memory regions allocated across two or more of the networked computing devices; a separate instance of an RDMA-based messaging module hosted on each of a plurality of the networked computing devices; a separate instance of an RDMA-based memory allocator module hosted on each of a plurality of the networked computing devices, each RDMA-based memory allocator module configured to perform allocations and de-allocations of shared memory on any of the networked computing devices; and applying the RDMA-based memory allocator module of one or more of the networked computing devices and the RDMA-based messaging module of one or more of the networked computing devices in combination with the distribution tables and metadata to concurrently perform any combination of RDMA-based lock-free memory operations on one or more of the plurality of memory regions. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10)
-
-
11. A computer-implemented process, comprising:
-
a shared memory comprising a plurality of memory regions allocated on two or more networked computing devices in communication via RDMA-enabled NICs; a separate instance of an RDMA-based memory allocator module hosted on each of a plurality of the networked computing devices, each RDMA-based memory allocator module configured to perform allocations and de-allocations of shared memory on any of the networked computing devices; each memory region being allocated by an instance of the RDMA-based memory allocator module hosted on one or more of the networked computing devices; one or more memory blocks being allocated from each memory region by an instance of the RDMA-based memory allocator module hosted on one or more of the networked computing devices; one or more memory slabs being allocated from each memory block by an instance of the RDMA-based memory allocator module hosted on one or more of the networked computing devices; applying one or more of the networked computing devices to host distribution tables and metadata delimiting parameters comprising references to the memory regions, memory blocks and memory slabs of the shared memory; and applying an instance of the RDMA-based messaging module hosted on one or more of the networked computing devices in combination with the distribution tables and metadata to concurrently perform any combination of RDMA-based lock-free reads and writes on one or more of the memory slabs. - View Dependent Claims (12, 13, 14, 15, 16)
-
-
17. A computer-readable storage device having computer executable instructions stored therein, said instructions causing a computing device to execute a method comprising:
-
applying an instance of an RDMA-based memory allocator to allocate a shared memory comprising memory slabs allocated from memory blocks allocated from memory regions hosted by a plurality of networked computing devices in communication via RDMA-enabled NICs, each of the plurality of networked computing devices hosting a separate instance of the RDMA-based memory allocator configured to perform allocations and de-allocations of shared memory on any of the networked computing devices; each memory slab of the shared memory further comprising a plurality of memory elements and a metadata checksum value computed from each of the memory elements of that memory slab; applying any of a plurality of the networked computing devices to perform concurrent self-directed lock-free RDMA reads of particular memory slabs of the shared memory; applying any of a plurality of the networked computing devices to perform concurrent self-directed lock-free RDMA writes to a particular memory slabs of the shared memory following application of an RDMA CAS operation on a corresponding entry in a memory slab reservation bitmap to reserve the particular memory slab; and applying an RDMA CAS operation on the corresponding entry in the reservation bitmap to release the reservation of the particular memory slab following completion of the RDMA write. - View Dependent Claims (18, 19, 20)
-
Specification