Lockless distributed redundant storage and NVRAM cache in a highly-distributed shared topology with direct memory access capable interconnect
First Claim
1. A method for data storage, comprising:
- in a system that comprises at least a storage controller, multiple servers and multiple multi-queue storage devices that communicate over a network, maintaining in a memory of the storage controller a data structure, which is shared among the servers and modified by the servers using remote direct memory access to the memory of the storage controller, and which specifies storage addresses of data blocks accessed by the servers;
sending a plurality of the data blocks from a server to a Non-Volatile Random-Access Memory (NVRAM) cache using remote direct memory access, so as to accumulate one or more stripes of the data blocks in the NVRAM cache;
transferring the stripes from the NVRAM cache to respective server-specific queues on the storage devices; and
preventing other servers from causing inconsistency in the data blocks during accumulation of the data blocks in the NVRAM cache or during transferal of the stripes from the NVRAM cache to the storage devices, by querying the data structure over the network using the remote direct memory access and verifying that, in accordance with the data structure, the storage addresses of the data blocks were not modified by another server during the accumulation of the data blocks or during the transferal of the stripes.
2 Assignments
0 Petitions
Accused Products
Abstract
A method for data storage includes, in a system that includes multiple servers and multiple multi-queue storage devices that communicate over a network, accumulating data blocks by a server in a Non-Volatile Random-Access Memory (NVRAM) cache that is accessible to the servers and to the storage devices, by sending the data blocks from the server to the NVRAM cache using remote direct memory access, so as to form one or more stripes. The stripes are transferred from the NVRAM cache to respective server-specific queues on the storage devices. Another server is prevented from causing inconsistency in the data blocks, by querying a data structure, which is shared and modified by the servers using remote direct memory access, and which specifies storage locations of the data blocks.
127 Citations
25 Claims
-
1. A method for data storage, comprising:
-
in a system that comprises at least a storage controller, multiple servers and multiple multi-queue storage devices that communicate over a network, maintaining in a memory of the storage controller a data structure, which is shared among the servers and modified by the servers using remote direct memory access to the memory of the storage controller, and which specifies storage addresses of data blocks accessed by the servers; sending a plurality of the data blocks from a server to a Non-Volatile Random-Access Memory (NVRAM) cache using remote direct memory access, so as to accumulate one or more stripes of the data blocks in the NVRAM cache; transferring the stripes from the NVRAM cache to respective server-specific queues on the storage devices; and preventing other servers from causing inconsistency in the data blocks during accumulation of the data blocks in the NVRAM cache or during transferal of the stripes from the NVRAM cache to the storage devices, by querying the data structure over the network using the remote direct memory access and verifying that, in accordance with the data structure, the storage addresses of the data blocks were not modified by another server during the accumulation of the data blocks or during the transferal of the stripes. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12)
-
-
13. A computing system, comprising:
-
multiple servers; a storage controller, configured to maintain in a memory thereof a data structure, which is shared among the servers and modified by the servers using remote direct memory access to the memory of the storage controller, and which specifies storage addresses of data blocks accessed by the servers; and multiple multi-queue storage devices, wherein the servers are configured to communicate with the storage controller and the storage devices over a network, and wherein a server among the servers is configured to; send a plurality of the data blocks to a Non-Volatile Random-Access Memory (NVRAM) cache using remote direct memory access, so as to accumulate one or more stripes of the data blocks in the NVRAM cache; transfer the stripes from the NVRAM cache to respective server-specific queues on the storage devices; and prevent another servers from causing inconsistency in the data blocks during accumulation of the data blocks in the NVRAM cache or during transferal of the stripes from the NVRAM cache to the storage devices, by querying the data structure over the network using the remote direct memory access and verifying that, in accordance with the data structure, the storage addresses of the data blocks were not modified by another server during the accumulation of the data blocks or during the transferal of the stripes. - View Dependent Claims (14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24)
-
-
25. A server in a system that comprises multiple servers, the server comprising:
-
a network interface for communicating over a network with multiple multi-queue storage devices and with a storage controller, wherein the storage controller maintains in a memory thereof a data structure, which is shared among the servers and modified by the servers using remote direct memory access to the memory of the storage controller, and which specifies storage addresses of data blocks accessed by the servers; and a processor, which is configured to; send a plurality of the data blocks to a Non-Volatile Random-Access Memory (NVRAM) cache using remote direct memory access, so as to accumulate one or more stripes of the data blocks in the NVRAM cache; transfer the stripes from the NVRAM cache to respective server-specific queues on the storage devices; and prevent other servers from causing inconsistency in the data blocks during accumulation of the data blocks in the NVRAM cache or during transferal of the stripes from the NVRAM cache to the storage devices, by querying data structure over the network using the remote direct memory access and verifying that, in accordance with the data structure, the storage addresses of the data blocks were not modified by another server during the accumulation of the data blocks or during the transferal of the stripes.
-
Specification