Partitioned key-value store with atomic memory operations

• US 9,563,426 B1
• Filed: 09/30/2014
• Issued: 02/07/2017
• Est. Priority Date: 12/30/2013
• Status: Active Grant

• Current Assignee: EMC Corporation (Dell Technologies Inc.), TRIAD National Security, LLC
• Original Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.), Los Alamos National Security LLC (Government of the United States of America)
• Inventors: Bent, John M., Faibish, Sorin, Grider, Gary
• Primary Examiner(s): Hussain, Tauqir
• Application Number:
  US14/502,231
• Time in Patent Office: 861 Days
• Field of Search: 709/203
• US Class Current: 1/1
• CPC Class Codes: G06F 16/9014 hash tables
• CPC Class Codes: G06F 9/3004 to perform operations on memory
• CPC Class Codes: G06F 9/52 Program synchronisation; Mutual exclusion, e.g. by means of semaphores
• CPC Class Codes: H04L 67/01 Protocols
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A partitioned key-value store is provided that supports atomic memory operations. A server performs a memory operation in a partitioned key-value store by receiving a request from an application for at least one atomic memory operation, the atomic memory operation comprising a memory address identifier; and, in response to the atomic memory operation, performing one or more of (i) reading a client-side memory location identified by the memory address identifier and storing one or more key-value pairs from the client-side memory location in a local key-value store of the server; and (ii) obtaining one or more key-value pairs from the local key-value store of the server and writing the obtained one or more key-value pairs into the client-side memory location identified by the memory address identifier. The server can perform functions obtained from a client-side memory location and return a result to the client using one or more of the atomic memory operations.

Claims

1. A method performed by a server for performing a memory operation in a partitioned key-value store, comprising:

receiving a request from an application for at least one atomic memory operation, wherein said application executes on a compute node having a client of said partitioned key-value store and said atomic memory operation comprises a memory address identifier of a memory location of said client; and

in response to said atomic memory operation, said server employing one or more remote memory operations to perform one or more of (i) reading a client-side memory location identified by said memory address identifier and storing one or more key-value pairs from said client-side memory location in a local key-value store of said server; and

(ii) obtaining one or more key-value pairs from said local key-value store of said server and writing said obtained one or more key-value pairs into said client-side memory location identified by said memory address identifier,

wherein program control returns to said application when said server receives said request from said application for said atomic memory operation such that said application performs one or more additional tasks while said server performs said one or more remote memory operations.

2. The method of claim 1, wherein said partitioned key-value store is based on a Multidimensional Data Hashing Indexing Middleware (MDHIM) framework.

3. The method of claim 1, wherein said partitioned key-value store employs Message Passing Interface (MPI) communications.

4. The method of claim 1, wherein said one or more of reading from and writing into said client-side memory location comprises a Remote Memory Access (RMA).

5. The method of claim 1, further comprising the steps of performing one or more functions obtained from said client-side memory location;

and returning a result of said one or more functions to said client using one or more of said atomic memory operations.

6. The method of claim 1, wherein said atomic memory operation comprises a list-based atomic memory operation, wherein said list-based atomic memory operation is executed for a plurality of key-value pairs in a list.

18. The method of claim 1, wherein said application queries for a status of said at least one atomic memory operation.

7. A server apparatus for performing a memory operation in a partitioned key-value store, the server apparatus comprising:

a memory; and

at least one hardware device, coupled to the memory, operative to implement the following steps;

receive a request from an application for at least one atomic memory operation, wherein said application executes on a compute node having a client of said partitioned key-value store and said atomic memory operation comprises a memory address identifier of a memory location of said client; and

in response to said atomic memory operation, said server apparatus employing one or more remote memory operations to perform one or more of (i) reading a client-side memory location identified by said memory address identifier and storing one or more key-value pairs from said client-side memory location in a local key-value store of said server; and

(ii) obtaining one or more key-value pairs from said local key-value store of said server and writing said obtained one or more key-value pairs into said client-side memory location identified by said memory address identifier,

wherein program control returns to said application when said server receives said request from said application for said atomic memory operation such that said application performs one or more additional tasks while said server performs said one or more remote memory operations.

8. The server apparatus of claim 7, wherein said partitioned key-value store is based on a Multidimensional Data Hashing Indexing Middleware (MDHIM) framework.

9. The server apparatus of claim 7, wherein said partitioned key-value store employs Message Passing Interface (MPI) communications.

10. The server apparatus of claim 7, wherein said one or more of reading from and writing into said client-side memory location comprises a Remote Memory Access (RMA).

11. The server apparatus of claim 7, further comprising the steps of performing one or more functions obtained from said client-side memory location;

and returning a result of said one or more functions to said client using one or more of said atomic memory operations.

12. The server apparatus of claim 7, wherein said atomic memory operation comprises a list-based atomic memory operation, wherein said list-based atomic memory operation is executed for a plurality of key-value pairs in a list.

19. The server apparatus of claim 7, wherein said application queries for a status of said at least one atomic memory operation.

13. An article of manufacture for performing a server-side memory operation in a partitioned key-value store, comprising a tangible machine readable recordable storage medium comprising one or more programs which when executed implement the steps of:

receiving a request from an application for at least one atomic memory operation, wherein said application executes on a compute node having a client of said partitioned key-value store and said atomic memory operation comprises a memory address identifier of a memory location of said client; and

in response to said atomic memory operation, said server employing one or more remote memory operations to perform one or more of (i) reading a client-side memory location identified by said memory address identifier and storing one or more key-value pairs from said client-side memory location in a local key-value store of said server; and

(ii) obtaining one or more key-value pairs from said local key-value store of said server and writing said obtained one or more key-value pairs into said client-side memory location identified by said memory address identifier,

wherein program control returns to said application when said server receives said request from said application for said atomic memory operation such that said application performs one or more additional tasks while said server performs said one or more remote memory operations.

14. The article of manufacture of claim 13, wherein said partitioned key-value store is based on a Multidimensional Data Hashing Indexing Middleware (MDHIM) framework.

15. The article of manufacture of claim 13, wherein said one or more of reading from and writing into said client-side memory location comprises a Remote Memory Access (RMA).

16. The article of manufacture of claim 13, further comprising the steps of performing one or more functions obtained from said client-side memory location;

and returning a result of said one or more functions to said client using one or more of said atomic memory operations.

17. The article of manufacture of claim 13, wherein said atomic memory operation comprises a list-based atomic memory operation, wherein said list-based atomic memory operation is executed for a plurality of key-value pairs in a list.

20. The article of manufacture of claim 13, wherein said application queries for a status of said at least one atomic memory operation.