Hardware message queues for intra-cluster communication

• US 9,158,810 B2
• Filed: 02/27/2013
• Issued: 10/13/2015
• Est. Priority Date: 10/02/2012
• Status: Active Grant

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: Oracle International Corporation (Oracle Corporation)
• Inventors: Aingaran, Kathirgamar, Bridge, William H. Jr., Swart, Garret F., Jairath, Sumti, Johnson, John G.
• Primary Examiner(s): Mudrick, Timothy A
• Application Number:
  US13/778,307
• Publication Number:
  US 20140096145A1
• Time in Patent Office: 958 Days
• Field of Search: 719/314
• US Class Current: 1/1
• CPC Class Codes: G06F 12/0804 with main memory updating G06F12/0806 takes precedence
• CPC Class Codes: G06F 12/082 Associative directories G06F12/0822 takes precedence
• CPC Class Codes: G06F 12/12 Replacement control
• CPC Class Codes: G06F 12/1408 by using cryptography for digital transmission H04L9/00
• CPC Class Codes: G06F 12/1475 in a virtual system, e.g. with translation means
• CPC Class Codes: G06F 13/102 where the programme performs an interfacing function, e.g. device driver G06F13/105 takes precedence; scheduling within device drivers G06F9/52; contention policies within device drivers G06F9/4881
• CPC Class Codes: G06F 13/362 with centralised access control
• CPC Class Codes: G06F 13/4068 Electrical coupling
• CPC Class Codes: G06F 15/17331 Distributed shared memory [DSM], e.g. remote direct memory access [RDMA]
• CPC Class Codes: G06F 15/7889 Reconfigurable logic implemented as a co-processor instruction execution using a coprocessor G06F9/3877
• CPC Class Codes: G06F 16/2237 Vectors, bitmaps or matrices
• CPC Class Codes: G06F 16/2453 Query optimisation
• CPC Class Codes: G06F 16/24553 of query operations
• CPC Class Codes: G06F 16/24558 Binary matching operations
• CPC Class Codes: G06F 16/2456 Join operations
• CPC Class Codes: G06F 16/24561 Intermediate data storage techniques for performance improvement
• CPC Class Codes: G06F 16/24569 Query processing with adaptation to specific hardware, e.g. adapted for using GPUs or SSDs
• CPC Class Codes: G06F 16/273 Asynchronous replication or reconciliation
• CPC Class Codes: G06F 16/284 Relational databases
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06F 21/6209 : to a single file or object, e.g. in a secure envelope, encrypted and accessed using a key, or with access control rules appended to the object itself
• CPC Class Codes: G06F 9/30065 : Loop control instructions; iterative instructions, e.g. LOOP, REPEAT
• CPC Class Codes: G06F 9/381 : Loop buffering
• CPC Class Codes: G06F 9/3851 : from multiple instruction streams, e.g. multistreaming
• CPC Class Codes: G06F 9/3867 : using instruction pipelines
• CPC Class Codes: G06F 9/546 : Message passing systems or structures, e.g. queues
• CPC Class Codes: H04L 69/14 : Multichannel or multilink protocols
• CPC Class Codes: Y02D 10/00 : Energy efficient computing, e.g. low power processors, power management or thermal management

Abstract

A method and apparatus for sending and receiving messages between nodes on a compute cluster is provided. Communication between nodes on a compute cluster, which do not share physical memory, is performed by passing messages over an I/O subsystem. Typically, each node includes a synchronization mechanism, a thread ready to receive connections, and other threads to process and reassemble messages. Frequently, a separate queue is maintained in memory for each node on the I/O subsystem sending messages to the receiving node. Such overhead increases latency and limits message throughput. Due to a specialized coprocessor running on each node, messages on an I/O subsystem are sent, received, authenticated, synchronized, and reassembled at a faster rate and with lower latency. Additionally, the memory structure used may reduce memory consumption by storing messages from multiple sources in the same memory structure, eliminating the need for per-source queues.

Claims

1. A device comprising:

enqueue circuitry;

said enqueue circuitry being coupled to a memory;

said enqueue circuitry being configured to, without said enqueue circuitry executing software from said memory;

receive an enqueue command that comprises a key with an encrypted code, wherein the key identifies a queue in said memory;

authenticate the encrypted code;

enqueue a queue entry to the queue identified by the received enqueue command in response to receiving said enqueue command if the encrypted code was successfully authenticated.

2. The device of claim 1, wherein said device includes an integrated circuit chip that includes, separate from said enqueue circuitry, a processor, said processor being coupled to said enqueue circuitry.

3. The device of claim 1, wherein:

said key is an RKEY; and

said RKEY comprises the encrypted code.

4. The device of claim 1, wherein said enqueue circuitry is configured to authenticate the encrypted code based on a base secret stored a register in the enqueue circuitry.

5. The device of claim 1, wherein said enqueue circuitry being further configured to execute enqueue commands serially.

6. The device of claim 1, wherein said enqueue command includes a payload received in a plurality of chunks, and said enqueue circuitry being further configured to reassemble said plurality of chunks in response to receiving said enqueue command and said plurality of chunks.

7. The device of claim 1, wherein said enqueue command includes a payload received in a plurality of chunks, wherein said plurality of chunks were received in a different order than said plurality of chunks were sent;

and said enqueue circuitry being further configured to reassemble said plurality of chunks in response to receiving said enqueue command and said plurality of chunks.

8. A method comprising a machine-executed operation involving software instructions, wherein said software instructions which, when executed by a computing device, cause performance of steps including;

transmitting a particular enqueue command to an enqueue circuitry to enqueue a queue entry into a particular queue stored in a memory coupled to said enqueue circuitry, said particular enqueue command identifying said particular queue;

wherein said enqueue circuitry is configured to, without said enqueue circuitry executing software from said memory;

receive an enqueue command that comprises a key with an encrypted code, wherein the key identifies a queue of a plurality of queues using data that identifies said queue in said memory;

authenticate the encrypted code based on a secret value that is associated with said queue, wherein the secret value is stored in a register in the enqueue circuitry;

responsive to receipt of the enqueue command, to enqueue a queue entry to the queue identified by the received enqueue command if the encrypted code was successfully authenticated;

dequeuing a queue entry from said queue.

9. The method of claim 8, wherein said enqueue circuitry is located on an integrated circuit chip that includes, separate from said enqueue circuitry, a processor, said processor being coupled to said enqueue circuitry.

10. The method of claim 8, wherein dequeuing a queue entry from said queue comprises setting a value in said queue entry to indicate that the queue entry has been read.

11. The method of claim 8, wherein dequeuing a queue entry from said queue comprises atomically setting a pointer to point to a queue entry that has not been read.

12. The method of claim 8, further comprising receiving an interrupt in response to said queue entry being enqueued.

13. The method of claim 8, wherein dequeuing a queue entry from said queue comprises copying said queue entry in memory.

14. A system comprising:

a first enqueue circuitry;

said first enqueue circuitry being operably coupled to a first memory and a second memory;

said first enqueue circuitry being configured to, without said first enqueue circuitry executing software from said first memory and said second memory;

receive a first enqueue command, which comprises a first key with a first encrypted code, wherein said first key identifies a first queue in said first memory;

authenticate said first encrypted code based on a first secret value that is associated with said first queue, wherein the first secret value is stored in a first register in the first enqueue circuitry;

in response to authenticating said first encrypted code, enqueue a first queue entry in said first memory based on said first key identifying said first memory;

receive a second enqueue command, which comprises a second key with a second encrypted code, wherein said second key-identifies a second queue in said second memory;

authenticate said second encrypted code based on a second secret value that is associated with said second queue, wherein the second secret value is stored in a second register in the first enqueue circuitry;

in response to authenticating said second encrypted code, enqueue a second queue entry in said second memory based on said second key identifying said second memory.

15. The system of claim 14, wherein:

said first key is a first RKEY that identifies said first memory;

said second key is a second RKEY that identifies said second memory.

16. The system of claim 15, wherein said first memory and said second memory both reside on the same physical memory.

17. The system of claim 14, wherein said first enqueue circuitry is further configured to:

execute said first enqueue command and said second enqueue command serially, wherein said second enqueue command is executed after said first enqueue command.

18. The system of claim 14, further comprising:

a second enqueue circuitry;

wherein said first enqueue command further comprises a first payload;

wherein said second enqueue circuitry configured to;

disassemble said first payload into numbered chunks;

send said numbered chunks to said first enqueue circuitry;

wherein said first enqueue circuitry is further configured to;

receive said numbered chunks;

reassemble said numbered chunks;

execute said first enqueue command.