Distributed queue pair state on a host channel adapter

• US 9,384,072 B2
• Filed: 12/20/2012
• Issued: 07/05/2016
• Est. Priority Date: 12/20/2012
• Status: Active Grant

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: Oracle International Corporation (Oracle Corporation)
• Inventors: Manula, Brian Edward, Sandven, Magne Vigulf
• Primary Examiner(s): Maung, Zarni
• Application Number:
  US13/721,671
• Publication Number:
  US 20140181232A1
• Time in Patent Office: 1,293 Days
• Field of Search: 709213-216, 709223-229, 709/250, 709/231, 711/118
• US Class Current: 1/1
• CPC Class Codes: G06F 12/0833 in combination with broadcast means (e.g. for invalidation or updating)
• CPC Class Codes: G06F 12/0837 with software control, e.g. non-cacheable data
• CPC Class Codes: G06F 2209/548 Queue
• CPC Class Codes: G06F 2212/6046 Using a specific cache allocation policy other than replacement policy
• CPC Class Codes: G06F 9/4881 Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
• CPC Class Codes: G06F 9/546 Message passing systems or structures, e.g. queues
• CPC Class Codes: H04L 65/1101 Session protocols
• CPC Class Codes: H04L 67/01 Protocols
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 67/568 Storing data temporarily at an intermediate stage, e.g. caching
• CPC Class Codes: H04L 69/329 in the application layer [OSI layer 7]

Abstract

A method for managing a distributed cache of a host channel adapter (HCA) that includes receiving a work request including a QP number, determining that a QP state identified by the QP number is not in the distributed cache, retrieving the QP state from main memory, and identifying a first portion and a second portion of the QP state. The method further includes storing the first portion into a first entry of a first sub-cache block associated with the first module, where the first entry is identified by a QP index number, storing the second portion into a second entry of a second sub-cache block associated with the second module, where the second entry is identified by the QP index number; and returning the QP index number of the QP state to the first module and the second module.

Claims

1. A method for managing a distributed cache of a host channel adapter (HCA), comprising:

receiving a work request comprising a queue pair (QP) number;

determining, by a cache control module of the HCA, that a QP state identified by the QP number is not in the distributed cache of the HCA,

wherein the QP state comprises a plurality of data fields, and

wherein the distributed cache comprises a plurality of sub-cache blocks;

retrieving the QP state from main memory;

identifying a first portion and a second portion of the QP state,

wherein the first portion comprises a first subset of the plurality of data fields required by a first module of the HCA, and

wherein the second portion comprises a second subset of the plurality of data fields required by a second module of the HCA;

storing the first portion into a first entry of a first sub-cache block associated with the first module, wherein the first entry is identified by a QP index number, wherein the first sub-cache block is one of the plurality of sub-cache blocks;

storing the second portion into a second entry of a second sub-cache block associated with the second module, wherein the second entry is identified by the QP index number,

wherein the second sub-cache block is one of the plurality of sub-cache blocks; and

returning the QP index number of the QP state to the first module and the second module.

2. The method of claim 1, wherein determining that the QP state is not in the distributed cache comprises:

attempting to retrieve, from a QP index table, an index entry associated with the QP state, wherein the index entry comprises the QP index number of each entry in the plurality of sub-cache blocks in which the QP state is located within the distributed cache;

determining that the index entry is not present within the QP index table.

3. The method of claim 2, wherein retrieving the QP state from main memory further comprises:

obtaining a location of the QP state in the main memory from an address table;

reading the QP state from the main memory.

4. The method of claim 2, further comprising:

selecting, based on a reference counter in the QP index table, a resident QP state to evict, wherein the reference counter represents a number of pending work requests to evict; and

evicting the resident QP state from the distributed cache.

5. The method of claim 1, wherein the plurality of data fields of the QP state comprises a private field, configured to be read and written by the first module, and wherein the private field is stored exclusively in the first entry.

6. The method of claim 1, wherein the plurality of data fields of the QP state comprises a read-only field, configured to be read by both the first module and the second module, and wherein a first copy of the read-only field is stored in the first entry and a second copy of the read-only field is stored in the second entry.

7. The method of claim 1, wherein the plurality of data fields of the QP state comprises a broadcast field, configured to be read by the second module and written by the first module, and wherein a first copy of the broadcast field is stored in the first entry and a second copy of the broadcast field is stored in the second entry.

8. The method of claim 7, further comprising:

broadcasting, by the first module, a change made to the first copy of the broadcast field;

receiving, by the second module, the change; and

altering, by the second module, the second copy of the broadcast field to reflect the change.

9. The method of claim 1, wherein the plurality of data fields of the QP state comprises a public field, configured to be read and written by the first module and the second module, wherein only one module can write the public field at a given time.

10. The method of claim 9, wherein the public field is stored in a central cache block of the HCA.

11. The method of claim 9, further comprising:

determining that the public field may be split into a plurality of non-public data fields;

partitioning the public field into the plurality of non-public data fields; and

storing the plurality of non-public data fields in the first entry and the second entry.