Node identification for distributed shared memory system

• US RE44,610 E1
• Filed: 05/10/2012
• Issued: 11/26/2013
• Est. Priority Date: 04/26/2007
• Status: Active Grant

• Current Assignee: Intellectual Ventures Holding 80 LLC (Intellectual Ventures LLC)
• Original Assignee: Intellectual Ventures Holding 80 LLC (Intellectual Ventures LLC)
• Inventors: Krakirian, Shahe Hagop, Akkawi, Isam
• Primary Examiner(s): HO, DUC CHI
• Application Number:
  US13/468,751
• Time in Patent Office: 565 Days
• Field of Search: 370/392, 370/389, 711/147
• US Class Current: 370/392
• CPC Class Codes: H04L 49/90 Buffering arrangements
• CPC Class Codes: H04L 49/9042 Separate storage for different parts of the packet, e.g. header and payload
• CPC Class Codes: H04L 49/9047 including multiple buffers, e.g. buffer pools

Abstract

An example embodiment of the present invention provides processes relating to a connection/communication protocol and a memory-addressing scheme for a distributed shared memory system. In the example embodiment, a logical node identifier comprises bits in the physical memory addresses used by the distributed shared memory system. Processes in the embodiment include logical node identifiers in packets which conform to the protocol and which are stored in a connection control block in local memory. By matching the logical node identifiers in a packet against the logical node identifiers in the connection control block, the processes ensure reliable delivery of packet data. Further, in the example embodiment, the logical node identifiers are used to create a virtual server consisting of multiple nodes in the distributed shared memory system.

Claims

1. A method, comprising:

receiving, at a distributed memory logic circuit of a first node, data for a packet destined to a distributed memory logic circuit of a second node, wherein the first and second nodes are connected by a network switch fabric and are parts of a distributed shared memory system, and wherein the data for the packet includes a physical memory address in which one or more bits in the physical memory address comprise a destination logical node identifier for the second node;

using the destination logical node identifier as an index into a connection control block to locate an entry for a connection between the first and second nodes, resulting in a located entry of the connection control block, wherein the connection control block is stored in a local memory on the first node;

building a the packet in a format of a connection and communication protocol using the data, the destination logical node identifier, and a logical node identifier for the first node, wherein the logical node identifier for the first node is included in the located entry of the connection control block entry;

adding, to the packet, a header that includes a switch fabric address for the second node, wherein the switch fabric address is identified in the located entry of the connection control block; and

transmitting the packet on a link to the switch fabric.

2. A method as in claim 1, wherein the distributed shared memory system is a cache coherent non-uniform memory access system.

3. A method as in claim 1, wherein the a distributed memory logic circuit in the first node sets the destination logical node identifier to zero if the destination logical node identifier in the physical memory address equals the logical node identifier for the first node.

4. A method, comprising:

receiving, at a distributed memory logic circuit of a first node, a packet from a distributed memory logic circuit of a second node, wherein the packet includes a source logical node identifier and wherein the first and second nodes are connected by a network switch fabric and are parts of a distributed shared memory system;

determining whether a destination switch fabric address included in the packet matches a switch fabric address for the first node;

using the source logical node identifier as an index into a connection control block to locate an entry for the a connection between the first and second nodes, resulting in a located entry of the connection control block, wherein the connection control block is stored in a local memory on the first node;

determining whether a destination logical node identifier included in the packet matches a logical node identifier for the first node, wherein the logical node identifier for the first node is identified in the located entry of the connection control block; and

accepting data in the packet for further processing by the first node.

5. The method of claim 4, wherein the packet is discarded if the destination switch fabric address included in the packet does not match the switch fabric address for the first node.

6. The method of claim 4, wherein the packet is discarded if the destination logical node identifier does not match the logical node identifier for the first node identified in the located entry of the connection control block.

7. The method of claim 4, wherein the distributed shared memory system is a cache coherent non-uniform memory access system.

8. A distributed memory logic circuit encoded with executable logic, the logic when executed operable to:

receive, at the distributed memory logic circuit of a first node, data for a packet destined to a distributed memory logic circuit of a second node, wherein the first and second nodes are connected by a network switch fabric and are parts of a distributed shared memory system,;

and wherein the data for the packet includes a physical memory address in which one or more bits in the physical memory address comprise a destination logical node identifier for the second node;

use the destination logical node identifier as an index into a connection control block to locate an entry for a connection between the first and second nodes, resulting in a located entry of the connection control block, wherein the connection control block is stored in a local memory on the first node;

build a the packet in a format of a connection and communication protocol using the data, the destination logical node identifier, and a logical node identifier for the first node, wherein the logical node identifier for the first node is included in the located entry of the connection control block entry;

add, to the packet, a header that includes a switch fabric address for the second node, wherein the switch fabric address is identified in the located entry of the connection control block; and

transmit the packet on a link to the switch fabric.

9. The distributed memory logic circuit of in claim 8, wherein the distributed shared memory system is a cache coherent non-uniform memory access system.

10. The distributed memory logic circuit of claim 8, wherein the distributed memory logic circuit of the first node sets logic is further operable to set the destination logical node identifier to zero if the destination logical node identifier in the physical memory address equals the logical node identifier for the first node.

11. A distributed memory logic circuit encoded with executable logic, the logic when executed operable to:

receive, at the distributed memory logic circuit of a first node, a packet from a distributed memory logic circuit of a second node, wherein the packet includes a source logical node identifier and wherein the first and second nodes are connected by a network switch fabric and are parts of a distributed shared memory system;

determine whether a destination switch fabric address included in the packet matches a switch fabric address for the first node;

use the source logical node identifier as an index into a connection control block to locate an entry for a connection between the first and second nodes, resulting in a located entry of the connection control block, wherein the connection control block is stored in a local memory on the first node;

determine whether a destination logical node identifier included in the packet matches a logical node identifier for the first node, wherein the logical node identifier for the first node is identified in the located entry of the connection control block; and

accept data in the packet for further processing by the first node.

12. The distributed memory logic circuit of claim 11, wherein the packet is discarded if the destination switch fabric address included in the packet does not match the switch fabric address for the first node.

13. The distributed memory logic circuit of claim 11, wherein the packet is discarded if the destination logical node identifier does not match the logical node identifier for the first node identified in the located entry of the connection control block.

14. The distributed memory logic circuit of claim 11, wherein the distributed shared memory system is a cache coherent non-uniform memory access system.

15. A distributed shared memory system comprising:

a network switch fabric;

two or more nodes in a distributed shared memory system connected by a the network switch fabric; and

wherein, each of the two or more nodes comprises comprising;

one or more processors,;

local memory;

and

a distributed shared memory logic circuit,

wherein the distributed memory logic circuit is encoded with executable logic, the logic that

when executed, is operable to;

receive, at the distributed memory logic circuit of a local node, data for a packet destined to a distributed memory logic circuit of a remote node of the two or more nodes in the distributed shared memory system, wherein the data for the packet includes a physical memory address in which one or more bits in the physical memory address comprise a destination logical node identifier for the remote node,

use the destination logical node identifier as an index into a connection control block to locate an entry for a connection between the local node and the remote node, resulting in a local entry of the connection control block, wherein the connection control block is stored in local memory on the local node,

build a the packet in a format of a connection and communication protocol using the data, the destination logical node identifier, and a logical node identifier for the local node, wherein the logical node identifier for the local node is included in the located entry of the connection control block entry,

add, to the packet, a header that includes a switch fabric address for the remote node, wherein the switch fabric address is identified in the located entry of the connection control block,

transmit the packet on a link to the network switch fabric, receive, at the distributed memory logic circuit of the local node, a second packet from a distributed memory logic circuit of the remote node or another remote node of the two or more nodes in the distributed shared memory system, wherein the second packet includes a source logical node identifier,

determine whether a destination switch fabric address included in the second packet matches a switch fabric address for the local node,

use the source logical node identifier as an index into the connection control block to locate an entry for a connection between the local and remote node, resulting in a second located entry of the connection control block, determine whether a destination logical node identifier included in the second packet matches a the logical node identifier for the local node, wherein the logical node identifier for the local node is identified in the second located entry of the connection control block, and

accept data in the packet for further processing by the local node.

16. A method comprising:

receiving, at a first node in a distributed shared memory system, a message from a second node in the distributed shared memory system, the distributed shared memory system comprising a plurality of interconnected nodes each having a unique logical node identifier, wherein the message indicates a memory operation related to a local memory of the first node and identifies a memory address;

if a first plurality of contiguous bits of the memory address equal a logical node identifier of the first node, changing the first plurality of contiguous bits to a predetermined value;

if the first plurality of contiguous bits of the memory address equal the predetermined value, changing the first plurality of contiguous bits to the logical node identifier of the first node; and

forwarding the message to a processor of the first node for processing.

17. The method of claim 16, wherein the predetermined value is zero.

18. The method of claim 16, wherein each node of the plurality of interconnected nodes internally accesses a respective local memory having memory addresses with a first plurality of contiguous bits set to the predetermined value.

19. The method of claim 16, wherein a given node of the plurality of interconnected nodes accesses a local memory of another node of the plurality of interconnected nodes that has a logical unit identifier equal to the predetermined value using the given node'"'"'s own respective logical node identifier for the another node.

20. The method of claim 16, wherein the memory operation is one of a read command, a write command, or a probe.

21. A method comprising:

receiving, at a first node in a distributed shared memory system, a message from a processor of the first node identifying a memory operation related to a local memory of a second node in the distributed shared memory system, the distributed shared memory system comprising a plurality of nodes each having a unique logical unit identifier, the plurality of nodes being interconnected by a switch fabric, wherein the message identifies a memory address;

if a first plurality of contiguous bits of the memory address equal a logical node identifier of the first node, changing the first plurality of contiguous bits to a predetermined value;

if the first plurality of contiguous bits of the memory address equal the predetermined value, changing the first plurality of contiguous bits to the logical node identifier of the first node; and

forwarding the message to the second node for processing.

22. A distributed shared memory system, comprising:

a network switch fabric; and

a plurality of nodes interconnected by the network switch fabric, each given node of the plurality of nodes comprising;

a logical node identifier of a plurality of contiguous bits;

a local memory;

a distributed shared memory management chip operative to share the local memory of the given node with others of the plurality of nodes in the distributed shared memory system to create a shared memory accessible using binary addresses comprising a plurality of bits, wherein a set of contiguous most-significant bits of the binary addresses collectively represent a logical node identifier of a node of the plurality of nodes; and

one or more processors each operative to access the local memory of the given node, the local memory accessed using binary addresses having the set of contiguous most-significant bits collectively set to a predetermined value,

wherein the distributed shared memory management chip is further operative to map the predetermined value to the logical node identifier of the given node in memory management traffic transmitted between the plurality of nodes that include one or more binary addresses of the shared memory.

23. The distributed shared memory system of claim 22, wherein the distributed shared memory management chip of each node of the plurality of nodes is further operative to:

if the set of contiguous most-significant bits of a given binary address equal the logical node identifier of the given node, change the set of contiguous most-significant bits of the given binary address to the predetermined value; and

if the set of contiguous most-significant bits of the given binary address equal the predetermined value, change the set of contiguous most-significant bits of the given binary address to the logical node identifier of the given node.