High performance recoverable communication method and apparatus for write-only networks

• US 6,049,889 A
• Filed: 01/13/1998
• Issued: 04/11/2000
• Est. Priority Date: 06/07/1995
• Status: Expired due to Term

• Current Assignee: HTC Corporation
• Original Assignee: Digital Equipment Corporation (HP Inc.)
• Inventors: Steely, Simon C. Jr., Gillett, Richard B. Jr., Garvey, Glenn P.
• Primary Examiner(s): HUA, LY
• Application Number:
  US09/006,115
• Time in Patent Office: 819 Days
• Field of Search: 395/182.02, 395/185.02, 395/182.16, 711/154, 711/141, 711/3, 711/129, 711/130, 711/144, 711/145, 711/147, 711/148, 711/153, 714/4, 714/49, 714/18
• US Class Current: 714/4.5
• CPC Class Codes: H04L 69/40 for recovering from a failure of a protocol instance or entity, e.g. service redundancy protocols, protocol state redundancy or protocol service redirection management of faults, events, alarms or notifications in data switching networks H04L41/06
• CPC Class Codes: H04L 9/40 Network security protocols

Abstract

A multi-node computer network includes a plurality of nodes coupled together via a data link. Each of the nodes includes a local memory, which further comprises a shared memory. Certain items of data that are to be shared by the nodes are stored in the shared portion of memory. Associated with each of the shared data items is a data structure. When a node sharing data with other nodes in the system seeks to modify the data, it transmits the modifications over the data link to the other nodes in the network. Each update is received in order by each node in the cluster. As part of the last transmission by the modifying node, an acknowledgement request is sent to the receiving nodes in the cluster. Each node that receives the acknowledgment request returns an acknowledgement to the sending node. The returned acknowledgement is written to the data structure associated with the shared data item. If there is an error during the transmission of the message, the receiving node does not transmit an acknowledgement, and the sending node is thereby notified that an error has occurred.

Claims

1. A method for transferring data between a plurality of nodes coupled to a common system data link comprising the steps of:

apportioning a memory at each of the plurality of nodes into local and shared portions, wherein the shared portions of each of the plurality of nodes together form a shared memory having a plurality of entries, with each entry accessible by each node of said plurality of nodes;

apportioning the local portion of the memory to provide a reflective memory portion, wherein write operations to an address of data stored in the reflective portion of local memory are broadcast on the data link;

storing a control structure in the reflective portion of the local memory of one of the nodes for each item of memory data to be shared by the one of the nodes;

the one of said plurality of nodes sharing the item of memory data updating said item of memory data by issuing to the data link a write command, the write command followed by an acknowledgment command; and

responsive to a receipt of said acknowledgment command by at least one of the other ones of the plurality of nodes sharing said item of memory data the at least one of the other ones of the plurality of nodes issuing an acknowledgment response for writing said corresponding entry in said control structure to indicate a receipt of said write command and said acknowledgment command.

2. The method according to claim 1, further comprising the step of:

responsive to one of the other ones of said plurality of nodes sharing said item of memory data incurring an error prior to said receipt of said acknowledgment command, precluding said one of the other ones of said plurality of nodes from issuing said acknowledgment response.

3. The method according to claim 1, where the control structure comprises a plurality of entries, each of said entries of the control structure comprises a valid bit and an error bit, and where said step of updating said item of memory data further comprises the step of:

each of said plurality of nodes, upon receipt of said acknowledgment command, issuing as part of said acknowledgment response a write to said control structure associated with said item of memory data to set the valid bit of said entry corresponding to said node.

4. The method according to claim 3 further comprising the step of:

responsive to an error occurring during the receipt of said acknowledgment command by one of the other ones of said plurality of nodes sharing said item of memory data, said one of the other ones of said plurality of nodes issuing a write to said corresponding entry in said control structure to set said error bit of said entry.

5. The method according to claim 1, where said acknowledgment command is issued by said one of the plurality of nodes sharing said item of data after at least one update command is issued by said one of the plurality of nodes to modify said item of data.

6. The method according to claim 1, further comprising the steps of:

assigning each of said plurality of nodes sharing the data item a unique index into said control structure;

for each node of said plurality of nodes sharing said data item, responsive to the receipt of said acknowledgment command, each of said nodes issuing said unique index as part of said acknowledgment response; and

at said one of said nodes, using said unique index from each of said acknowledgment responses to index said control structure associated with said item of memory data.

7. A computer system comprising a plurality of nodes coupled by an interconnect, wherein each of the plurality of nodes comprises:

a memory, the memory comprising;

a shared portion, wherein the shared portions of each of the plurality of nodes together forms a shared memory of the computer system, the shared memory for storing data accessible by at least two of the plurality of nodes; and

a local portion, accessible to only the associated node of the plurality of nodes, wherein the local portion includes a reflective portion, wherein writes to the reflective portion of the local portion are forwarded to other ones of the plurality of nodes over the interconnect; and

a data structure, stored in the reflective portion of the memory, the data structure associated with an item of data stored in the shared memory, the data structure comprising a number of entries corresponding to a number of the plurality of nodes sharing the item of data, wherein each of the entries includes a bit for indicating whether commands directed to the item of data have been received without error at the corresponding node.

8. The computer system according to claim 7, wherein each of the nodes of the plurality of nodes further comprises:

an adaptor, disposed between the memory and the interconnect, for transferring data between the associated node and other ones of the plurality of nodes coupled to the interconnect, wherein the adaptor further comprises;

a control table, the control table comprising a number of entries corresponding to a number of pages of data in the shared portion of memory at the associated node, each of the entries including transmit and receive control information for data at the associated page.

9. The computer system according to claim 7, wherein each entry of the control table further comprises a generate acknowledgment bit, for indicating whether writes made to the associated page of the shared portion of memory generate an acknowledgment response from a receiving node for updating the data structure.

10. A method for transferring data between a plurality of nodes coupled to an interconnect, wherein each of the plurality of nodes includes a memory apportioned into a local portion, accessible to only the associated node, and a shared portion, accessible by at least two of the plurality of nodes, the method including the steps of:

storing, in a reflective portion of the local memory of each of the plurality of nodes, a control structure for each item of data to be shared by the respective one of the plurality of nodes, the control structure including information for indicating whether commands, transferred between the respective one of the plurality of nodes and the shared memory, are received by other ones of the plurality of nodes that share the item of data; and

issuing, by one of said plurality of nodes, a command to modify data in said shared memory, wherein the command is issued over the interconnect;

issuing an acknowledgment request over the interconnect to request an acknowledgment from at least one of the plurality of nodes that share the data to indicate that the at least one node received the command.

11. The method according to claim 10 further comprising the step of:

the at least one of the plurality of nodes, in response to receipt of the acknowledgment response, writing status information in a control structure in the reflective memory of the at least one node, wherein the control structure is associated with the data; and

in response to the at least one node writing status information in the control structure of the reflective memory, forwarding the status information over the interconnect to allow the status information to be written to the control structure of the one of the plurality of nodes.