Inter-proximity communication within a rendezvous federation

• US 8,095,600 B2
• Filed: 06/30/2006
• Issued: 01/10/2012
• Est. Priority Date: 10/22/2004
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Hasha, Richard L., Xun, Lu, Kakivaya, Gopala Krishna R.
• Primary Examiner(s): Hwang, Joon H
• Assistant Examiner(s): LEE IV, THOMAS E
• Application Number:
  US11/428,133
• Publication Number:
  US 20060282547A1
• Time in Patent Office: 2,020 Days
• Field of Search: 709201-206, 709217-223, 709/227, 340 11- 161, 712/15, 370/452, 370/453, 370/454, 370/455, 370/456
• US Class Current: 709/206
• CPC Class Codes: H04L 12/4637 Interconnected ring systems
• CPC Class Codes: H04L 45/02 Topology update or discovery

Abstract

The present invention extends to methods, systems, and computer program products for facilitating inter-proximity communication within a rendezvous federation. Nodes maintain collateral ring set entry tables that include collateral rings and corresponding entry nodes into the collateral rings. Nodes can exchange collateral ring set entry state to update one another on the configuration of rings within a tree of rings. Nodes can refer to collateral ring set entry tables, as well as to other nodes, to identify entry nodes into rings that are collateral rings of the node. Messages can be sent to entry nodes in collateral rings. A message can include an indication that an entry node in a target proximity ring is to resolve the message to the node in the target proximity ring which has a node ID closest to an indicated destination node.

Claims

1. At a computer system, a method for sending communication in a federation infrastructure represented by a linked list of nodes partitioned into a hierarchical tree of rings, including a root ring and a plurality of lower sub-ring levels of sub-rings below the root ring, the plurality of lower sub-ring levels defined in accordance with a plurality of different user-defined proximity category types corresponding to each lower sub-ring level,

wherein the root ring includes all the nodes in the linked list of nodes, and the plurality of lower sub-ring levels are arranged relative to one another within the hierarchical tree in accordance with a plurality of different proximity criteria representing the different user-defined proximity category types, each lower sub-ring level including a plurality of different sub-rings, each different sub-ring representing a corresponding different equivalence class of nodes based on assigned values for one or more of the plurality of different proximity criteria for the corresponding different proximity criteria category type for the lower sub-ring level,

wherein nodes of the same equivalence class within the same lower sub-ring level have the same value for the one or more of the plurality of different proximity criteria in the same proximity category type corresponding to the same lower sub-ring level and nodes of different equivalence classes within the same lower sub-ring level have at least one different value for the plurality of different proximity criteria in the same proximity category type corresponding to the same lower sub-ring level, and

wherein nodes within the same equivalence class use intra-proximity communication to communicate between one another and nodes within different equivalence classes use inter-proximity communication to communicate between one another, the method comprising:

an act of receiving a message at a first sub-ring;

an act of determining a node having membership in the first sub-ring is to send a message to another sub-ring,

wherein the node is a member of the root ring and a plurality of sub-rings in a plurality of lower sub-ring levels based on the node matching different proximity criteria in different user-defined proximity category types, including (i) the first sub-ring representing a first equivalence class in a first proximity category type corresponding to a first lower sub-ring level, the first sub-ring being one of a plurality of sub-rings in the lower sub-ring level arranged within the tree hierarchy based on the first user-defined proximity category type and (ii) a second sub-ring representing a second equivalence class in a second proximity category type that is different from the first proximity category type, the second proximity category type corresponding to a second lower sub-ring level, the second sub-ring being one of a plurality of sub-rings in the second sub-ring level, the second sub-ring level arranged between the first lower sub-ring level and the root ring within the tree hierarchy based on the second user-defined proximity category type, the first and second sub-rings in each of the first and second sub-ring levels indicative of a first ring path forming a spine of sub-rings from the first sub-ring to the root ring based on values for one or more of the plurality of proximity criteria, the node storing separate routing information for the root ring, the first sub-ring, and a sub-ring in each of one or more other sub-ring levels including the second sub-ring, the routing information used by the node to determine membership in the root ring and in each sub-ring, and

wherein the other sub-ring represents another different equivalence class, the other sub-ring being a collateral sub-ring of at least one specified sub-ring in the ring path, the first equivalence class and the other different equivalence class having at least one differing value for a proximity criteria, collateral sub-rings comprising a specialized type of rings including peer rings of the specified sub-ring and peer rings of the specified ring'"'"'s ancestor rings, the collateral sub-ring of the specified sub-ring comprising a collateral sub-ring of nodes included in the specified sub-ring;

an act of using intra-ring communication within the first sub-ring to identify a further node with a collateral ring set having an entry node into an additional sub-ring in a second ring path,

wherein the collateral ring set comprises a set of one or more collateral sub-rings from the perspective of the specified sub-ring and comprises a specialized set of sub-rings in the tree of rings, wherein the specialized set of rings is unique for each ring, each collateral sub-ring node storing collateral sub-ring membership information and corresponding entry node information for each sub-ring, and

wherein the second ring path forms a spine of sub-rings from the other sub-ring to the root ring based on values for one or more of the plurality of proximity criteria, the additional sub-ring being at the same specified sub-ring level as one of the sub-rings in the first ring path, the specified sub-ring level being below the root ring in the hierarchical tree of rings;

an act of using inter-ring communication to send the message from the further node to the additional sub-ring at the same specified sub-ring level so as to bypass the root ring, including;

an act of the further node accessing the entry node into the additional sub-ring from the further node'"'"'s collateral ring set entry table; and

an act of the further node sending the message to the entry node into the additional sub-ring.

2. The method as recited in claim 1, wherein the act of determining a node having membership in a first sub-ring is to send a message to another sub-ring comprises an act of receiving an indication from an application related with the node.

3. The method as recited in claim 1, wherein the act of determining a node having membership in a first sub-ring is to send a message to another sub-ring comprises an act of receiving an indication from another node in the network infrastructure.

4. The method as recited in claim 1, wherein the act of determining a node having membership in a first sub-ring is to send a message to another sub-ring comprises an act of the node accessing one or more collateral sub-ring entry node items.

5. The method as recited in claim 1, wherein the act of using intra-ring communication to identify a further node with a collateral ring set entry having an entry node into an additional sub-ring in a second ring path comprises an act of identifying a collateral sub-ring entry node item identifying the additional sub-ring and a plurality of entry nodes into the additional sub-ring.

6. The method as recited in claim 5, further comprising:

an act of resolving the plurality of entry nodes into an appropriate subset of entry nodes.

7. The method as recited in claim 5, further comprising:

an act of resolving the plurality of entry nodes into a single appropriate entry node.

8. The method as recited in claim 1, wherein the plurality of different proximity criteria includes geographical boundaries, routing characteristics, and administrative domains, and wherein user-defined proximity categories are defined from a combination of values for geographical boundaries, routing characteristics, and administrative domains.

9. The method as recited in claim 1, wherein sub-rings within the federation infrastructure a partitioned by using a type proximity criteria a plurality of times.

10. The method as recited in claim 1, wherein the separate routing information stored at the node includes, for each ring:

a predecessor node to the node, the predecessor node preceding the node in a first direction within the ring;

a successor node to the node, the successor node succeeding the node in the first a second opposite direction within the ring;

appropriate neighborhood nodes of the node, the neighborhood nodes identified in both the first direction and in the second opposite direction relative to the node based on a neighborhood range and neighborhood size for the ring; and

appropriate routing nodes of the node, defined by the logarithmic index of other nodes.

11. The method as recited in claim 1, wherein the first proximity category type comprises a domain boundary, and wherein the second proximity category type comprises a non-domain boundary, such that the node is a member of the first sub-ring in the first sub-ring level based on the node matching domain boundary proximity criteria and the node is a member of the second sub-ring in the second sub-ring level based on the node matching non-domain boundary proximity criteria.

12. The method as recited in claim 11, wherein the first proximity category type comprises an administrative domain boundary, and wherein the second proximity category type comprises a jurisdiction.

13. At a computer system, a method for sending communication in a network infrastructure represented by a linked list of nodes partitioned into a hierarchical tree of rings, including a root ring and a plurality of lower sub-ring levels of sub-rings below the root ring, the plurality of lower sub-ring levels defined in accordance with a plurality of different user-defined proximity category types corresponding to each lower sub-ring level,

wherein the root ring includes all the nodes in the linked list of nodes, and the plurality of lower sub-ring levels are arranged relative to one another within the hierarchical tree in accordance with a plurality of different proximity criteria representing the different user-defined proximity category types, each lower sub-ring level including a plurality of different sub-rings, each different sub-ring representing a corresponding different equivalence class of nodes based on assigned values for one or more of the plurality of different proximity criteria for the corresponding different proximity criteria category type for the lower sub-ring level,

wherein nodes of the same equivalence class within the same lower sub-ring level have the same value for the one or more of the plurality of proximity criteria in the same proximity category type corresponding to the same lower sub-ring level and nodes of different equivalence classes within the same lower sub-ring level have at least one different value for the one or more of the plurality of different proximity criteria in the same proximity category type corresponding to the same lower sub-ring level, and

wherein nodes within the same equivalence class use intra-proximity communication to communicate between one another and nodes within different equivalence class use inter-proximity communication to communicate between one another, the method comprising:

an act of a node determining an originating node in a sub-ring that intends to route a message to a destination node closest to a given node ID within another sub-ring,

wherein the sub-ring represents one equivalence class and the other sub-ring representing a specified equivalence class, the node not being a member of the other sub-ring,

wherein inter-proximity communication is used to send the message to the specified equivalence class, the specified equivalence class identified from values for one or more of the plurality of different proximity criteria corresponding to the message,

wherein the sub-ring is included in a first ring path forming a spine of sub-rings from the sub-ring to the root ring based on values for one or more of the plurality of proximity criteria, each sub-ring in the first ring path corresponding to a different sub-ring level, including at least a first sub-ring level defined by a first proximity criteria category type and a second sub-ring level defined by a second, different, proximity criteria category type, the node storing separate routing information for the root ring, the sub-ring, and at least one other sub-ring between the sub-ring and the root ring, the routing information used by the node to determine membership in the root ring and in each sub-ring, and

wherein the sub-ring included in the first ring path comprises a collateral sub-ring, collateral sub-rings comprising a specialized type of ring comprising peer rings of the specified sub-ring and peer rings of the specified ring'"'"'s ancestor rings, the collateral sub-ring of the specified sub-ring comprises a collateral sub-ring of those nodes included in the specified sub-ring;

an act of using intra-ring communication within the sub-ring to identify a further node with a collateral ring set having at least one entry node into an additional sub-ring in a second ring path,

wherein the collateral ring set comprises a set of one or more collateral sub-rings from the perspective of the specified sub-ring, and comprises a specialized set of sub-rings in the tree of rings, wherein the specialized set of rings is unique for each ring, each collateral sub-ring node storing collateral sub-ring membership information and corresponding entry node information for each sub-ring, and

wherein the second ring path forms a spine of sub-rings from the other sub-ring to the root ring based on values for one or more of the plurality of proximity criteria, the additional sub-ring being at the same specified sub-ring level as one of the sub-rings in the first ring path, the specified sub-ring level being below the root ring in the hierarchical tree of rings; and

an act of using inter-ring communication to send the message from the further node to the additional sub-ring at the same specified sub-ring level so as to bypass the root ring, including;

an act of the further node accessing the entry node into the additional sub-ring from the further node'"'"'s collateral ring set entry table; and

an act of the further node sending the message to the entry node into the additional sub-ring, the message indicating the identified entry node is to resolve the message to the node having a node ID closest to an indicated destination node within the specified equivalence class.

14. The method as recited in claim 13, wherein the act of a node determining an originating node in a sub-ring intends to route a message to a destination node closest to a given node ID within another sub-ring comprises an act of the node accessing an indication the originating node intends to route a message to the destination node closest to the given node ID within the other sub-ring.

15. The method as recited in claim 14, wherein the act of the node determining an originating node in a sub-ring intends to route a message to a destination node closest to a given node ID within another sub-ring an act of the node receiving an indication a sending node intends to route the message to the destination node closest to the given node ID within the other sub-ring.

16. The method as recited in claim 14, wherein the act of the node determining an originating node in the sub-ring intends to route a message to a destination node closest to a given node ID within another sub-ring comprises an act of receiving an indication from an application related to the node.

17. The method as recited in claim 14, wherein the act of the node determining an originating node in the sub-ring intends to route a message to a destination node closest to a given node ID within another sub-ring comprises an act of receiving an indication in another message.

18. The method as recited in claim 17, wherein the act of receiving an indication in another message comprises an act of receiving an indication in a message from another node in the network infrastructure.

19. The method as recited in claim 13, wherein the act of using intra-ring communication to identify a further node with a collateral ring set entry table having at least one entry node into an additional sub-ring comprises an act of referring to local knowledge at the node to identify any nodes in the target proximity ring.

20. The method as recited in claim 13, wherein the act of using intra-ring communication to identify a further node with a collateral ring set entry table having at least one entry node into an additional sub-ring comprises an act of identifying an entry node is a member of the sub-ring.

21. The method as recited in claim 13, wherein the act of using intra-ring communication to identify a further node with a collateral ring set entry table having at least one entry node into an additional sub-ring comprises an act of identifying an entry node is a member of an ancestor sub-ring of the sub-ring.

22. The method as recited in claim 21, wherein the act of identifying an entry node is a member of an ancestor sub-ring of the sub-ring comprises an act of identifying an entry node is a member of an ancestor sub-ring of the sub-ring being the closest ancestor sub-ring of the sub-ring in the second ring path.

23. The method as recited in claim 13, wherein the act of using intra-ring communication to identify a further node with a collateral ring set entry table having at least one entry node into an additional sub-ring comprises:

an act of the node referring to a collateral ring set entry table to identify a node in a sending node'"'"'s collateral ring set entry table which is in an ancestor sub-ring of the sub-ring; and

an act of sending the message to the identified node in the ancestor sub-ring.

24. The method as recited in claim 23, further comprising:

an act of the node in the ancestor sub-ring referring to local knowledge to identify any nodes in the sub-ring.

25. The method as recited in claim 23, further comprising:

an act of the node in the ancestor sub-ring referring to a collateral ring set entry table to identify a further node in the identified node'"'"'s collateral ring set, in the further node in the closest ancestor sub-ring of the sub-ring; and

an act of sending the message to the further node.

26. The method as recited in claim 13, wherein the act of using intra-ring communication to identify a further node with a collateral ring set entry table having at least one entry node into an additional sub-ring comprises an act of the node utilizing an entry node directory mechanism to identify an entry node known to be a member node of the sub-ring.

27. The method as recited in claim 26, wherein the act of the node utilizing an entry node directory mechanism to identify an entry node known to be a member node of the sub-ring comprises:

an act of routing an entry node lookup request message to a rendezvous point; and

an act of receiving an entry node lookup response corresponding to the lookup request, the entry node lookup response including a plurality of entry nodes.

28. The method as recited in claim 27, wherein the act of receiving an entry node lookup response corresponding to the lookup request comprises an act of receiving an entry node lookup response from the rendezvous point.

29. The method as recited in claim 27, wherein the act of receiving an entry node lookup response corresponding to the lookup request comprises an act of receiving an entry node lookup response including one or more entry nodes registered with an entry node directory mechanism.

30. The method as recited in claim 29, wherein the act of receiving an entry node lookup response including one or more entry nodes registered with an entry node directory mechanism comprises an act of receiving an entry node lookup response including one or more entry nodes registered with the rendezvous point.

31. The method as recited in claim 29, wherein the act of receiving an entry node lookup response including one or more entry nodes registered with an entry node directory mechanism comprises an act of receiving an entry node lookup response including an entry node having registered itself with the rendezvous point.

32. The method as recited in claim 29, wherein the act of receiving an entry node lookup response including one or more entry nodes registered with an entry node directory mechanism comprises an act of receiving an entry node lookup response including an entry node registered with the entry node directory mechanism by another party.

33. The method as recited in claim 32, wherein the act of receiving an entry node lookup response including an entry node registered with the entry node directory mechanism by another party comprises an act of receiving an entry node lookup response including an entry node registered with the entry node directory mechanism by another rendezvous point.

34. The method as recited in claim 27, further comprising:

an act of receiving an entry node registration request for an entry node; and

an act of registering the entry node with the rendezvous point.

35. The method as recited in claim 13, wherein the separate routing information stored at the node includes, for each ring:

a predecessor node to the node, the predecessor node preceding the node in a first direction within the ring;

a successor node to the node, the successor node succeeding the node in the first a second opposite direction within the ring;

appropriate neighborhood nodes of the node, the neighborhood nodes identified in both the first direction and in the second opposite direction relative to the node based on a neighborhood range and neighborhood size for the ring; and

appropriate routing nodes of the node, defined by the logarithmic index of other nodes.

36. The method as recited in claim 13, wherein the first proximity category type comprises a domain boundary, and wherein the second proximity category type comprises a non-domain boundary, such that the node is a member of the first sub-ring in the first sub-ring level based on the node matching domain boundary proximity criteria and the node is a member of the second sub-ring in the second sub-ring level based on the node matching non-domain boundary proximity criteria.

37. A computer program product for use at a computer system, the computer program product for implementing a method for sending inter-proximity communication in a federation infrastructure represented by a linked list of nodes partitioned into a hierarchical tree of rings, including a root ring and a plurality of lower sub-ring levels of sub-rings below the root ring, the plurality of lower sub-ring levels defined in accordance with a plurality of different proximity category types corresponding to each lower sub-ring level,

wherein the root ring includes all the nodes in the linked list of nodes, and the plurality of lower sub-ring levels are arranged relative to one another within the hierarchical tree in accordance with a plurality of different proximity criteria representing the different user-defined proximity category types, each lower sub-ring level including a plurality of different sub-rings, each different sub-ring representing a corresponding different equivalence class of nodes based on assigned values for one or more of the plurality of different proximity criteria for the corresponding different proximity criteria category type for the lower sub-ring level,

wherein nodes of the same equivalence class within the same lower sub-ring level have the same value for the plurality of proximity criteria in the same proximity category type corresponding to the same lower sub-ring level and nodes of different equivalence classes within the same lower sub-ring level have at least one different value for the plurality of proximity criteria in the same proximity category type corresponding to the same lower sub-ring level, and

wherein nodes within the same equivalence class use intra-proximity communication to communicate between one another and nodes within different equivalence classes use inter-proximity communication to communicate between one another, the computer program product comprising one or more computer storage devices having stored thereon computer-executable instructions that, when executed by a processor, cause a receiving node to perform the following:

receive a message at a first sub-ring;

determine a node having membership in the first sub-ring is to send a message to another sub-ring,

wherein the node is a member of the root ring and a plurality of sub-rings in a plurality of lower sub-ring levels based on the node matching different proximity criteria in different user-defined proximity category types, including (i) the first sub-ring representing a first equivalence class in a first proximity category type corresponding to a first lower sub-ring level, the first sub-ring being one of a plurality of sub-rings in the lower sub-ring level arranged within the tree hierarchy based on the first user defined proximity category type and (ii) a second sub-ring representing a second equivalence class in a second proximity category type that is different from the first proximity category type, the second proximity category type corresponding to a second lower sub-ring level, the second sub-ring being one of a plurality of sub-rings in the second sub-ring level, the second sub-ring level arranged between the first lower sub-ring level and the root ring within the tree hierarchy based on the second user-defined proximity category type, the first and second sub-rings in each of the one or more other sub-ring levels indicative of a first ring path forming a spine of sub-rings from the first sub-ring to the root ring based on values for one or more of the plurality of proximity criteria, the node storing separate routing information for the root ring, the first sub-ring, and a sub-ring in each of one or more other sub-ring levels including the second sub-ring, the routing information used by the node to determine membership in the root ring and in each sub-ring, and

wherein the other sub-ring represents another different equivalence class, the other sub-ring being a collateral sub-ring of at least one specified sub-ring in the ring path, the first equivalence class and the other different equivalence class having at least one differing value for a proximity criteria, collateral sub-rings comprising a specialized type of rings including peer rings of the specified sub-ring and peer rings of the specified ring'"'"'s ancestor rings, the collateral sub-ring of the specified sub-ring comprising a collateral sub-ring of nodes included in the specified sub-ring;

use intra-ring communication within the first sub-ring to identify a further node with a collateral ring set having an entry node into an additional sub-ring in a second ring path,

wherein the collateral ring set comprises a set of one or more collateral sub-rings from the perspective of the specified sub-ring and comprises a specialized set of sub-rings in the tree of rings, wherein the specialized set of rings is unique for each ring, each collateral sub-ring node storing collateral sub-ring membership information and corresponding entry node information for each sub-ring, and

wherein the second ring path forms a spine of sub-rings from the other sub-ring to the root ring based on values for one or more of the plurality of proximity criteria, the additional sub-ring being at the same specified sub-ring level as one of the sub-rings in the first ring path, the specified sub-ring level being below the root ring in the hierarchical tree of rings;

use inter-ring communication to send the message from the further node to the additional sub-ring at the same specified sub-ring level so as to bypass the root ring, including;

a further node accessing the entry node into the additional sub-ring from the further node'"'"'s collateral ring set entry table; and

the further node sending the message to entry node into the additional sub-ring.

38. The computer program product as recited in claim 37, wherein the computer-readable storage media comprise system memory.

39. The computer program product as recited in claim 37, wherein the computer-readable storage media comprise a magnetic disk.

40. A computer program product for use at a computer system, the computer program product for implementing a method for sending inter-proximity communication in a federation infrastructure represented by a linked list of nodes partitioned into a hierarchical tree of rings, including a root ring and a plurality of lower sub-ring levels of sub-rings below the root ring, the plurality of lower sub-ring levels define in accordance with a plurality of different user-defined proximity category types corresponding to each lower sub-ring level,

wherein the root ring includes all the nodes in the linked list of nodes, and the plurality of lower sub-ring levels are arranged relative to one another within the hierarchical tree in accordance with a plurality of different proximity criteria representing the different user-defined proximity category types, each lower sub-ring level including a plurality of different sub-rings, each different sub-ring representing a corresponding different equivalence class of nodes based on assigned values for one or more of the plurality of different proximity criteria for the corresponding different proximity criteria category type for the lower sub-ring level,

wherein nodes of the same equivalence class within the same lower sub-ring level have the same value for the one or more of the plurality of different proximity criteria in the same proximity category type corresponding to the same lower sub-ring level and nodes of different equivalence classes within the same lower sub-ring level have at least one different value for the plurality of different proximity criteria in the same proximity category type corresponding to the same lower sub-ring level, and

wherein nodes within the same equivalence claims use intra-proximity communication to communicate between one another and nodes within different proximity classes use inter-proximity communication to communicate between one another, the computer program product comprising one or more computer storage devices having stored thereon computer-executable instruction that, when executed by a processor, cause a node to perform the following:

determine an originating node in a sub-ring intends to route a message to a destination node closest to a given node ID within another sub-ring,

wherein the sub-ring represents one equivalence class and the other sub-ring representing a specified equivalence class, the node not being a member of the other sub-ring,

wherein inter-proximity communication is used to send the message to the specified equivalence class, the specified equivalence class identified from values of one or more of the plurality of different proximity criteria corresponding to the message,

wherein the sub-ring is included in a first ring path forming a spine of sub-rings from the sub-ring to the root ring based on values for one or more of the plurality of proximity criteria, each sub-ring in the first ring path corresponding to a different sub-ring level, including at least a first sub-ring level defined by a first proximity criteria category type and a second sub-ring level defined by a second, different, proximity criteria category type, the node storing separate routing information for the root ring, the sub-ring, and at least one other sub-ring between the sub-ring and the root ring, the routing information used by the node to determine membership in the root ring and in each sub-ring, and

wherein the sub-ring included in the first ring path comprises a collateral sub-ring collateral sub-rings comprising a specialized type of ring comprising peer rings of the specified sub-ring and peer rings of the specified ring'"'"'s ancestor rings, the collateral sub-ring of the specified sub-ring comprises a collateral sub-ring of those nodes included in the specified sub-ring;

using intra-ring communication within the sub-ring to identify a further node with a collateral ring set having at least one entry node into an additional sub-ring in a second ring path,

wherein the collateral ring set comprises a set of one or more collateral sub-rings from the perspective of the specified sub-ring, and comprises a specialized set of sub-rings in the tree of rings, wherein the specialized set of rings is unique for each ring, each collateral sub-ring node storing collateral sub-ring membership information and corresponding entry node information for each sub-ring, and

wherein the second ring path forms a spine of sub-rings from the other sub-ring to the root ring based on values for one or more of the plurality of proximity criteria, the additional sub-ring being at the same specified sub-ring level as one of the sub-rings in the first ring path, the specified sub-ring level being below the root ring in the hierarchical tree of rings; and

using inter-ring communication to send the message from the further node to the additional sub-ring at the same specified sub-ring level so as to bypass the root ring, including;

access, at the further node, the entry node into the additional sub-ring from the further node'"'"'s collateral ring set entry table; and

send the message from the further node to the entry node into the additional sub-ring, the message indicating the identified entry node is to resolve the message to the node which has a node ID closest to an indicated destination node within the specified equivalence class.

41. The computer program product as recited in claim 40, wherein the computer-readable storage media comprise system memory.

42. The computer program product as recited in claim 40, wherein the computer-readable storage media comprise a magnetic disk.