Apparatus and Method for Detecting System Reconfiguration and Maintaining Persistent I/O Configuration Data in a Clustered Computer System

• US 20090198807A1
• Filed: 01/31/2008
• Published: 08/06/2009
• Est. Priority Date: 01/31/2008
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Block, Timothy Roy, Bailey, David Alan
• Granted Patent: US 7,877,471 B2
• Time in Patent Office: Days
• US Class Current: 709/223
• CPC Class Codes: H04L 12/40039 Details regarding the setting of the power status of a node according to activity on the bus
• CPC Class Codes: H04L 12/42 Loop networks

Abstract

In a clustered computer system with multiple power domains, a bus number manager within each power domain manages multiple nodes independently of other power domains. A node within a specified power domain includes a non-volatile memory that includes bus numbering information for its own buses as well as bus numbering information for two of its logically-interconnected neighbors. This creates a distributed database of the interconnection topology for each power domain. Because a node contains bus numbering information about its logical neighbor node(s), the bus numbers for the buses in the nodes are made persistent across numerous different system reconfigurations. The clustered computer system also includes a bus number manager that reads the non-volatile memories in the nodes during initial program load (i.e., boot) that reconstructs the interconnection topology from the information read from the non-volatile memories, and that assigns bus numbers to the buses according to the derived interconnection topology.

Claims

1. A clustered computer system comprising:

a first power domain manager, a second power domain manager, and a plurality of nodes all coupled together in a loop network, wherein the first power domain manager can individually power on and power off any of a first plurality of nodes in the loop network and the second power domain manager can individually power on and off any of a second plurality of nodes in the loop network; and

a bus number manager in the first power domain manager, the bus number manager determining at least one logical neighbor for each of the first plurality of nodes, the bus number manager ignoring any nodes in the loop network that are in the second power domain in determining the at least one logical neighbor for each of the first plurality of nodes, the bus number manager writing to non-volatile memory in each of the first plurality of nodes bus numbering information for at least one bus in each node and bus numbering information for at least one bus in each logical neighbor node.

2. The clustered computer system of claim 1 wherein the bus numbering information comprises a beginning bus number and a number of buses.

3. The clustered computer system of claim 1 wherein the first domain manager comprises a non-volatile memory including a bus number mask that indicates bus numbers in use in the first power domain.

4. The clustered computer system of claim 1 wherein the bus number manager detects a change in configuration of the first power domain and reads the bus numbering information from the non-volatile memory in at least one of the first plurality of nodes to determine an appropriate bus number for at least one bus in the first plurality of nodes.

5. A clustered computer system comprising:

a first power domain defined by a first power domain manager coupled to a first plurality of nodes wherein the first power domain manager can individually power on and power off any of the first plurality of nodes;

a second power domain defined by a second power domain manager coupled to a second plurality of nodes wherein the second power domain manager can individually power on and power off any of the second plurality of nodes;

wherein the first domain manager, the first plurality of nodes, the second domain manager, and the second plurality of nodes are all coupled together in a loop network, and wherein a first node in the first power domain is directly connected to a second node in the second power domain in the loop network and the second node in the second power domain is directly connected to a third node in the first power domain in the loop network;

the first node comprising;

at least one numbered bus; and

a non-volatile memory that contains;

bus numbering information for numbered buses in the first node; and

bus numbering information for numbered buses in the third node;

the third node comprising;

at least one numbered bus; and

a non-volatile memory that contains;

bus numbering information for numbered buses in the first node; and

bus numbering information for numbered buses in the third node.

6. The clustered computer system of claim 5 further comprising a bus number manager in the first power domain manager that determines the bus numbering information and writes the bus numbering information to the non-volatile memory in each of the first plurality of nodes.

7. The clustered computer system of claim 5 wherein the bus numbering information comprises a beginning bus number and a number of buses.

8. The clustered computer system of claim 5 wherein the first domain manager comprises:

a non-volatile memory including a bus number mask that indicates bus numbers in use in the first power domain; and

a bus number manager that detects a change in configuration of the first power domain and reads the bus numbering information from the non-volatile memory in the first and third nodes to determine an appropriate bus number for at least one bus in the first and third nodes.

9. The clustered computer system of claim 5 wherein the second domain manager comprises:

a non-volatile memory including a bus number mask that indicates bus numbers in use in the second power domain; and

a bus number manager that detects a change in configuration of the second power domain and reads the bus numbering information from the non-volatile memory in the second node to determine an appropriate bus number for at least one bus in the second node.

10. A computer-implemented method for storing configuration data in a clustered computer system, the method comprising the steps of:

identifying in a loop network a first plurality of nodes located within a first power domain with a first power domain manager, each of the plurality of nodes having a non-volatile memory;

assigning bus numbering information to each of the plurality of nodes to reside in each non-volatile memory;

logically connecting the plurality of nodes so that each of the plurality of nodes has at least one neighbor node, wherein the logical connection between the plurality of nodes ignores any nodes in the loop network from any power domain other than the first power domain; and

storing in each non-volatile memory bus numbering information for the at least one logical neighbor node.

11. The method of claim 10 wherein the bus numbering information comprises unique bus numbers assigned by a bus numbering manager.

12. The method of claim 10 wherein two of the plurality of nodes are connected to a third node in a different power domain in the loop network and a logical connection between the two nodes ignores the presence of the third node.

13. The method of claim 10 wherein the bus numbering information comprises a beginning bus number and a number of buses.

14. The method of claim 10 further comprising the steps of:

detecting a change in configuration of the first power domain;

reading the bus numbering information from the non-volatile memory in the first plurality of nodes; and

determining an appropriate bus number for all buses in the first plurality of nodes.

15. An article of manufacture comprising:

a bus number manager for a first power domain manager in a first power domain, wherein the first power domain manager can individually power on and power off any of a first plurality of nodes in a loop network, the bus number manager determining at least one logical neighbor for each of the first plurality of nodes, the bus number manager ignoring any nodes in the loop network that are in a different power domain than the first power domain in determining logical neighbors for each of the first plurality of nodes, the bus number manager writing to each of the first plurality of nodes bus numbering information for at least one bus in each node and bus numbering information for at least one bus in each logical neighbor node; and

recordable media bearing the bus number manager.

16. The article of manufacture of claim 15 wherein the bus numbering information comprises a beginning bus number and a number of buses.

17. The article of manufacture of claim 15 wherein the first domain manager comprises:

a non-volatile memory including a bus number mask that indicates bus numbers in use in the first power domain; and

a bus number manager that detects a change in configuration of the first power domain and reads the bus numbering information from the non-volatile memory in the first and third nodes to determine an appropriate bus number for at least one bus in the first and third nodes.

18. The article of manufacture of claim 15 wherein the second domain manager comprises:

a non-volatile memory including a bus number mask that indicates bus numbers in use in the second power domain; and

a bus number manager that detects a change in configuration of the second power domain and reads the bus numbering information from the non-volatile memory in the second node to determine an appropriate bus number for at least one bus in the second node.

19. A computer-implemented method for detecting a change in a loop network and for assigning bus numbers in the loop network corresponding to the change, the method comprising the steps of:

identifying a first plurality of nodes located within a first power domain with a first power domain manager, each of the plurality of nodes having a non-volatile memory;

assigning bus numbering information to each of the plurality of nodes to reside in each non-volatile memory;

logically connecting the plurality of nodes so that each of the plurality of nodes has a left neighbor node and a right neighbor node, wherein the logical connection between the plurality of nodes ignores any nodes from any power domain other than the first power domain;

storing in each non-volatile memory bus numbering information for the left logical neighbor node and the right logical neighbor node;

detecting the change in the loop network;

determining whether any node in the loop network includes bus numbering information that is not valid;

if the bus numbering information in a selected node is invalid, performing the steps of;

reading the non-volatile memory of at least one logical neighbor node for the selected node; and

assigning buses in the selected node bus number indicated by the non-volatile memory of the at least one logical neighbor node.

20. The method of claim 19 wherein if any new bus numbers were assigned, updating a bus number mask that reflects assigned bus numbers in the first power domain.