System and method for a hierarchical system management architecture of a highly scalable computing system

• US 6,845,410 B1
• Filed: 03/10/2000
• Issued: 01/18/2005
• Est. Priority Date: 09/29/1999
• Status: Expired due to Term

• Current Assignee: Hewlett Packard Enterprise Development LP (Hewlett-Packard Enterprise Company)
• Original Assignee: Silicon Graphics Incorporated (Graphics Properties Holdings, Inc.)
• Inventors: Brown, Michael, Ingebritson, Richard T., Hein, Steven, Gustafson, Kim, Deneroff, Martin M., Cutler, Robert
• Primary Examiner(s): Fleming, Fritz
• Assistant Examiner(s): KNAPP, JUSTIN R
• Application Number:
  US09/522,695
• Time in Patent Office: 1,775 Days
• Field of Search: 710/1, 710/18, 710/7, 710/29, 710/36, 710/41, 710/100, 700/2, 700/20, 700/108, 700/83, 712/225
• US Class Current: 710/29
• CPC Class Codes: G06F 11/30 Monitoring
• CPC Class Codes: G06F 11/3006 where the computing system is distributed, e.g. networked systems, clusters, multiprocessor systems multiprogramming arrangements G06F9/46; allocation of resources G06F9/50
• CPC Class Codes: G06F 11/3048 where the topology of the computing system or computing system component explicitly influences the monitoring activity, e.g. serial, hierarchical systems
• CPC Class Codes: G06F 11/3051 Monitoring arrangements for monitoring the configuration of the computing system or of the computing system component, e.g. monitoring the presence of processing resources, peripherals, I/O links, software programs verification or detection of system hardware configuration G06F11/2247
• CPC Class Codes: G06F 11/3055 Monitoring arrangements for monitoring the status of the computing system or of the computing system component, e.g. monitoring if the computing system is on, off, available, not available error or fault processing without redundancy G06F11/0703; error detection or correction by redundancy in data representation G06F11/08; error detection or correction by redundancy in operation G06F11/14; error detection or correction by redundancy in hardware G06F11/16
• CPC Class Codes: G06F 11/3058 Monitoring arrangements for monitoring environmental properties or parameters of the computing system or of the computing system component, e.g. monitoring of power, currents, temperature, humidity, position, vibrations thermal management in cooling arrangements of a computing system G06F1/206

Abstract

A modular computer system includes at least two processing functional modules each including a processing unit adapted to process data and adapted to input/output data to other functional modules through at least two ports with each port including a plurality of data lines. At least one routing functional module is adapted to route data and adapted to input/output data to other functional modules through at least two ports with each port including a plurality of data lines. At least one input or output functional module is adapted to input or output data and adapted to input/output data to other functional modules through at least one port including a plurality of data lines. Each processing, routing and input or output functional module includes a local controller adapted to control the local operation of the associated functional module, wherein the local controller is adapted to input and output control information over control lines connected to the respective ports of its functional module. At least one system controller functional module is adapted to communicate with one or more local controllers and provide control at a level above the local controllers. Each of the functional modules adapted to be cabled together with a single cable that includes a plurality of data lines and control lines such that control lines in each module are connected together and data lines in each unit are connected together. Each of the local controllers adapted to detect other local controllers to which it is connected and to thereby collectively determine the overall configuration of a system.

Claims

1. In a computing system having a plurality of interconnected computer units, wherein each computer unit includes a processor and a level one controller, a method of controlling individual computing units, the method comprising:

connecting operably each level one controllers to a level two controller;

connecting operably each level two controllers to a level three controller;

sending a request from one of the controllers to a controller at a different level;

receiving a response at the one of the controllers from the controller at a different level;

sending an event message from one of the controllers to a controller at a different level;

receiving the event message at the controller at a different level yielding a received event message;

performing a pre-determined action by the controller at a different level in response to the received event message; and

sending a request from one of the controllers to a controller at the same level.

2. The method of controlling individual computing units as in claim 1, wherein sending an event message comprises sending an event message to a higher level controller.

3. The method of controlling individual computing units as in claim 1, wherein sending an event message comprises sending an event message to a controller at the same level.

4. The method of controlling individual computing units as in claim 1, wherein:

sending a request comprises transmitting a request through a pre-defined communication interface, wherein the request is in a standardized format common to all controllers;

receiving a response comprises receiving a response through a pre-defined communication interface, wherein the response is in a standardized format common to all controllers.

5. The method of controlling individual computing units as in claim 4, further comprising transmitting to the pre-defined communication interface information indicating at least one controller that the controller associated with the pre-defined communication interface will exchange information with exclusively.

6. The method of controlling individual computing units as in claim 1,

wherein sending a request from one of the controllers to a controller at a different level further comprises sending a request from a level one controller to a controller at a different level;

and

wherein receiving a response at the one of the controllers from the controller at a different level further comprises receiving a response at a level one controller from the controller at a different level.

7. The method of controlling individual computing units as in claim 6, wherein sending a request from a level one controller to a controller at a different level is performed in response to receiving transmission from a device selected from the group consisting of an input/out computing unit, a processor computing unit, a system management computing unit, a routing computing unit, a bedrock computing unit, a level two controller, and a level three controller.

8. The method of controlling individual computing units as in claim 1,

wherein sending a request from one of the controllers to a controller at a different level further comprises sending a request from a level two controller to a controller at a different level and

wherein receiving a response at the one of the controllers from the controller at a different level further comprises receiving a response at a level two controller from the controller at a different level.

9. The method of controlling individual computing units as in claim 8, wherein sending a request from a level two controller to a controller at a different level is performed in response to receiving transmission from a device selected from the group consisting of a system management computing unit, a level one controller, a level two controller, and a level three controller.

10. The method of controlling individual computing units as in claim 8, wherein sending a request from a level three controller to a controller at a different level is performed in response to receiving a transmission from a device selected from the group consisting of a system management computing unit, a level one controller, and a level two controller.

11. The method of controlling individual computing units as in claim 1,

wherein sending a request from one of the controllers to a controller at a different level further comprises sending a request from a level three controller to a controller at a different level;

and

wherein receiving a response at the one of the controllers from the controller at a different level further comprises receiving a response at a level three controller from the controller at a different level.

12. A method of controlling a plurality of individual computing units, including a first and a second computing unit, wherein each computing unit includes a processor and a level one controller, the method comprising:

connecting operably each of the level one controllers to a level three controller;

learning presence of each computing unit through the operably connected level one controllers;

identifying each of the computing units;

sending a request from the level one controller of the first computing unit to the level one controller of the second computing unit;

sending a request from the level one controller of the first computing unit to a controller at a different level; and

receiving a response at the controller of the first computing unit from the controller at a different level.

13. The method of controlling a plurality of individual computing units according to claim 12, wherein the method further comprises:

managing a configuration of the computing units through the operably connected level one controllers; and

monitoring each computing unit through the operably connected level one controllers.

14. A method of controlling individual computing units within a system having a plurality of computer units wherein each computer unit includes a processor the method comprising:

controlling a first level of an individual computing unit;

controlling a second level of a plurality of individual computer units; and

controlling a third level of a plurality of first levels;

wherein controlling the first level comprises;

receiving a first request from the second level;

transmitting a first request response to the second level; and

transmitting a first event message to the second level;

wherein controlling the second level comprises;

transmitting the first request to the first level;

receiving the first request response from first level;

receiving the first event message from first level;

receiving a second request from the third level;

transmitting a second request response to the third level; and

transmitting a second event message to the third level;

wherein controlling the third level comprises;

transmitting the second request to the second level;

receiving the second request response from the second level;

receiving the second event message from the second level;

learning presence of each of a plurality of operably connected systems;

identifying each of the operably connected systems;

managing a configuration of the plurality of operably connected systems; and

monitoring each of the operably connected systems.

15. A method of controlling individual computing units within a system as in claim 14, wherein the monitoring each of the operably connected systems further comprises monitoring the physical environment of each of the operably connected systems and further comprises:

monitoring a power state of the individual computing unit;

monitoring a temperature of the individual computing unit;

monitoring a cooling fan status of the individual computing unit;

monitoring a voltage regulator module state of the individual computing unit;

monitoring a heartbeat state of the individual computing unit;

monitoring a password state of the individual computing unit;

an activity-log state of the individual computing unit;

monitoring a debug-level state of the individual computing unit;

monitoring a non-volatile random access memory state of the individual computing unit;

an identification-number state of the individual computing unit;

monitoring a partition state of the individual computing unit;

monitoring a display state of the individual computing unit;

monitoring a LED display state of the individual computing unit;

monitoring a firmware version state of the individual computing unit; and

monitoring a flash memory state of the individual computing unit.

16. The method of controlling individual computing units within a system as in claim 14, wherein the managing a configuration of the plurality of operably connected systems comprises:

receiving information describing a configuration of operably connected systems; and

creating a master system configuration of individual computer units in the first level of aggregated individual computer units.

17. The method of controlling individual computing units within a system as in claim 14, wherein the monitoring each of the operably connected systems further comprises:

monitoring activity of individual computer units in the first level of aggregated individual computer units; and

routing information associated with the activity of individual computer units to one of the plurality of second level of aggregated first levels of aggregated individual computer units associated with the activity.

18. The method of controlling individual computing units within a system as in claim 14, wherein learning presence of operably connected devices further comprises:

learning presence of slave individual computing units;

learning presence of local console, modem or display;

learning presence of a master first level of aggregated individual computer unit;

learning presence of peer first level of aggregated individual computer units; and

learning presence of a master second level of aggregated first levels of aggregated individual computer units.

19. The method of controlling individual computing units within a system as in claim 14, wherein identifying each of the operably connected devices further comprises:

identifying slave individual computing units;

identifying a peer first level of aggregated individual computer units; and

identifying master second level of aggregated first levels of aggregated individual computer units; and

identifying a primary master second level of aggregated first levels of aggregated individual computer units.

20. A distributed processing system comprising:

a plurality of computing units, each having a first controller, wherein each of the computing units is operably connected to the first controller of each of the other computing units, wherein the first controller of each of the plurality of computing units communicates to each of the first controllers of the plurality of computing units through a first connection;

a second controller operably connected to the plurality of computing units through the first controller of each computing unit, wherein the second controller communicates to each of the first controllers of the plurality of computing units through a Universal Serial Bus compliant connection; and

a third controller operably connected to the second controller and communicating to the second controller through a third connection;

wherein at least one of the plurality of computing units is operably connected to at least one device selected from the group consisting of an input/out computing unit, a processor unit, a system management computing unit, a routing computing unit, an I2C low unit, an I2C high unit, and a bedrock computing unit; and

wherein the first controller comprises;

a first task manager for each one of the at least one devices operably connected between the pre-defined communication interface of the first controller and the at least one device;

a second task manager operably connected between the pre-defined communication interface and the Universal Serial Bus compliant connection; and

a first library for each one of the at least one devices operably connected between the pre-defined communication interface of the first controller and the at least one device.

21. The distributed processing system as in claim 20, wherein the pre-defined communication interface of the first controller accepts from the first task manager of the first controller information indicating at least one controller that the first task manager of the first controller will exchange information with exclusively.

22. The distributed processing system as in claim 20, wherein the I2C low unit, and the I2C high unit includes a power sensor of the individual computing unit, a temperature sensor of the individual computing unit, a cooling fan sensor, a voltage regulator module sensor, a heartbeat sensor, a password sensor, a CMOS sensor, a non-volatile random access memory sensor, a display sensor, a LED display sensor, a flash memory sensor.

23. The A distributed processing system comprising:

a plurality of computing units, each having a first controller, wherein each of the computing units is operably connected to the first controller of each of the other computing units, wherein the first controller of each of the plurality of computing units communicates to each of the first controllers of the plurality of computing units through a RS-422 compliant connection;

a second controller operably connected to the plurality of computing units through the first controller of each computing unit, wherein the second controller communicates to the first controllers of the plurality of computing units through a Universal Serial Bus compliant connection; and

a third controller operably connected to the second controller and communicating to the second controller through an Ethernet compliant connection;

the second controller also operably connected to at least one device selected from the group consisting of a display unit and a system management computing unit; and

wherein the second controller comprises;

a first task manager for each one of the at least one devices operably connected between the pre-defined communication interface of the second controller and the at least one device;

a second task manager operably connected between the pre-defined communication interface and the Universal Serial Bus compliant connection; and

a third task manager operably connected between the pre-defined communication interface and the Ethernet compliant connection.

24. The distributed processing system as in claim 23, wherein the pre-defined communication interface of the second controller accepts from the first task manager of the second controller information indicating at least one controller that the first task manager of the second controller will exchange information with exclusively.

25. The A distributed processing system comprising:

a plurality of computing units, each having a first controller, wherein each of the computing units is operably connected to the first controller of each of the other computing units, wherein the first controller of each of the plurality of computing units communicates to each of the first controllers of the plurality of computing units through a first connection;

a second controller operably connected to the plurality of computing units through the first controller of each computing unit, wherein the second controller communicates to each of the first controllers of the plurality of computing units through a second connection; and

a third controller operably connected to the second controller and communicating to the second controller through an Ethernet compliant connection;

the third controller also operably connected to at least one device selected from the group consisting of a display unit and a system management computing unit;

wherein the third controller comprises;

a first task manager for each one of the at least one devices operably connected between the pre-defined communication interface of the third controller and the at least one device; and

a second task manager operably connected between the pre-defined communication interface and the Ethernet compliant connection.

26. The distributed processing system as in claim 25, wherein the pre-defined communication interface of the third controller accepts from the first task manager of the third controller information indicating at least one controller that the first task manager of the third controller will exchange information with exclusively.

27. A computer system comprising:

at least two processing functional modules each including a processing unit adapted to process data and adapted to input/output data to other functional modules through at least two ports with each port including a plurality of data lines;

at least one routing functional module adapted to route data and adapted to input/output data to other functional modules through at least two ports with each port including a plurality of data lines;

at least one input or output functional module adapted to input or output data and adapted to input/output data to other functional modules through at least one port including a plurality of data lines;

each processing, routing and input or output functional module including a local controller adapted to control the local operation of the associated functional module, wherein the local controller is adapted to input and output control information over control lines connected to the respective ports of its functional module;

at least one system controller functional module adapted to communicate with one or more local controllers and provide control at a level above the local controllers;

each of the functional modules adapted to be cabled together with a single cable that includes a plurality of data lines and control lines such that control lines in each module are connected together and data lines in each unit are connected together; and

each of the local controllers adapted to detect other local controllers to which it is connected and to thereby collectively determine the overall configuration of a system.