High density compute center resilient booting

• US 7,434,102 B2
• Filed: 12/29/2004
• Issued: 10/07/2008
• Est. Priority Date: 12/29/2004
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Zimmer, Vincent J., Rothman, Michael A., Swanson, Robert C.
• Primary Examiner(s): Iqbal; Nadeem
• Application Number:
  US11/026,407
• Publication Number:
  US 20060143602A1
• Time in Patent Office: 1,378 Days
• Field of Search: 714/36, 714 10- 13, 714/15, 714/16, 714/18, 714/38
• US Class Current: 714/36
• CPC Class Codes: G06F 11/0709 in a distributed system consisting of a plurality of standalone computer nodes, e.g. clusters, client-server systems
• CPC Class Codes: G06F 11/076 by exceeding a count or rate limit, e.g. word- or bit count limit
• CPC Class Codes: G06F 11/0766 Error or fault reporting or storing
• CPC Class Codes: G06F 11/0793 Remedial or corrective actions recovery from an exception in an instruction pipeline G06F9/3861; by retry G06F11/1402; for recovering from a failure of a protocol instance or entity H04L69/40
• CPC Class Codes: G06F 11/1441 Resetting or repowering
• CPC Class Codes: G06F 11/2033 switching over of hardware resources
• CPC Class Codes: G06F 11/2035 without idle spare hardware
• CPC Class Codes: G06F 11/2048 where the redundant components share neither address space nor persistent storage

Abstract

A system and method to implement a resilient compute center. A plurality of processing systems is initialized. Each of the processing systems capable of operation communicates status information about its operational health to a management module responsible for managing the processing systems. The management module reinitializing any of the processing systems, if the management module determines that any of the processing systems is operating in a degraded state based on the status information communicated to the management module.

Claims

1. A method, comprising:

initializing a plurality of processing systems;

communicating status information about the operational health of each of the processing systems capable of operation to a management module responsible for managing the processing systems;

reinitializing one or more of the processing systems, if the management module determines that the one or more of the processing systems is operating in a degraded state based on the status information communicated to the management module from each of the processing systems capable of operation;

reseting one or more of the processing systems, if the management module determines that the one or more of the processing systems is incapable of operation;

maintaining a failure count of a number of the resets for each of the processing systems incapable of operation; and

alerting an administrator system about any of the processing systems having a failure count greater than a predefined number.

2. The method of claim 1, wherein the processing systems comprise processing blades.

3. The method of claim 2, wherein the status information includes at least one of memory status information, processor status information, and option ROM status information.

4. The method of claim 2, further comprising claiming first chair status by one of the processing blades to be a first chair blade, the first chair blade to assume surrogate management module duties if the management module is incapacitated.

5. The method of claim 4, wherein the first chair status is claimed by the one of the processing blades based on an initialization time stamp.

6. The method of claim 4, further comprising:

querying the management module by the first chair blade to determine an operating state of the management module; and

resetting the management module, if the operating state is determined to be a non-healthful state.

7. The method of claim 6, further comprising:

assuming the surrogate management module duties by the first chair blade, while the management module is incapacitated; and

alerting an administrator system of the non-healthful state of the management module.

8. The method of claim 1, further comprising generating system event logs within each of the processing systems to track progress of each the initializations, the status information including entries from the system event logs.

9. A processing blade, comprising

at least one processor to execute instructions;

system memory coupled to the at least one processor;

a communication link to communicatively couple to a management module for managing a rack of processing blades including the processing blade;

an error module configured to generate status information about the operational health of the processing blade and to communicate the status information to the management module via the communication link; and

a surrogate management module, the surrogate management module configured to query the management module for managing the rack of processing blades and to assume tasks for managing the rack of processing blades from the management module if the management module is disabled.

10. The processing blade of claim 9, wherein the status information includes at least one of a first status indication about the system memory and a second status indication about the at least one processor.

11. The processing blade of claim 10, wherein the error module is further configured to interrogate the at least one processor and the system memory during a pre-boot phase of the processing blade to generate the status information.

12. The processing blade of claim 9, wherein the surrogate management module and the error module comprise software entities executed by the at least one processor.

13. The processing blade of claim 11, wherein the error module comprises a service processor and a service processor firmware unit coupled to the service processor.

14. The processing blade of claim 13, further comprising first sensor lines coupled between the at least one processor and the service processor to obtain the status indication about the at least one processor and second sensor lines coupled between the system memory and the service processor to obtain the status indication about the system memory.

15. The processing blade of claim 9, wherein the communication link is configured to couple to a communication plane for interconnecting the rack of processing blades to the management module.

16. A system, comprising:

a chassis;

a management module supported by the chassis;

a communication plane coupled to the management module; and

a plurality of processing blades supported by the chassis and coupled to the communication plane, one of the processing blades including;

at least one processor to execute instructions;

system memory coupled to the at least one processor; and

a surrogate management module, the surrogate management module coupled to query the management module to determine an operational health of the management module and to assume management duties of the management module if the management module is disabled, the management module coupled to the communication plane to manage the plurality of processing blades.

17. The system of claim 16, wherein the one of the processing blades further includes an error module, the error module to generate status information about the operational health of the one of the processing blades and to communicate the status information to the management module.

18. The system of claim 17, wherein the error module comprises service processor and firmware unit.

19. The system of claim 17, wherein the error module comprises a software agent stored within a non-volatile memory device coupled to the at least one processor and executable by the at least one processor.

20. The system of claim 16, wherein the management module includes:

a fail counter to maintain a failure count for each of the processing blades that fails to initialize; and

a degraded state counter to maintain a degraded state count for each of the processing blades that fails to initialize into a fully operational state.