Generating a data structure to maintain error and connection information on components and use the data structure to determine an error correction operation

• US 9,298,541 B2
• Filed: 04/22/2014
• Issued: 03/29/2016
• Est. Priority Date: 04/22/2014
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Gao, Xue Dong, Lu, Chang Ping, Sorenson, Todd C., Steffan, Jeffrey R.
• Primary Examiner(s): Manoskey, Joseph D
• Application Number:
  US14/259,135
• Publication Number:
  US 20150301881A1
• Time in Patent Office: 707 Days
• Field of Search: 714/48, 714/14, 714/25, 714/26, 714/33, 714/40, 714/44
• US Class Current: 1/1
• CPC Class Codes: G06F 11/0727 in a storage system, e.g. in a DASD or network based storage system drivers for digital recording or reproducing units G06F3/06; circuits for error detection or correction within digital recording or reproducing units G11B20/18; for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS], H04L67/1097
• CPC Class Codes: G06F 11/079 Root cause analysis, i.e. error or fault diagnosis in a hardware test environment G06F11/22; in a software test environment G06F11/36
• CPC Class Codes: G06F 11/0793 Remedial or corrective actions recovery from an exception in an instruction pipeline G06F9/3861; by retry G06F11/1402; for recovering from a failure of a protocol instance or entity H04L69/40
• CPC Class Codes: G06F 11/1608 Error detection by comparing the output signals of redundant hardware G06F11/1629, G06F11/1666 take precedence; error detection or correction in information storage based on relative movement between record carrier and transducer G11B20/18; checking static stores for correct operation G11C29/00; for logic circuits H03K19/003, H03K19/007; for pulse counters or frequency dividers H03K21/40
• CPC Class Codes: G06F 11/20 using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements
• CPC Class Codes: G06F 11/2015 Redundant power supplies power supply failure G06F1/30
• CPC Class Codes: G06F 2201/85 Active fault masking without idle spares active fault masking without idle spare hardware where processing functionality is redundant G06F11/2035

Abstract

Provided are a computer program product, system, and method for generating data structure to maintain error and connection information on components and use the data structure to determine an error correction operation. For each of a plurality of first level components in enclosures connected to second level components, errors at the first level component and a connection between the first level component to one of the second level components are determined and error variables are set to indicate whether an error was reported at the first level component. A data structure is generated indicating connections among the first level components and the second level components. The error variable values and the data structure are used to determine an error correction operation with respect to at least one of the first level component and the connected second level component.

Claims

1. A computer program product for determining error handling operations at enclosures, wherein each enclosure includes at least one first level component, wherein each of the first level components in each of the enclosures is connected to at least one second level component, the computer program product comprising a computer readable storage medium having computer readable program code embodied therein that when executed performs operations, the operations comprising:

for each of the first level components, performing;

communicating with the enclosure including the first level component to determine errors at the first level component and a connection between the first level component to one of the second level components;

setting an error variable to indicate whether an error corresponding to the error variable was reported at the first level component;

generating a data structure indicating connections among the first level components and the second level components; and

using error variables for the first level components and the data structure indicating the connections to determine an error correction operation with respect to at least one of the first level component and a connected second level component for those first level components having error variables indicating errors.

2. The computer program product of claim 1, wherein the first level components comprise heterogeneous components that report different error codes for a same type of error, wherein the error variables are set based on the reported error codes from the first level components, and wherein the error variables are same for the heterogeneous components.

3. The computer program product of claim 1, wherein generating the data structure comprises:

generating a second level component index including an entry for each second level component, wherein each entry for the second level components identifies first level component information for each first level component connected to the second level component of the entry, wherein the first level component information for each of the first level components includes the error variables for the first level component.

4. The computer program product of claim 3, wherein generating the data structure further comprises:

generating an enclosure data structure including enclosure entries for each of the enclosures including first level components, wherein each enclosure entry is associated with the first level component information for the at least one of the first level components included in the enclosure associated with the enclosure data structure.

5. The computer program product of claim 1, wherein the using the error variables for the first level components and the data structure indicating the connections to determine the error correction operation comprises:

setting the error correction operation to indicate at a first priority to check the connection between the first level components having the error variables indicating the error and the connected second level component and to indicate at a second priority to replace the connected second level component in response to determining that the error variables indicate the error for multiple of the first level components connected to one second level component.

6. The computer program product of claim 5, wherein the determining the error correction operation comprises:

setting the error correction operation to indicate at a first priority to check the connection between the first level component having the error variable indicating the error and the connected second level component and to indicate at a second priority to replace the first level component in response to determining that the error is indicated in the error variable of only one of the first level components connected to the second level component.

7. The computer program product of claim 1, wherein the first level components comprise power supply units (PSUs) to supply power to the enclosures including the PSUs and the second level components comprise power distribution units (PDUs) to supply power to the PSUs, wherein one of the error variables for each of the first level components comprises a PDU power failure variable indicating a failure of power supply from a connected PDU, wherein the determining the error correction operation comprises:

setting the error correction operation to indicate at a first priority to check the connections between the PSUs having the PDU power failure variable indicating the error and the connected PDU and indicate at a second priority to replace the connected PDU in response to determining that the PDU power failure variables of multiple of the PSUs connected to one of the PDUs indicates the error; and

setting the error correction operation to indicate at the first priority to check the connection between a PSU having the PDU power failure variable indicating the error and the connected PDU and indicate at a second priority to replace the PSU in response to determining that the PDU power failure variable of only one of the PSUs connected to one of the PDUs indicates the error.

8. The computer program product of claim 7, wherein one of the error variables for each of the first level components comprises a PSU power failure variable indicating a failure of power supply at the PSU, wherein the determining the error correction operation comprises:

setting the error correction operation to indicate to replace the PSU having the PSU power failure variable indicating the error in response to determining the PSU power failure variable of one of the PSUs indicates an error.

9. The computer program product of claim 7, wherein one of the error variables for each of the first level components comprises an Early Power Off Warning (EPOW) variable indicating reporting of an EPOW event at the first level component or enclosure, wherein the determining the error correction operation comprises:

setting the error correction operation to indicate to replace the connected PDU in response to determining that the EPOW variable for multiple of the PSUs connected to one of the PDUs indicates the EPOW event; and

setting the error correction operation to indicate to replace the PSU having the EPOW variable indicating the EPOW event in response to only one of the PSUs having the EPOW variable indicating the EPOW event.

10. The computer program product of claim 1, wherein the operations further comprise:

providing a plurality of class interfaces, wherein each of the class interfaces are for different first level component types, wherein a plurality of the first level component types report errors with different error codes;

for each of the first level components in each of the enclosures, performing;

calling methods of the class interface for a first level component type of the first level component to access the enclosure to determine errors with respect to the first level component in the enclosure;

determining connections from the first level component to one of the second level components to which the first level component connects; and

updating at least one error variable of the class interface indicating any determined errors at the first level component, wherein the error variables in different class interface types are common across the class interface types.

11. A system for determining error handling operations at enclosures, wherein each enclosure includes at least one first level component, wherein each of the first level components in each of the enclosures is connected to at least one second level component, comprising:

a processor; and

a computer readable storage medium including program code that when executed by the processor, performs;

for each of the first level components, performing;

communicating with the enclosure including the first level component to determine errors at the first level component and a connection between the first level component to one of the second level components;

setting an error variable to indicate whether an error corresponding to the error variable was reported at the first level component;

generating a data structure indicating connections among the first level components and the second level components; and

using error variables for the first level components and the data structure indicating the connections to determine an error correction operation with respect to at least one of the first level component and a connected second level component for those first level components having error variables indicating errors.

12. The system of claim 11, wherein the first level components comprise heterogeneous components that report different error codes for a same type of error, wherein the error variables are set based on the reported error codes from the first level components, and wherein the error variables are same for the heterogeneous components.

13. The system of claim 11, wherein generating the data structure comprises:

generating a second level component index including an entry for each second level component, wherein each entry for the second level components identifies first level component information for each first level component connected to the second level component of the entry, wherein the first level component information for each of the first level components includes the error variables for the first level component.

14. The system of claim 11, wherein the using the error variables for the first level components and the data structure indicating the connections to determine an error correction operation comprises:

setting the error correction operation to indicate at a first priority to check the connection between the first level components having the error variables indicating the error and the connected second level component and to indicate at a second priority to replace the connected second level component in response to determining that the error variables indicate the error for multiple of the first level components connected to one second level component.

15. The system of claim 11, wherein the first level components comprise power supply units (PSUs) to supply power to the enclosures including the PSUs and the second level components comprise power distribution units (PDUs) to supply power to the PSUs, wherein one of the error variables for each of the first level components comprises a power distribution unit (PDU) power failure variable indicating a failure of power supply from a connected PDU, wherein the determining the error correction operation comprises:

setting the error correction operation to indicate at a first priority to check the connections between the PSUs having the PDU power failure variable indicating the error and a connected PDU and indicate at a second priority to replace the connected PDU in response to determining that the PDU power failure variables of multiple of the PSUs connected to one of the PDUs indicates the error; and

setting the error correction operation to indicate at the first priority to check the connection between a PSU having the PDU power failure variable indicating the error and the connected PDU and indicate at a second priority to replace the PSU in response to determining that the PDU power failure variable of only one of the PSUs connected to one of the PDUs indicates the error.

16. The system of claim 11, wherein the operations further comprise:

providing a plurality of class interfaces, wherein each of the class interfaces are for different first level component types, wherein a plurality of the first level component types report errors with different error codes;

for each of the first level components in each of the enclosures, performing;

calling methods of the class interface for a first level component type of the first level component to access the enclosure to determine errors with respect to the first level component in the enclosure;

determining connections from the first level component to one of the second level components to which the first level component connects; and

updating at least one error variable of the class interface indicating any determined errors at the first level component, wherein the error variables in different class interface types are common across the class interface types.

17. A method for determining error handling operations at enclosures, wherein each enclosure includes at least one first level component, wherein each of the first level components in each of the enclosures is connected to at least one second level component, comprising:

for each of the first level components, performing;

communicating with the enclosure including the first level component to determine errors at the first level component and a connection between the first level component to one of the second level components;

setting an error variable to indicate whether an error corresponding to the error variable was reported at the first level component;

generating a data structure indicating connections among the first level components and the second level components; and

using error variables for the first level components and the data structure indicating the connections to determine an error correction operation with respect to at least one of the first level component and a connected second level component for those first level components having error variables indicating errors.

18. The method of claim 17, wherein the first level components comprise heterogeneous components that report different error codes for a same type of error, wherein the error variables are set based on the reported error codes from the first level components, and wherein the error variables are same for the heterogeneous components.

19. The method of claim 17, wherein generating the data structure comprises:

generating a second level component index including an entry for each second level component, wherein each entry for the second level components identifies first level component information for each first level component connected to the second level component of the entry, wherein the first level component information for each of the first level components includes the error variables for the first level component.

20. The method of claim 17, wherein using the error variables for the first level components and the data structure indicating the connections to determine the error correction operation comprises:

setting the error correction operation to indicate at a first priority to check the connection between the first level components having the error variables indicating the error and the connected second level component and to indicate at a second priority to replace the connected second level component in response to determining that the error variables indicate the error for multiple of the first level components connected to one second level component.

21. The method of claim 17, wherein the first level components comprise power supply units (PSUs) to supply power to the enclosures including the PSUs and the second level components comprise power distribution units (PDUs) to supply power to the PSUs, wherein one of the error variables for each of the first level components comprises a PDU power failure variable indicating a failure of power supply from a connected PDU, wherein the determining the error correction operation comprises:

setting the error correction operation to indicate at the first priority to check the connections between a PSUs having the PDU power failure variable indicating the error and a connected PDU and indicate at a second priority to replace the connected PDU in response to determining that the PDU power failure variables of multiple of the PSUs connected to one of the PDUs indicates the error; and

setting the error correction operation to indicate at the first priority to check the connection between a PSU having the PDU power failure variable indicating the error and the connected PDU and indicate at a second priority to replace the PSU in response to determining that the PDU power failure variable of only one of the PSUs connected to one of the PDUs indicates the error.

22. The method of claim 17, further comprising:

providing a plurality of class interfaces, wherein each of the class interfaces are for different first level component types, wherein a plurality of the first level component types report errors with different error codes;

for each of the first level components in each of the enclosures, performing;

calling methods of the class interface for a first level component type of the first level component to access the enclosure to determine errors with respect to the first level component in the enclosure;

determining connections from the first level component to one of the second level components to which the first level component connects; and

updating at least one error variable of the class interface indicating any determined errors at the first level component, wherein the error variables in different class interface types are common across the class interface types.