Method and apparatus to detect/manage faults in a system

• US 20070088974A1
• Filed: 09/26/2005
• Published: 04/19/2007
• Est. Priority Date: 09/26/2005
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Hiremath, Chetan, Chandwani, Neelam, Balakrishnan, Santosh, Mukheriee, Udayan, Dodeia, Rakesh
• Granted Patent: US 7,424,666 B2
• Time in Patent Office: Days
• US Class Current: 714/6
• CPC Class Codes: G06F 11/0724 in a multiprocessor or a multi-core unit multiprocessors per se G06F15/80
• CPC Class Codes: G06F 11/0757 by exceeding a time limit, i.e. time-out, e.g. watchdogs
• CPC Class Codes: G06F 11/076 by exceeding a count or rate limit, e.g. word- or bit count limit
• CPC Class Codes: G06F 11/0781 Error filtering or prioritizing based on a policy defined by the user or on a policy defined by a hardware/software module, e.g. according to a severity level

Abstract

A fault module supports detection, analysis, and/or logging of various faults in a processor system. In one embodiment, the system is provided on a multi-core, single die device.

Claims

1. A system, comprising:

a processor system including

a plurality of processing engines including error detection hooks, the plurality of processing engines formed from multiple cores on a single die; and

a fault module to monitor error information from the error detection hooks, the fault module including a failure prediction layer coupled to a fault detection layer, which is coupled to the error detection hooks of the processing engines.

2. The system according to claim 1, wherein the processor includes a host processor coupled to the plurality of processing engines.

3. The system according to claim 1, wherein the system further includes a control processor coupled to a host processor.

4. The system according to claim 1, wherein the system further includes a dedicated microcontroller.

5. The system according to claim 1, further including a peripheral device having error detection hooks for monitoring errors in devices and coupled to the fault detection and failure prediction layer.

6. The system according to claim 1, wherein the system further includes one or more memory components having error detection hooks for monitoring memory errors and coupled to the fault detection and the failure prediction layer.

7. The system according to claim 1, wherein the system includes hardware accelerators.

8. The system according to claim 2, wherein fault module executes on the host processor.

9. The system according to claim 4, wherein fault module executes on the dedicated microcontroller.

10. The system according to claim 2, wherein fault module executes on a virtual partition of the host processor.

11. The system according to claim 3, wherein fault module executes on the control processor.

12. The system according to claim 2, wherein the fault module is implemented as a set of native instructions set on the processor optimized for implementing failure prediction mechanisms.

13. The system according to claim 2, wherein the fault module is implemented as a hardware functional block in an application specific integrated circuit.

14. The system according to claim 2, wherein the fault module is implemented as a hardware functional block instantiated in a processor core.

15. The system according to claim 2, wherein the fault module is implemented as a hardware functional block instantiated in a processor chipset.

16. The system according to claim 2, wherein the fault module is implemented as combination of hardware functional blocks instantiated in a processor core and chipset.

17. The system according to claim 2, wherein the fault module is implemented as a hardware functional block in a field programmable gate array.

18. The system according to claim 2, wherein the fault module is implemented on a dedicated core in a many-core or a multi-core processor architecture.

19. The system according to claim 1, wherein fault module implements failure prediction capabilities to determine likelihood of critical failure of components in the system.

20. The system according to claim 1, wherein fault module includes error correlation.

21. A method of managing faults in a processor system, comprising:

monitoring errors detected in components of the processor system using error detection hooks; and

analyzing errors in a fault module to predict future failure of the components using a failure prediction layer coupled with a fault detection layer.

22. The method according to claim 21, wherein the fault module runs on a processor coupled to a plurality of processing engines formed from multiple cores on a single die having error detection hooks.

23. The method according to claim 21, further including correlating the errors to avoid false alarms and isolate root cause of a failure.

24. The method according to claim 21, wherein the fault module includes a training mechanism to build a data set of errors potentially leading to system failure and to determine suitable heuristics parameters to be used in failure prediction layer.

25. The method according to claim 24, wherein the heuristics parameters include one or more of error rates, timing windows, thresholds, conditional probabilities.

26. The method according to claim 25, wherein the processor includes a host processor coupled to the plurality of processing engines.

27. The method according to claim 22, wherein the processor includes a control processor coupled to a host processor.

28. The method according to claim 22, wherein the processor includes a dedicated microcontroller.

29. The method according to claim 21, wherein the system includes further components as one or more of memory, interface circuit cards, and peripheral devices.

30. An article, comprising:

a storage medium having stored thereon instructions that when executed by a machine result in the following;

running a fault module on a processor coupled to a processing engines formed from multiple cores on a single die having error detection hooks; and

monitoring error information by the error module from error detection hooks, the fault module including a failure prediction layer coupled to a fault detection layer, which is coupled to the error detection hooks of the processing engines.

31. The article according to claim 30, wherein the processor includes a host processor coupled to the plurality of processing engines.

32. The article according to claim 30, wherein the processor includes a control processor coupled to a host processor.

33. The article according to claim 30, wherein the processor includes a dedicated microcontroller.

34. A system, comprising:

a host processor having error detection hooks;

processing engines on multiple cores on a single die coupled to the host processor, the processing engines having error detection hooks;

a microcontroller coupled to the host processor and the processing engines, the microcontroller having a fault module to run on the microcontroller to collect error information from the error detection hooks;

wherein the fault module includes a failure prediction layer coupled to a fault detection layer.

35. The system according to claim 34, wherein the fault detection layer includes an error handler layer.

36. The system according to claim 35, wherein the fault detection layer includes a platform hardware interface layer.

37. The system according to claim 34, wherein the failure prediction layer includes a heuristics layer to predict component failures.