Computer flight recorder with active error detection

• US 9,910,758 B2
• Filed: 08/13/2014
• Issued: 03/06/2018
• Est. Priority Date: 01/17/2014
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Gregg, Leon E., Tilbury, James L.
• Primary Examiner(s): TRUONG, LOAN
• Application Number:
  US14/458,881
• Publication Number:
  US 20150205651A1
• Time in Patent Office: 1,301 Days
• CPC Class Codes: G06F 11/0739 in a data processing system embedded in automotive or aircraft systems
• CPC Class Codes: G06F 11/0751 Error or fault detection not based on redundancy power supply failures G06F1/30; network fault management H04L41/06
• CPC Class Codes: G06F 11/3466 Performance evaluation by tracing or monitoring
• CPC Class Codes: G06F 11/3476 Data logging G06F11/14, G06F11/2205 take precedence

Abstract

A method, apparatus and program product utilize a flight recorder having active error detection functionality to proactively detect errors in a computer or a sub-system or component thereof. The active error detection may be based on one or more trends detected in the trace data logged by the flight recorder and reflective of particular types of errors that may be present in a computer during operation, such that an error may be logged, and in some instances, an exception may be triggered.

Claims

1. An apparatus, comprising:

a removable media system that provides an interface to one or more removable storage devices;

a computer including at least one processor, the computer configured to execute one or more instrumented software entities resident in the computer to issue work requests to the removable media system; and

a flight recorder configured to execute on the at least one processor, the flight recorder configured to log trace data for the one or more instrumented software entities in response to specific points in executable program code for the one or more instrumented software entities being reached during operational use of the computer, detect a trend in the logged trace data, and determine an error based on the detected trend in the logged trace data, wherein the flight recorder is configured to log the trace data in response to calls to the flight recorder from the one or more instrumented software entities, wherein the flight recorder is configured to log trace data associated with the work requests issued to the removable media system, wherein the logged trace data comprises a plurality of logged trace points, wherein each logged trace point in the plurality of logged trace points is associated with a work request issued to the removable media system, wherein the flight recorder is configured to detect the trend by detecting a plurality of trace points in the logged trace data associated with a first work request among the work requests issued to the removable media system, and wherein the flight recorder is further configured to assert an exception and terminate the first work request in response to determining the error such that availability of the removable media system is restored without having to perform a manual reboot for the removable media system.

2. The apparatus of claim 1, wherein the logged trace data comprises a plurality of logged trace points.

3. The apparatus of claim 2, wherein the program code is configured to detect the trend by detecting a plurality of trace points in the logged trace data that meet a trend criterion.

4. The apparatus of claim 3, wherein each logged trace point in the plurality of logged trace points includes a trace point identifier, and wherein the program code is configured to detect the plurality of trace points in the logged trace data that meet the trend criterion by detecting a plurality of consecutive trace points having the same trace point identifier.

5. The apparatus of claim 3, wherein each logged trace point in the plurality of logged trace points is associated with an operation, and wherein the program code is configured to detect the plurality of trace points in the logged trace data that meet the trend criterion by detecting a plurality of consecutive trace points associated with the same operation.

6. The apparatus of claim 5, wherein each logged trace point in the plurality of logged trace points includes an operation identifier, wherein the program code is configured to detect the plurality of consecutive trace points associated with the same operation by detecting a plurality of consecutive trace points having the same operation identifier.

7. The apparatus of claim 6, wherein the operation identifier for each logged trace point in the plurality of logged trace points includes a major operation identifier and a minor operation identifier, wherein the program code is configured to detect the plurality of consecutive trace points having the same operation identifier by detecting a plurality of consecutive trace points having the same major operation identifier.

8. The apparatus of claim 7, wherein the program code is configured to detect the plurality of consecutive trace points having the same operation identifier further by detecting that the plurality of consecutive trace points having the same operation identifier have the same major and minor operation identifiers.

9. The apparatus of claim 3, wherein each logged trace point in the plurality of logged trace points is associated with an instrumented software entity from among the one or more instrumented software entities in the computer, and wherein the program code is configured to detect the plurality of trace points in the logged trace data that meet the trend criterion by detecting a plurality of consecutive trace points associated with the same instrumented software entity.

10. The apparatus of claim 9, wherein each of the one or more instrumented software entities comprises a task, and wherein the program code is configured to detect the plurality of consecutive trace points associated with the same instrumented software entity by detecting a plurality of consecutive trace points associated with the same task.

11. The apparatus of claim 9, wherein each of the one or more instrumented software entities is identified by an entity identifier, wherein each logged trace point in the plurality of logged trace points includes an entity identifier, and wherein the program code is configured to detect the plurality of consecutive trace points associated with the same instrumented software entity by detecting a plurality of consecutive trace points having the same entity identifier.

12. The apparatus of claim 3, wherein the program code is configured to detect the plurality of trace points in the logged trace data that meet the trend by includes detecting a plurality of trace points matching a predetermined pattern.

13. The apparatus of claim 3, wherein the program code is configured to determine the error by determining that the detected plurality of logged trace points meeting the trend criterion meet an error criterion.

14. The apparatus of claim 13, wherein the program code is configured to detect the trend by counting a number of logged trace points that meet the trend criterion, and wherein the program code is configured to determine that the detected plurality of logged trace points meeting the trend criterion meet the error criterion by comparing the number of logged trace points that meet the trend criterion against an error threshold value.

15. The apparatus of claim 1, wherein the program code is further configured to log the error with the flight recorder in response to determining the error.

16. A program product, comprising:

a non-transitory computer readable medium; and

a flight recorder stored on the non-transitory computer readable medium and configured upon execution by at least one processor of a computer to log trace data for one or more instrumented software entities resident in the computer in response to specific points in executable program code for the one or more instrumented software entities being reached during operational use of the computer, wherein execution of the one or more instrumented software entities by the computer issues work requests to a removable media system of the computer that provides an interface to one or more removable storage devices, wherein the flight recorder is configured to detect a trend in the logged trace data, and determine an error based on the detected trend in the logged trace data, wherein the flight recorder is configured to log the trace data in response to calls to the flight recorder from the one or more instrumented software entities, wherein the flight recorder is configured to log trace data associated with the work requests issued to the removable media system, wherein the logged trace data comprises a plurality of logged trace points, wherein each logged trace point in the plurality of logged trace points is associated with a work request issued to the removable media system, wherein the flight recorder is configured to detect the trend by detecting a plurality of trace points in the logged trace data associated with a first work request among the work requests issued to the removable media system, and wherein the flight recorder is further configured to assert an exception and terminate the first work request in response to determining the error such that availability of the removable media system is restored without having to perform a manual reboot for the removable media system.