Computer flight recorder with active error detection

• US 9,996,445 B2
• Filed: 01/17/2014
• Issued: 06/12/2018
• Est. Priority Date: 01/17/2014
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Gregg, Leon E., Tilbury, James L.
• Primary Examiner(s): Truong, Loan L. T.
• Application Number:
  US14/158,146
• Publication Number:
  US 20150205654A1
• Time in Patent Office: 1,607 Days
• CPC Class Codes: G06F 11/0739 in a data processing system embedded in automotive or aircraft systems
• CPC Class Codes: G06F 11/0751 Error or fault detection not based on redundancy power supply failures G06F1/30; network fault management H04L41/06
• CPC Class Codes: G06F 11/3466 Performance evaluation by tracing or monitoring
• CPC Class Codes: G06F 11/3476 Data logging G06F11/14, G06F11/2205 take precedence

Abstract

A method, apparatus and program product utilize a flight recorder having active error detection functionality to proactively detect errors in a computer or a sub-system or component thereof. The active error detection may be based on one or more trends detected in the trace data logged by the flight recorder and reflective of particular types of errors that may be present in a computer during operation, such that an error may be logged, and in some instances, an exception may be triggered.

Claims

1. A method of detecting errors in a computer using a flight recorder resident in the computer, the flight recorder configured to log trace data for one or more instrumented software entities resident in the computer in response to specific points in executable program code for the one or more instrumented software entities being reached, the method comprising:

executing the one or more instrumented software entities by the computer to issue work requests to a removable media system of the computer that provides an interface to one or more removable storage devices;

with the flight recorder, and in response to calls to the flight recorder from the one or more instrumented software entities, logging trace data for the one or more instrumented software entities during operational use of the computer, wherein the flight recorder is configured to log trace data associated with the work requests issued to the removable media system, wherein the logged trace data comprises a plurality of logged trace points, and wherein each logged trace point in the plurality of logged trace points is associated with a work request issued to the removable media system;

with the flight recorder, detecting a trend in the logged trace data, wherein detecting the trend includes detecting a plurality of trace points in the logged trace data associated with a first work request among the work requests issued to the removable media system;

with the flight recorder, determining an error associated with the first work request based on the detected trend in the logged trace data; and

with the flight recorder, asserting an exception and terminating the first work request in response to determining the error such that availability of the removable media system is restored without having to perform a manual reboot for the removable media system.

2. The method of claim 1, wherein the logged trace data comprises a plurality of logged trace points.

3. The method of claim 2, wherein detecting the trend includes detecting a plurality of trace points in the logged trace data that meet a trend criterion.

4. The method of claim 3, wherein each logged trace point in the plurality of logged trace points includes a trace point identifier, and wherein detecting the plurality of trace points in the logged trace data that meet the trend criterion includes detecting a plurality of consecutive trace points having the same trace point identifier.

5. The method of claim 3, wherein each logged trace point in the plurality of logged trace points is associated with an operation, and wherein detecting the plurality of trace points in the logged trace data that meet the trend criterion includes detecting a plurality of consecutive trace points associated with the same operation.

6. The method of claim 5, wherein each logged trace point in the plurality of logged trace points includes an operation identifier, wherein detecting the plurality of consecutive trace points associated with the same operation includes detecting a plurality of consecutive trace points having the same operation identifier.

7. The method of claim 6, wherein the operation identifier for each logged trace point in the plurality of logged trace points includes a major operation identifier and a minor operation identifier, wherein detecting the plurality of consecutive trace points having the same operation identifier includes detecting a plurality of consecutive trace points having the same major operation identifier.

8. The method of claim 7, wherein detecting the plurality of consecutive trace points having the same operation identifier further includes detecting that the plurality of consecutive trace points having the same operation identifier have the same major and minor operation identifiers.

9. The method of claim 3, wherein each logged trace point in the plurality of logged trace points is associated with an instrumented software entity from among the one or more instrumented software entities in the computer, and wherein detecting the plurality of trace points in the logged trace data that meet the trend criterion includes detecting a plurality of consecutive trace points associated with the same instrumented software entity.

10. The method of claim 9, wherein each of the one or more instrumented software entities comprises a task, and wherein detecting the plurality of consecutive trace points associated with the same instrumented software entity comprises detecting a plurality of consecutive trace points associated with the same task.

11. The method of claim 9, wherein each of the one or more instrumented software entities is identified by an entity identifier, wherein each logged trace point in the plurality of logged trace points includes an entity identifier, and wherein detecting the plurality of consecutive trace points associated with the same instrumented software entity comprises detecting a plurality of consecutive trace points having the same entity identifier.

12. The method of claim 3, wherein detecting the plurality of trace points in the logged trace data that meet the trend criterion includes detecting a plurality of trace points matching a predetermined pattern.

13. The method of claim 3, wherein determining the error includes determining that the detected plurality of logged trace points meeting the trend criterion meet an error criterion.

14. The method of claim 13, wherein detecting the trend includes counting a number of logged trace points that meet the trend criterion, and wherein determining that the detected plurality of logged trace points meeting the trend criterion meet the error criterion includes comparing the number of logged trace points that meet the trend criterion against an error threshold value.

15. The method of claim 1, further comprising logging the error with the flight recorder in response to determining the error.

16. The method of claim 3, wherein detecting the plurality of trace points in the logged trace data that meet the trend criterion includes detecting a plurality of consecutive trace points in the logged trace data having a same characteristic.

17. The method of claim 16, wherein:

logging the trace data during operational use of the computer includes, in response to a call from an instrumented software entity among the one or more instrumented software entities to the flight recorder;

logging a trace point;

determining whether the logged trace point has the same characteristic as a prior logged trace point;

if the logged trace point has the same characteristic as the prior logged trace point, incrementing a counter; and

if the logged trace point does not have the same characteristic as the prior logged trace point, resetting the counter; and

determining the error includes determining that the detected plurality of logged trace points meeting the trend criterion meet an error criterion by comparing the counter with an error threshold value.