Method, apparatus, and program for detecting sequential and distributed path errors in MPIO

• US 7,076,573 B2
• Filed: 11/20/2003
• Issued: 07/11/2006
• Est. Priority Date: 11/20/2003
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Tee, Stephen M., Tinnakul, Teerasit, Allen, James Patrick, Damron, Timothy M.
• Primary Examiner(s): PEYTON, TAMMARA R
• Application Number:
  US10/718,096
• Publication Number:
  US 20050154797A1
• Time in Patent Office: 964 Days
• Field of Search: 710/17, 710/15, 710/45, 709/223, 709/224, 714/30, 714/39, 714/43, 714/48
• US Class Current: 710/17
• CPC Class Codes: G06F 11/0727 in a storage system, e.g. in a DASD or network based storage system drivers for digital recording or reproducing units G06F3/06; circuits for error detection or correction within digital recording or reproducing units G11B20/18; for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS], H04L67/1097
• CPC Class Codes: G06F 11/076 by exceeding a count or rate limit, e.g. word- or bit count limit
• CPC Class Codes: G06F 11/201 between storage system components
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

An error detection mechanism is provided for detecting sequential and distributed errors in a device I/O stream. The sensitivity of the errors is user definable. The result of the error detection is fed back into the path management software, which may use the error information to decide whether a device path should be disabled. The error detection mechanism sets a time span for a time window and counts the number of errors that occur during the time window. Each time a time window ends with at least one error, the sequential error count and the distributed error count are incremented. However, if an I/O returns without an error, the sequential error count is cleared. If the sequential error count reaches a predetermined limit, the path is disabled. After a predetermined number of time windows, if the distributed error count reaches a predetermined limit, the path is disabled.

Claims

1. A method for detecting errors in a device path, the method comprising:

setting a time span for a time window based on a time to process a successful input/output command;

starting the time window;

responsive to the time window ending, determining whether at least one input/output error occurs on a device path during the time window;

responsive to one or more input/output errors occurring on the device path during the time window, incrementing an error count by one;

determining whether the error count reaches a predetermined limit; and

responsive to the error count reaching the predetermined limit, disabling the device path.

2. The method of claim 1, wherein the time span is dependent upon system loads and variations in transport speeds.

3. The method of claim 1, wherein monitoring a device input/output stream includes monitoring input/output retries, input/output errors, and a time to process an input/output command during the time window.

4. The method of claim 1, further comprising:

responsive to an input/output command returning successful, determining a new time span for the time window.

5. The method of claim 4, wherein determining a new time span for the time window includes:

determining an average time to process a good input/output operation.

6. The method of claim 1, further comprising:

responsive to an input/output command returning with a timestamp outside a current time window, starting a new time window.

7. The method of claim 6, further comprising:

setting a start time of the new time window equal to the timestamp of the input/output command.

8. The method of claim 1, wherein the error count is a sequential error count.

9. The method of claim 8, further comprising:

responsive to an input/output command returning successful, setting the sequential error count to zero.

10. The method of claim 1, wherein the error count is a distributed error count.

11. The method of claim 10, wherein determining whether the error count reaches a predetermined limit is performed responsive to a predetermined number of time windows being counted.

12. The method of claim 1, wherein determining whether one or more input/output errors occur on a device path during a time window is performed responsive to an input/output command returning with a timestamp outside a current time window.

24. The method of claim 1, wherein determining a new time span for the time window comprises:

determining if the time span equals zero;

responsive to the time span equaling zero, setting the time span to command time; and

responsive to the time span not equaling zero, setting the time span equal to (19* time span+command time)/20.

13. An apparatus for detecting errors in a device path, the apparatus comprising:

means for setting a time span for a time window based on a time to process a successful input/output command;

means for starting the time window;

means, responsive to the time window ending, for determining whether one or more input/output errors occur on a device path during a time window;

means, responsive to one or more input/output errors occurring on the device path during the time window, for incrementing an error count;

means for determining whether the error count reaches a predetermined limit; and

means, responsive to the error count reaching the predetermined limit, for disabling the device path.

14. The apparatus of claim 13, further comprising:

means, responsive to an input/output command returning successful, for determining a new time span for the time window.

15. The apparatus of claim 14, wherein the means for determining a new time span for the time window includes:

means for determining an average time to process a good input/output operation.

16. The apparatus of claim 13, further comprising:

means, responsive to an input/output command returning with a timestamp outside a current time window, for starting a new time window.

17. The apparatus of claim 16, further comprising:

means for setting a start time of the new time window equal to the timestamp of the input/output command.

18. The apparatus of claim 13, wherein the error count is a sequential error count.

19. The apparatus of claim 18, further comprising:

means, responsive to an input/output command returning successful, for setting the sequential error count to zero.

20. The apparatus of claim 13, wherein the error count is a distributed error count.

21. The apparatus of claim 20, wherein determining whether the error count reaches a predetermined limit is performed responsive to a predetermined number of time windows being counted.

22. The apparatus of claim 13, wherein determining whether at least one input/output error occurs on a device path during a time window is performed responsive to an input/output command returning with a timestamp outside a current time window.

25. The apparatus of claim 13, wherein means for determining a new time span for the time window further comprises:

means for determining if the time span equals zero;

means, responsive to the time span equaling zero, for setting to time span to command time; and

means, responsive to the time scan not equaling zero, for setting time span equal to (19* time span+command time)/20.

23. A computer program product, in a computer readable medium, for detecting errors in a device path, the computer program product comprising:

instructions for setting a time span for a time window based on a time to process a successful input/output command;

instructions for starting the time window;

instructions, responsive to the time window ending, for determining whether at least one input/output error occurs on a device path during the time window;

instructions, responsive to one or more input/output errors occurring on the device path during the time window, for incrementing an error count by one;

instructions for determining whether the error count reaches a predetermined limit; and

instructions, responsive to the error count reaching the predetermined limit, for disabling the device path.

26. The computer program product of claim 23, wherein means for determining a new time span for the time window further comprises:

instructions for determining if time span equals zero;

instructions, responsive to time span equaling zero, for setting the time span to command time; and

instructions, responsive to the time span not equaling zero, for setting time span equal to (19* time span+command time)/20.