Method, apparatus, and program for detecting sequential and distributed path errors in MPIO

• US 20050154797A1
• Filed: 11/20/2003
• Published: 07/14/2005
• Est. Priority Date: 11/20/2003
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Tee, Stephen M., Tinnakul, Teerasit, Allen, James Patrick, Damron, Timothy M.
• Granted Patent: US 7,076,573 B2
• Time in Patent Office: Days
• US Class Current: 710/1
• CPC Class Codes: G06F 11/0727 in a storage system, e.g. in a DASD or network based storage system drivers for digital recording or reproducing units G06F3/06; circuits for error detection or correction within digital recording or reproducing units G11B20/18; for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS], H04L67/1097
• CPC Class Codes: G06F 11/076 by exceeding a count or rate limit, e.g. word- or bit count limit
• CPC Class Codes: G06F 11/201 between storage system components
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

An error detection mechanism is provided for detecting sequential and distributed errors in a device I/O stream. The sensitivity of the errors is user definable. The result of the error detection is fed back into the path management software, which may use the error information to decide whether a device path should be disabled. The error detection mechanism sets a time span for a time window and counts the number of errors that occur during the time window. Each time a time window ends with at least one error, the sequential error count and the distributed error count are incremented. However, if an I/O returns without an error, the sequential error count is cleared. If the sequential error count reaches a predetermined limit, the path is disabled. After a predetermined number of time windows, if the distributed error count reaches a predetermined limit, the path is disabled.

Claims

1. A method for detecting errors in a device path, the method comprising:

setting a time span for a time window based on a time to process a successful input/output command; and

monitoring a device input/output stream during the time window to detect sequential and distributed errors having a sensitivity that is user definable.

2. The method of claim 1, wherein the time span is dependent upon system loads and variations in transport speeds.

3. The method of claim 1, wherein monitoring a device input/output stream includes monitoring input/output retries, input/output errors, and a time to process an input/output command during the time window.

4. A method for detecting errors in a device path, the method comprising:

setting a time span for a time window;

starting the time window;

responsive to the time window ending, determining whether at least one input/output error occurs on a device path during the time window; and

responsive to one or more input/output errors occurring on the device path during the time window, incrementing an error count by one.

5. The method of claim 4, further comprising:

responsive to an input/output command returning successful, determining a new time span for the time window.

6. The method of claim 5, wherein determining a new time span for the time window includes:

determining an average time to process a good input/output operation.

7. The method of claim 4, further comprising:

responsive to an input/output command returning with a timestamp outside a current time window, starting a new time window.

8. The method of claim 7, further comprising:

setting a start time of the new time window equal to the timestamp of the input/output command.

9. The method of claim 4, further comprising:

determining whether the error count reaches a predetermined limit; and

responsive to the error count reaching the predetermined limit, failing the device path.

10. The method of claim 9, wherein the error count is a sequential error count.

11. The method of claim 10, further comprising:

responsive to an input/output command returning successful, setting the sequential error count to zero.

12. The method of claim 9, wherein the error count is a distributed error count.

13. The method of claim 12, wherein determining whether the error count reaches a predetermined limit is performed responsive to a predetermined number of time windows being counted.

14. The method of claim 4, wherein determining whether one or more input/output errors occur on a device path during a time window is performed responsive to an input/output command returning with a timestamp outside a current time window.

15. An apparatus for detecting errors in a device path, the apparatus comprising:

means for setting a time span for a time window;

means for starting the time window;

means, responsive to the time window ending, for determining whether one or more input/output errors occur on a device path during a time window; and

means, responsive to one or more input/output errors occurring on the device path during the time window, for incrementing an error count.

16. The apparatus of claim 15, further comprising:

means, responsive to an input/output command returning successful, for determining a new time span for the time window.

17. The apparatus of claim 16, wherein the means for determining a new time span for the time window includes:

means for determining an average time to process a good input/output operation.

18. The apparatus of claim 15, further comprising:

means, responsive to an input/output command returning with a timestamp outside a current time window, for starting a new time window.

19. The apparatus of claim 18, further comprising:

means for setting a start time of the new time window equal to the timestamp of the input/output command.

20. The apparatus of claim 15, further comprising:

means for determining whether the error count reaches a predetermined limit; and

means, responsive to the error count reaching the predetermined limit, for failing the device path.

21. The apparatus of claim 20, wherein the error count is a sequential error count.

22. The apparatus of claim 21, further comprising:

means, responsive to an input/output command returning successful, for setting the sequential error count to zero.

23. The apparatus of claim 20, wherein the error count is a distributed error count.

24. The apparatus of claim 23, wherein determining whether the error count reaches a predetermined limit is performed responsive to a predetermined number of time windows being counted.

25. The apparatus of claim 15, wherein determining whether at least one input/output error occurs on a device path during a time window is performed responsive to an input/output command returning with a timestamp outside a current time window.

26. A computer program product, in a computer readable medium, for detecting errors in a device path, the computer program product comprising:

instructions for setting a time span for a time window;

instructions for starting the time window;

instructions, responsive to the time window ending, for determining whether at least one input/output error occurs on a device path during the time window; and

instructions, responsive to one or more input/output errors occurring on the device path during the time window, for incrementing an error count by one.

27. The computer program product of claim 26, further comprising:

instructions for determining whether the error count reaches a predetermined limit; and

instructions, responsive to the error count reaching the predetermined limit, for failing the device path.