Method and apparatus for monitoring computer systems during manufacturing, testing and in the field

• US 6,138,249 A
• Filed: 12/11/1997
• Issued: 10/24/2000
• Est. Priority Date: 12/11/1997
• Status: Expired due to Term

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: Nolet, James F.
• Primary Examiner(s): Beausoliel, Jr., Robert W.
• Assistant Examiner(s): Elisca, Pierre E.
• Application Number:
  US08/988,732
• Time in Patent Office: 1,048 Days
• Field of Search: 714/25, 714/4, 714/8, 714/18, 714/52, 714/48, 714/49, 714/43, 714/57, 714/38, 714/47, 714/36, 714/30, 714/31, 714/6, 700/95, 700/96, 700/101, 700/102, 700/103, 700/108
• US Class Current: 714/25
• CPC Class Codes: G06F 11/008 Reliability or availability analysis
• CPC Class Codes: G06F 11/2294 by remote test
• CPC Class Codes: H04L 41/069 using logs of notifications; Post-processing of notifications
• CPC Class Codes: H04L 43/00 Arrangements for monitoring or testing data switching networks
• CPC Class Codes: H04L 43/0817 by checking functioning
• CPC Class Codes: H04L 43/10 Active monitoring, e.g. heartbeat, ping or trace-route
• CPC Class Codes: H04L 43/106 using time related information in packets, e.g. by adding timestamps

Abstract

A method and apparatus for monitoring a plurality of data processing systems from a monitoring system. The data processing systems may be coupled to the monitoring system via a network cloud. When one of the plurality of data processing systems experiences a failure, the failure is detected at the monitoring system based upon communications over the network. The data processing systems may each have a service processor directly coupled to the network cloud. The monitoring system can also be employed to monitor the status of the data processing systems, either in a manufacture/test environment or in the field. The monitored status can include an inventory of parts for the data processing systems. Each part can be provided with identification information that is readable by the data processing system when the part is installed, and the identification information can be used to automatically generate an inventory of parts for each of the data processing systems. The monitoring system can also be used to automatically download an updated a piece of software to the data processing systems. In one aspect of the invention, bidirectional communication is employed between the monitoring system and the data processing systems. When an event occurs on the data processing system, the system sends a service request to the monitoring system notifying it of the event. The monitoring system also sends periodic communications to the data processing systems to ensure that each is functioning properly.

Claims

1. A method of monitoring a plurality of remote data processing systems installed at a plurality of remote customer sites from a local monitoring system disposed at a local customer service site to determine when any of the remote data processing systems experiences a failure, the method comprising steps of:

(A) coupling the plurality of remote data processing systems to the local monitoring system via a network cloud; and

(B) when one of the plurality of remote data processing systems experiences a failure, detecting the failure based upon communications over the network cloud between the one of the plurality of remote data processing systems at the remote customer site and the local monitoring system at the customer service site.

2. The method of claim 1, wherein step (B) includes steps, performed by the monitoring system, of:

periodically transmitting communications over the network cloud to the one of the plurality of data processing systems to determine whether the one of the plurality of data processing systems is capable of returning a responsive communication over the network cloud; and

when no responsive communication is received from the one of the plurality of data processing systems, determining that the failure has occurred in the one of the data processing systems.

3. The method of claim 2, wherein step (B) includes steps, performed by the one of the data processing systems, of:

detecting the failure; and

broadcasting a service request over the network cloud from the one of the plurality of data processing systems to the monitoring system, the service request indicating that the failure has occurred in the one of the plurality of data processing systems.

4. The method of claim 1, further including a step, performed by the monitoring system, of notifying service personnel of the failure in the one of the plurality of data processing systems.

5. The method of claim 1, wherein step (B) includes steps, performed by the one of the data processing systems, of:

detecting the failure;

storing information identifying a nature of the failure; and

broadcasting a service request over the network cloud from the one of the plurality of data processing systems to the monitoring system, the service request indicating that the failure has occurred in the one of the plurality of data processing systems.

6. The method of claim 5, wherein step (B) further includes steps of:

transmitting, over the network cloud, a response from the monitoring system to the service request indicating that the monitoring system is prepared to process the service request; and

transmitting, from the one of the plurality of data processing systems to the monitoring system, the stored information indicating the nature of the failure.

7. The method of claim 6, wherein the monitoring system includes a database, and wherein step (B) further includes a step of storing in the database the information indicating the nature of the failure.

8. The method of claim 5, wherein step (B) further includes a step of:

when a response is received from the monitoring system indicating that the monitoring system is prepared to process the service request, transmitting from the one of the plurality of data processing systems to the monitoring system the stored information indicating the nature of the failure.

9. The method of claim 1, wherein each of the plurality of data processing systems is a storage system that includes a plurality of disc drives.

10. The method of claim 9, wherein each of the plurality of data processing systems has a service processor, and wherein step (A) includes a step of directly coupling the service processor of each of the plurality of data processing systems to the network cloud.

11. The method of claim 1, wherein the monitoring system includes a database, and wherein step (B) further includes a step of storing in the database information indicating the nature of the failure.

12. The method of claim 1, wherein step (B) further includes steps, performed by the monitoring system, of periodically transmitting inquiries over the network cloud to the one of the plurality of data processing systems requesting information as to whether the one of the plurality of data processing systems has experienced a failure.

13. The method of claim 12, wherein step (B) further includes steps, performed by the one of the data processing systems, of:

detecting the failure;

storing information indicating the nature of the failure; and

responding to the inquiries by transmitting the stored information, indicating the nature of the failure, over the network cloud to the monitoring system.

14. The method of claim 1, wherein step (B) further includes steps, performed by the one of the data processing systems, of:

detecting the failure;

storing information indicating the nature of the failure; and

responding to periodic communications from the monitoring system by transmitting to the monitoring system the stored information indicating the nature of the failure.

15. The method of claim 1, wherein each of the plurality of data processing systems has a service processor, and wherein step (A) includes a step of directly coupling the service processor of each of the plurality of data processing systems to the network cloud.

16. The method of claim 1, wherein step (A) includes a step of coupling the plurality of data processing systems to the monitoring system via a network cloud that includes the Internet.

17. The method of claim 16, wherein each of the plurality of data processing systems is a storage system that includes a plurality of disc drives.

18. The method of claim 1, wherein step (B) includes steps, performed by the one of the data processing systems, of:

detecting the failure; and

broadcasting a service request over the network cloud from the one of the plurality of data processing systems to the monitoring system, the service request indicating that the failure has occurred in the one of the plurality of data processing systems.

19. A method of monitoring a plurality of data processing systems from a monitoring system to determine when any of the data processing systems experiences a failure, wherein the plurality of data processing systems and the monitoring system each is installed in a manufacture/test facility, the method comprising steps of:

(A) coupling the plurality of data processing systems to the monitoring system via a network cloud;

(B) executing a plurality of tests on each of the plurality of data processing systems to test the functional operation of the plurality of data processing systems, each one of the plurality of tests generating a failure when one of the plurality of data processing systems does not properly execute the one of the plurality of tests; and

(C) when one of the plurality of data processing systems experiences a failure, detecting the failure at the monitoring system based upon communications over the network cloud between the one of the plurality of data processing systems and the monitoring system.

20. The method of claim 19, wherein step (B) includes a step of:

when the one of the plurality of data processing systems experiences a failure, transferring information from the one of the plurality of data processing systems to the monitoring system through the network cloud, the information indicating to the monitoring system that the one of the plurality of data processing systems has experienced the failure.

21. The method of claim 19, wherein step (B) includes steps, performed by the one of the data processing systems, of:

detecting the failure; and

broadcasting a service request over the network cloud from the one of the plurality of data processing systems to the monitoring system, the service request indicating that the failure has occurred in the one of the plurality of data processing systems.

22. The method of claim 19, wherein each of the plurality of data processing systems has a service processor, and wherein step (A) includes a step of directly coupling the service processor of each of the plurality of data processing systems to the network cloud.

23. The method of claim 19, wherein the monitoring system includes a database, and wherein step (B) further includes a step of storing in the database information indicating the nature of the failures of each of the plurality of data processing systems in the manufacture/test facility.

24. The method of claim 19, wherein step (A) includes a step of coupling the plurality of data processing systems to the monitoring system via a network cloud that includes the Internet.

25. The method of claim 19, wherein step (B) includes steps, performed by the one of the data processing systems, of:

detecting the failure;

storing information identifying a nature of the failure; and

broadcasting a service request over the network cloud from the one of the plurality of data processing systems to the monitoring system, the service request indicating that the failure has occurred in the one of the plurality of data processing systems.

26. The method of claim 25, wherein step (B) further includes steps of:

transmitting, over the network cloud, a response from the monitoring system to the service request indicating that the monitoring system is prepared to process the service request; and

transmitting, from the one of the plurality of data processing systems to the monitoring system, the stored information indicating the nature of the failure.

27. The method of claim 19, wherein step (B) includes steps, performed by the monitoring system, of:

periodically transmitting communications over the network cloud to the one of the plurality of data processing systems to determine whether the one of the plurality of data processing systems is capable of returning a responsive communication over the network cloud; and

when no responsive communication is received from the one of the plurality of data processing systems, determining that the failure has occurred in the one of the data processing systems.

28. The method of claim 27, wherein step (B) includes steps, performed by the one of the data processing systems, of:

detecting the failure; and

broadcasting a service request over the network cloud from the one of the plurality of data processing systems to the monitoring system, the service request indicating that the failure has occurred in the one of the plurality of data processing systems.

29. The method of claim 19, wherein step (B) further includes steps, performed by the monitoring system, of periodically transmitting inquiries over the network cloud to the one of the plurality of data processing systems requesting information as to whether the one of the plurality of data processing systems has experienced a failure.

30. The method of claim 29, wherein step (B) further includes steps, performed by the one of the data processing systems, of:

detecting the failure;

storing information indicating the nature of the failure; and

responding to the inquiries by transmitting the stored information, indicating the nature of the failure, over the network cloud to the monitoring system.

31. The method of claim 19, wherein each of the plurality of data processing systems is a storage system that includes a plurality of disc drives.

32. The method of claim 31, wherein each of the plurality of data processing systems has a service processor, and wherein step (A) includes a step of directly coupling the service processor of each of the plurality of data processing systems to the network cloud.

33. An apparatus, comprising:

a network cloud;

a plurality of remote data processing systems installed at a plurality of remote customer sites and coupled to the network cloud; and

a local monitoring system disposed at a local customer service site and coupled to the network cloud, wherein the local monitoring system monitors the plurality of remote data processing systems to determine when any of the remote data processing systems experiences a failure, and wherein the local monitoring system detects a failure in one of the remote data processing systems based upon communications over the network cloud between the one of the plurality of remote data processing systems and the local monitoring system.

34. The apparatus of claim 33, wherein each one of the data processing systems includes:

means for detecting a failure in the one of the data processing systems; and

means for, when a failure is detected, broadcasting a service request over the network cloud to the monitoring system indicating that the failure has occurred.

35. The apparatus of claim 34, wherein the monitoring system includes a plurality of servers capable of responding to each service request broadcast over the network by one of the plurality of data processing systems.

36. The apparatus of claim 34, wherein each of the data processing systems broadcasts a different type of service request for different types of failures, and wherein the monitoring system includes a plurality of servers capable of responding to each type of service request broadcast over the network by one of the plurality of data processing systems.

37. The apparatus of claim 36, wherein the apparatus includes means for selecting a one of the plurality of servers capable of responding to each type of service request most efficiently.

38. The apparatus of claim 33, wherein the monitoring system includes:

means for periodically transmitting communications over the network cloud to each of the plurality of data processing systems to determine whether each of the plurality of data processing systems is capable of returning a responsive communication over the network cloud; and

means for determining that the failure has occurred in one of the data processing systems when no responsive communication is received from the one of the plurality of data processing systems.

39. The apparatus of claim 38, wherein each one of the data processing systems includes:

means for detecting a failure in the one of the data processing systems; and

means for, when a failure is detected, broadcasting a service request over the network cloud to the monitoring system indicating that the failure has occurred.

40. The apparatus of claim 33, wherein the monitoring system includes means for notifying service personnel when a failure is detected in one of the plurality of data processing systems.

41. The apparatus of claim 33, wherein each of the plurality of data processing systems has a service processor directly coupled to the network cloud.

42. The apparatus of claim 41, wherein the monitoring system includes a database and means for storing in the database the information indicating the nature of each failure.

43. The apparatus of claim 33, wherein the monitoring system includes a database and means for storing in the database information indicating the nature of each failure.

44. The apparatus of claim 33, wherein the monitoring system further includes polling means for periodically transmitting inquiries over the network cloud to each one of the plurality of data processing systems requesting information as to whether the one of the plurality of data processing systems has experienced a failure.

45. The apparatus of claim 44, wherein each one of the data processing systems includes:

means for detecting a failure in the one of the data processing systems;

means for storing information indicating the nature of each failure; and

means, responsive to the inquiries, for transmitting the stored information that indicates the nature of the failure over the network cloud to the monitoring system.

46. The apparatus of claim 33, wherein the network cloud includes the Internet.

47. The apparatus of claim 46, wherein each of the plurality of data processing systems is a storage system that includes a plurality of disc drives.

48. The apparatus of claim 33, wherein each one of the data processing systems includes:

means for detecting a failure in the one of the data processing systems;

means for storing information identifying a nature of each failure; and

means for, when a failure is detected, broadcasting a service request over the network cloud to the monitoring system indicating that the failure has occurred.

49. The apparatus of claim 48, wherein each of the plurality of data processing systems includes means for, when a response is received from the monitoring system indicating that the monitoring system is prepared to process the service request, transmitting from the one of the plurality of data processing systems to the monitoring system the stored information indicating the nature of the failure.

50. The apparatus of claim 33, wherein each of the plurality of data processing systems is a storage system that includes a plurality of disc drives.

51. The apparatus of claim 50, wherein each of the plurality of data processing systems has a service processor directly coupled to the network cloud.

52. An apparatus comprising:

a network cloud;

a plurality of data processing systems installed in a manufacture/test facility coupled to the network cloud, each one of the plurality of data processing systems executing a plurality of tests to test the functional operation of the one of the plurality of data processing systems, each one of the plurality of tests generating a failure when the one of the plurality of data processing systems does not properly execute the one of the plurality of tests; and

a monitoring system, coupled to the network cloud, to monitor the plurality of data processing systems to determine when any of the data processing systems experiences a failure, wherein the monitoring system detects a failure in one of the data processing systems based upon communications over the network cloud between the one of the plurality of data processing systems and the monitoring system.

53. The apparatus of claim 52, further including means for, when one of the plurality of data processing systems experiences a failure, transferring information from the one of the plurality of data processing systems to the monitoring system through the network cloud, the information indicating to the monitoring system that the one of the plurality of data processing systems has experienced the failure.

54. The apparatus of claim 52, wherein each of the plurality of data processing systems has a service processor directly coupled to the network cloud.

55. The apparatus of claim 52, wherein each one of the data processing systems includes:

means for detecting a failure in the one of the data processing systems;

means for storing information identifying a nature of each failure; and

means for, when a failure is detected, broadcasting a service request over the network cloud to the monitoring system indicating that the failure has occurred.

56. The apparatus of claim 55, wherein each of the plurality of data processing systems includes means for, when a response is received from the monitoring system indicating that the monitoring system is prepared to process the service request, transmitting from the one of the plurality of data processing systems to the monitoring system the stored information indicating the nature of the failure.

57. The apparatus of claim 52, wherein the monitoring system includes a database, and wherein the apparatus further includes means for storing in the database information indicating the nature of the failures of each of the plurality of data processing systems in the manufacture/test facility.

58. The apparatus of claim 52, wherein each of the plurality of data processing systems is a storage system that includes a plurality of disc drives.

59. The apparatus of claim 58, wherein each of plurality of data processing systems has a service processor directly coupled to the network cloud.

60. The apparatus of claim 52, wherein the network cloud includes the Internet.

61. The apparatus of claim 52, wherein the monitoring system includes:

means for periodically transmitting communications over the network cloud to each of the plurality of data processing systems to determine whether each of the plurality of data processing systems is capable of returning a responsive communication over the network cloud; and

means for determining that the failure has occurred in one of the data processing systems when no responsive communication is received from the one of the plurality of data processing systems.

62. The apparatus of claim 61, wherein each one of the data processing systems includes:

means for detecting a failure in the one of the data processing systems; and

means for, when a failure is detected, broadcasting a service request over the network cloud to the monitoring system indicating that the failure has occurred.

63. The apparatus of claim 52, wherein each one of the data processing systems includes:

means for detecting a failure in the one of the data processing systems; and

means for, when a failure is detected, broadcasting a service request over the network cloud to the monitoring system indicating that the failure has occurred.

64. The apparatus of claim 52, wherein the monitoring system further includes polling means for periodically transmitting inquiries over the network cloud to each one of the plurality of data processing systems requesting information as to whether the one of the plurality of data processing systems has experienced a failure.

65. The apparatus of claim 64, wherein each one of the data processing systems includes:

means for detecting a failure in the one of the data processing systems;

means for storing information indicating the nature of each failure; and

means, responsive to the inquiries, for transmitting the stored information that indicates the nature of the failure over the network cloud to the monitoring system.

66. A method of automatically downloading an updated a piece of software to a plurality of data processing systems, the plurality of data processing systems each being coupled to a service center, the method comprising steps of:

(A) providing the updated piece of software on the service center;

(B) periodically receiving service requests from each of the plurality of data processing systems, each service request including information from which a determination can be made as to whether the data processing system that transmitted the request has a copy of the updated piece of software;

(C) in response to the service requests, automatically determining which of the plurality of data processing systems do not have a copy of the updated piece of software; and

(D) automatically downloading a copy of the updated piece of software to the data processing systems that do not have a copy of the updated piece of software.

67. The method of claim 66, wherein the plurality of data processing systems is coupled to the service center via a network cloud, and wherein step (D) includes a step of automatically downloading a copy of the updated piece of software over the network cloud.

68. The method of claim 66, wherein steps (B)-(D) execute automatically, without any operator participation.

69. A method of using a monitoring system to monitor the status of a plurality of data processing systems in a manufacture/test facility, the method comprising steps of:

(A) executing a plurality of tests on each of the plurality of data processing systems to test the functional operation of the plurality of data processing systems, each one of the plurality of tests generating a failure when one of the plurality of data processing systems does not properly execute the one of the plurality of tests;

(B) when a failing one of the plurality of data processing systems experiences a failure;

storing information in the failing one of the plurality of data processing systems identifying a nature of the failure; and

broadcasting a service request from the failing one of the plurality of data processing systems to the monitoring system, the service request indicating that the failure has occurred; and

(C) storing information in the monitoring system to record the failure in response to information provided by the failing one of the plurality of data processing systems.

70. The method of claim 69, wherein step (C) further includes steps of:

tranmsitting a response from the monitoring system to the service request indicating that the monitoring system is prepared to process the service request;

receiving, from the failing one of the plurality of data processing systems, the stored information indicating the nature of the failure; and

storing information in the monitoring system to record the failure based upon the stored information indicating the nature of the failure.

71. The method of claim 70, further including steps, performed by the monitoring system, of:

periodically transmitting communications to each of plurality of data processing systems to determine whether each of the plurality of data processing systems is capable of returning a responsive communication over the network cloud; and

when no responsive communication is received from one of the plurality of data processing systems, determining that a failure has occurred in the one of the data processing systems.

72. A method of using a monitoring system to monitor the status of a plurality of data processing systems in a manufacture/test facility, the method comprising steps of:

(A) executing a plurality of tests on each of the plurality of data processing systems to test the functional operation of the plurality of data processing systems, each one of the plurality of tests generating a failure when one of the plurality of data processing systems does not properly execute the one of the plurality of tests;

(B) periodically transmitting inquiries from the monitoring system to each of the plurality of data processing systems requesting information as to whether the one of the plurality of data processing systems has experienced a failure; and

(C) when a failing one of the plurality of data processing systems experiences a failure;

storing information in the failing one of the plurality of data processing systems identifying a nature of the failure; and

responding to one of the periodic inquiries by transmitting the stored information that indicates the nature of the failure to the monitoring system.

73. A method of monitoring a plurality of data processing systems from a monitoring system to determine when any of the data processing systems experiences a failure, the method comprising steps of:

(A) coupling the plurality of data processing systems to the monitoring system via a network cloud; and

(B) when one of the plurality of data processing systems experiences a failure, detecting the failure at the monitoring system based upon communications over the network cloud between the one of the plurality of data processing systems and the monitoring system,

wherein step (B) further includes steps of;

(B1) detecting the failure at the one of the data processing systems, and broadcasting a service request over the network cloud from the one of the plurality of data processing systems to the monitoring system, the service request indicating that the failure has occurred in the one of the plurality of data processing systems; and

(B2) periodically transmitting communications over the network cloud from the monitoring system to the one of the plurality of data processing systems to determine whether the one of the plurality of data processing systems is capable of returning a responsive communication over the network cloud, and when no responsive communication is received from the one of the plurality of data processing systems, determining that the failure has occurred in the one of the data processing systems.