Remote maintenance monitoring system

• US 5,123,017 A
• Filed: 09/29/1989
• Issued: 06/16/1992
• Est. Priority Date: 09/29/1989
• Status: Expired due to Fees

• Current Assignee: United States Of America As Represented By The Administrator Of The National Aeronautics And Space Administration
• Original Assignee: United States Of America As Represented By The Administrator Of The National Aeronautics And Space Administration
• Inventors: Rochette, Donn A., Simpkins, Lorenz G., Owens, Richard C.
• Primary Examiner(s): Ruggiero, Joseph
• Assistant Examiner(s): Cass, Russell
• Application Number:
  US07/414,817
• Time in Patent Office: 991 Days
• Field of Search: 371/15.1, 371/29.1, 371/11, 371/20.1, 364/513
• US Class Current: 714/26
• CPC Class Codes: G06F 11/0754 by exceeding limits
• CPC Class Codes: G06F 11/10 Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's
• CPC Class Codes: G06F 11/2257 using expert systems
• CPC Class Codes: G06F 11/2294 by remote test
• CPC Class Codes: G06F 11/3006 where the computing system is distributed, e.g. networked systems, clusters, multiprocessor systems multiprogramming arrangements G06F9/46; allocation of resources G06F9/50
• CPC Class Codes: G06F 11/3058 Monitoring arrangements for monitoring environmental properties or parameters of the computing system or of the computing system component, e.g. monitoring of power, currents, temperature, humidity, position, vibrations thermal management in cooling arrangements of a computing system G06F1/206
• CPC Class Codes: Y10S 706/911 Nonmedical diagnostics

Abstract

A remote maintenance monitoring system retrofits to a given hardware device with a sensor implant which gathers and captures failure data from the hardware device, without interfering with its operation. Failure data is continuously obtained from predetermined critical points within the hardware device, and is analyzed with a diagnostic expert system, which isolates failure origin to a particular component within the hardware device. For example, monitoring of a computer-based device may include monitoring of parity error data therefrom, as well as monitoring power supply fluctuations therein, so that parity error and power supply anomaly data may be used to trace the failure origin to a particular plane or power supply within the computer-based device. A plurality of sensor implants may be rerofit to corresponding plural devices comprising a distributed large-scale system. Transparent interface of the sensors to the devices precludes operative interference with the distributed network. Retrofit capability of the sensors permits monitoring of even older devices having no built-in testing technology. Continuous real time monitoring of a distributed network of such devices, coupled with diagnostic expert system analysis thereof, permits capture and analysis of even intermittent failures, thereby facilitating maintenance of the monitored large-scale system.

Claims

1. A remote maintenance monitoring system for monitoring an electronic hardware device, said system comprising:

non-intrusive sensor implant means, transparently interfaced with a given electronic hardware device so as to not interfere with the operation thereof, for continuously gathering operational data from preselected locations within such device; and

diagnostic expert means for receiving and analyzing data gathered by said sensor implant means, said expert means being programmed for diagnosing failure causes of the given device based on operational data gathered therefrom;

whereby relatively remote diagnostic analysis is achieved using failure data captured near its point of origin;

wherein said sensor implant means is adapted to be retrofit to an existing hardward device, for continuously monitoring and gathering data from such device in real time so as to capture data of even intermittent failures of such device.

2. A remote maintenance monitoring system as in claim 1, further including:

a plurality of said sensor implant means, respectively associated with a plurality of hardware devices, for real-time monitoring of a distributed network of such devices; and

wherein

such devices comprise computer-based and noncomputer-based types of hardware.

3. A remote maintenance monitoring system as in claim 2, wherein said diagnostic expert means comprises a hybrid model-based/rule-based system for diagnosing multiple, dependent failures in such plurality of devices.

4. A remote maintenance monitoring system as in claim 2, further including:

data interface and multiplexer means for respectively interconnecting each of said plurality of sensor implant means with said diagnostic means; and

wherein

each of said sensor implant means interfaced with a computer-based hardware device respectively includes a microprocessor-based component having plural digital and analog monitoring ports for respectively receiving memory parity error signals and power supply levels from its corresponding computer-based device, through capacitive coupling with appropriate edge connectors associated with such device, and wherein each such sensor implant means further includes a trace buffer for storing a memory dump from its corresponding computer-based device upon the occurrence of a failure thereof.

5. A remote maintenance monitoring system as in claim 1, wherein said expert system includes a forwardchaining interface engine with a conflict resolution mechanism and rules for operating on and diagnosing a given set of captured failure data for a respective monitored device, so as to isolate failure cause thereof to a line-replaceable-unit level therein.

6. A remote maintenance monitoring system for monitoring an electronic hardware device so as to achieve relatively remote diagnostic analysis using failure data captured near its point of origin, said system comprising:

non-intrusive sensor implant means, transparently interfaced with a given electronic hardware device so as to not interfere with the operation thereof, for continuously gathering operational data from preselected locations within such device, and

diagnostic expert means for receiving and analyzing data gathered by said sensor implant means, said expert means being programmed for diagnosing failure causes of the given device based on operational data gathered therefrom;

wherein

said sensor implant means is adapted for interface with a computer-based hardware device, for gathering data of parity errors and power supply anomalies occurring within a given such computer-based device to which it is interfaced; and

said diagnostic expert means utilizes such parity error and power supply anomaly data for determining the cause of such errors and anomalies in such computer-based device, and isolating the origin thereof to a line-replaceable-unit level within such device.

7. A remote maintenance monitoring system as in claim 6, wherein said expert system includes a Tektronix 4404 artificial intelligence computer programmed with Lisp.

8. A remote maintenance monitoring system as in claim 6, wherein said expert system analyzes said parity error and power supply anomaly data by tracing the parity error signals back through the corresponding computer-based device, correlated with the specific power supply fluctuations therein, to isolate the source of the original hardware problem to a memory plane, a CPU plane, a respective power supply thereof, or other specific replaceable component of such device.

9. A remote maintenance monitoring system for online hardware monitoring of an operational large-scale distributed computer system having a plurality of both computer-based and non-computer-based devices, said monitoring system comprising:

a plurality of data collection probes, respectively associated with the plurality of computer-based and non-computer-based devices comprising such distributed system, for transparently monitoring same and outputting corresponding failure data therefrom, respectively;

maintenance computer means, including an expert system using a multi-phased interactive approach, for monitoring and analyzing such failure data so as to locate specific failed hardware components in such respective devices, to permit time-effective replacement of such specific components; and

data interface and multiplexer means, for interfacing between said maintenance computer means and said plurality of data collection probes, respectively, said interface and multiplexer means including buffer means for receiving failure data from said probes for subsequent analysis by said maintenance computer means.

10. A remote maintenance monitoring system as in claim 9, wherein said data collection probes continuously monitor in real time their respectively associated distributed system devices, so as to capture intermittent failure data as well as hard failure data relative such devices.

11. A remote maintenance monitoring system as in claim 9, wherein each of said data collection probes comprises a microprocessor having plural digital monitoring ports, and plural analog monitoring ports with programmable over and under voltage limits.

12. A remote maintenance monitoring system as in claim 11, wherein:

said microprocessor comprises a Z8002B-based 16-bit microprocessor, said digital monitoring ports are respectively associated with memory plane parity error signals from an associated device for monitoring intermittent memory parity errors, and said analog monitoring ports are respectively associated with power supply points in such associated device for monitoring power supply fluctuations greater than said voltage limits; and

wherein

said data collection probes each further include a data trace buffer for capturing memory dump data at failure to permit subsequent trace analysis thereof; and

further wherein

said expert system includes a forward-chaining inference engine with a conflict resolution mechanism and rules for trace analyzing said buffer-captured data and data received at said digital and analog monitoring ports.

13. A remote maintenance monitoring system as in claim 9, wherein said data collection probes are adapted for capturing memory dump data, including address and instruction trace data, during and at least for a given time subsequent to, a failure occurrence within its associated device.

14. A remote maintenance monitoring system as in claim 9, wherein said maintenance computer means expert system incorporates therein integrated diagnostics adapted for predicting failures in the large-scale distributed computer system being monitored, with such predicting being based on analyzed correlations of monitored cumulative hardware degradation among the respective plural devices comprising such large-scale system, and hardware failures of the large-scale system.

15. A remote maintenance monitoring system as in claim 9, wherein said data collection probes are adapted to be retrofit respectively to existing hardware devices, for continuously monitoring and gathering data from such devices in real time so as to capture data of even intermittent failures of such devices.

16. A retrofit monitoring system for assisting integrated diagnostic maintenance of a large-scale computer system having a plurality of distributed hardware devices, said monitoring system comprising:

a plurality of data acquisition modules, respectively associated with the plurality of distributed hardware devices, for performing continuous, real-time monitoring of such devices and capturing intermittent and hard failure data therefrom, without interfering with operational integrity of such devices;

data interface and multiplexer means, respectively interconnected with each of said plurality of data acquisition modules, for collecting failure data from said plurality of modules into a centralized failure data analysis location relatively remote from the respective hardware devices; and

a diagnostic expert system, situated relatively near said centralized failure data analysis location, for receiving and analyzing collected failure data so as to determine and isolate hardware failures within each respective hardware device to specific replaceable components thereof, whereby multiple, dependent failures in such large-scale, distributed-device system are analyzed and isolated to facilitate replacement of such specific hardware components, thereby resulting in automated, time-effective maintenance of such largescale system.

17. A retrofit monitoring system as in claim 16, wherein:

said data acquisition modules each comprise microprocessor-based sensor implant means, having respective plural analog and digital monitoring ports for continuously monitoring critical point analog and digital signals, respectively, within its associated hardware device, and further having a data trace buffer for capturing memory dump failure data for its associated hardware device to permit subsequent component-isolation analysis thereof; and

wherein

said expert system includes a hybrid modelbased/rule-based system for analyzing and diagnosing such plural failure data and such analog and digital signals.

18. A retrofit monitoring system as in claim 16, wherein:

the large-scale computer system being maintained includes over 200 distributed computer-based devices; and

said data acquisition modules associated with such computer-based devices non-intrusively monitor both digital and analog signals thereof, associated with memory parity error failure signals generated by its associated computer-based device, and analog signals thereof, corresponding to power supply levels therein so as to detect power fluctuations influencing various hardware components of such associated computer-based device;

whereby capturing of such memory parity errors and power supply fluctuations permits parity error trace analysis thereof for the isolation of hardware failures to particular line-replaceable components of such associated computer-based device.

19. A retrofit monitoring system as in claim 16, wherein said expert system is adapted for predicting failures in the large-scale computer system being maintained, based on monitoring of the plurality of distributed devices comprising such large-scale system, and correlating data from such devices with computer system failure experience.

20. A retrofit monitoring system as in claim 16, further including a plurality of terminals and peripheral devices, such as printers and other graphics output devices, associated with said diagnostic expert system for facilitating user interface therewith.