System and method for testing computer hardware and software

• US 5,157,782 A
• Filed: 01/31/1990
• Issued: 10/20/1992
• Est. Priority Date: 01/31/1990
• Status: Expired due to Fees

• Current Assignee: Hewlett-Packard Company (HP Inc.)
• Original Assignee: Hewlett-Packard Company (HP Inc.)
• Inventors: Tuttle, Myron R., Low, Danny
• Primary Examiner(s): Baker, Stephen M.
• Assistant Examiner(s): Hua, Ly V.
• Application Number:
  US07/472,697
• Time in Patent Office: 993 Days
• Field of Search: 364/200 MS File, 364/900 MS File, 371/67.1, 371/22.4, 371/25.1, 371/24, 371/71, 371/19, 371/15.1, 340/715
• US Class Current: 714/45
• CPC Class Codes: G06F 11/22 Detection or location of defective computer hardware by testing during standby operation or during idle time, e.g. start-up testing
• CPC Class Codes: G06F 11/32 with visual or acoustical indication of the functioning of the machine
• CPC Class Codes: G06F 11/3688 for test execution, e.g. scheduling of test suites
• CPC Class Codes: G06F 2201/83 the solution involving signatures

Abstract

A computer related system and method uses regression testing techniques for testing computer hardware and/or software application(s). Input data and commands from a user are stored on a host, and sent to an interface device, which then sends them to a hardware/software system under test. Visual display data on a visual display device is thereby affected. Signatures (representative of the visual display data) are generated by the interface device as a result of input data and commands sent to the interface device and system under test. These signatures are received and stored by the host. On command of a user, the stored signatures, input data and commands are subsequently sent to the hardware/software system under test, and new signatures are generated. These new signatures are compared with the stored signatures, and the results of this comparison are used as an indication that the hardware/software system under test is performing as expected.

Claims

1. A system for testing a system under test (SUT) having at least one of computer hardware and software applications residing thereon, the SUT having a software application responsive to stored input data and a visual display device responsive to the software application for generating visual display signals, the testing system comprising:

(a) a host with a memory having the stored input data and having stored signatures of visual display signals; and

,

(b) an interface device, connected to said host and to the SUT, comprising;

(1) means for sending the stored input data from said host to the SUT,

(2) means for comparing new signatures of visual display signals from the visual display device with said stored signatures of visual display signals from said host in a non-intrusive manner, and

(3) means for indicating whether the SUT is functioning properly.

2. The system of claim 1, wherein said host comprises means for sending the stored input data and said stored signature of visual display signals to said interface device during a playback mode.

3. The system of claim 1, wherein said host further comprises a software module, and

wherein said software module causes said host to retrieve said stored signature of visual display signals from said memory during a playback mode.

4. The system of claim 1, wherein said host further comprises a software module, and

wherein said software module causes said host to retrieve the stored input data from said memory during a playback mode.

5. The system of claim 1, wherein said memory comprises means for storing DVPU/Host commands.

6. The system of claim 5, wherein said host further comprises a software module, and

wherein said software module causes said host to retrieve said stored DVPU/Host commands from said memory during a playback mode.

7. The system of claim 1, wherein said interface device further comprises a micro-controller sub-system under firmware and hardware control.

8. The system of claim 7, wherein said firmware of said micro-controller sub-system comprises a routine for performing a comparison between said stored visual display signals and said new visual display signals.

9. The system of claim 7, wherein said interface device further comprises a video signature analyzer sub-system.

10. The system of claim 9, wherein said firmware of said micro-controller sub-system comprises a routine for instructing said video signature analyzer sub-system to capture in a non-intrusive manner the visual display signals from the visual display device.

11. A system for testing a system under test (SUT) having at least one of computer hardware and software applications residing thereon, the SUT having a software application responsive to input data and a visual display device responsive to the software application for generating visual display signals, the testing system comprising:

(a) a host for receiving first input data from an input device, said host comprising,

(1) first means for storing said first input data as stored input data,

(2) second means for sending said first input data to an interface device during a record mode,

(3) third means for sending a take-signature command to said interface device during said record mode, and

(4) fourth means for storing captured signatures of visual display signals as stored signatures of visual display signals; and

(b) said interface device, being connected to said host and to the SUT for sending said first input data to the software application of the SUT for capturing signatures of the visual display signals from the visual display device, said interface device comprising,

(1) fifth means for capturing in a non-intrusive manner signatures of the visual display signals of the visual display device in response to said take-signature command, and

(2) sixth means, responsive to said fifth means, for sending captured signatures of visual display signals to said fourth means for storage as said stored signatures of visual display signals.

12. The system of claim 11, wherein said host comprises a microcomputer.

13. The system of claim 11, wherein said host has a software module controlling said first means, second means, third means, and/or fourth means during said record mode.

14. The system of claim 11, wherein said interface device further comprises a micro-controller sub-system under firmware and hardware control for controlling said fifth means and/or sixth means during said record mode.

15. A system for testing a system under test (SUT) having at least one of computer hardware and software applications residing thereon, the SUT having a software application responsive to stored input data and a visual display device responsive to the software application, the testing system comprising:

(a) a host with a memory having the stored input data and having first stored signatures of visual display signals; and

(b) an interface device, connected to said host and to the SUT, comprising;

(1) means for sending the stored input data from said host to said SUT,

(2) means for capturing in a non-intrusive manner second signatures of visual display signals from the visual display device, and

(3) means for sending said second signatures of visual display signals to said memory on said host for storage.

16. A system for testing a system under test (SUT) having at least one of computer hardware and software applications residing thereon, the SUT having a software application responsive to stored input data and a visual display device responsive to the software application for generating visual display signals, the testing system comprising:

(a) a host for receiving the first input data from an input device, said host comprising;

(1) first means for storing said first input data as stored input data,

(2) second means for sending said first input data to an interface device during a playback mode,

(3) third means for sending the first input data to said interface device during a record mode,

(4) fourth means for sending a take-signature command to said interface device during said record mode, and

(5) fifth means for storing captured signatures of visual display signals as stored signatures of visual display signals during said record mode; and

,

(b) said interface device, being connected to said host and to the SUT, which sends during said record mode the first input data to the software application of the SUT, which sends during said playback mode the stored input data, and which captures during said playback mode and said record mode signatures of the visual display signals from the visual display device, said interface device comprising,

(1) sixth means for capturing in a non-intrusive manner signatures of the visual display signals of the visual display device in response to said take-signature command during said record mode,

(2) seventh means for capturing in a non-intrusive manner signatures of the visual display signals of the visual display device during said playback mode,

(3) eighth means, responsive to said sixth means, for sending said captured signatures of visual display signals to said fifth means for storage as the stored signatures of visual display signals, and

(4) ninth means, responsive to said seventh means and to said second means, for comparing said stored signatures of the visual display signals with captured signatures of the visual display signals, so as to provide an indication as to whether the SUT is functioning properly.

17. The system of claim 16, wherein said host has a software module controlling said first means, third means, fourth means and/or fifth means during said record mode.

18. The system of claim 16, wherein said interface device further comprises means for generating a compact representation of the visual display signals.

19. The system of claim 16, wherein said interface device further comprises means for emulating said input device.

20. The system of claim 16, wherein said interface device further comprises a UART connected to said ninth means and to said host, whereby the results of the comparison are provided to said host.

21. The system of claim 16, wherein the visual display device is a video graphics array.

22. A method for testing a system under test (SUT) having at least one of computer hardware and software applications residing thereon, the SUT having a software application responsive to input data and a visual display device responsive to the software application for generating visual display signals, the testing method being operable in a record mode and a playback mode, said method comprising the steps of:

(a) in the record mode,

(1) receiving the input data from an input device,

(2) storing the input data from step (1) to thereby form stored input data;

(3) sending the input data from step (1) to the software application on the SUT;

(4) generating visual display signals in response to the input data from step (3); and

(5) capturing in a non-intrusive manner signatures of the visual display signals from step (4);

(6) storing said signatures of the visual display signals from step (5); and

(b) in the playback mode,

(7) sending the stored input data from step (2) to the software application on the SUT;

(8) generating new signatures of the visual display signals in response to the stored input data from step (6);

(9) capturing in a non-intrusive manner said new signatures of the visual display signals from step (7); and

(10) comparing said signatures of the visual display signals from step (5) and said new signature of the visual display signals from step (8) so as to indicate whether or not the SUT is functioning properly.

23. The testing method of claim 22, further comprising a step after said step (4) of storing the visual display signals in a memory on a host.

24. The testing method of claim 22, wherein said step (5) comprises a step of generating a compact representation of the visual display signals.

25. The testing method of claim 22, wherein said step (3) comprises a step of emulating said input device.

26. The testing method of claim 22, wherein said step (6) comprises a step of emulating said input device.

27. A method for testing a system under test (SUT) having at least one of computer hardware and software applications residing thereon, the SUT having a software application responsive to input data and a visual display device responsive to the software application for generating visual display signals, the testing method being operable in a record mode, said method comprising the steps of:

(1) receiving the input from an input device;

(2) storing the input data form step (1) to thereby form stored input data;

(3) sending the input data from step (1) to the software application on the SUT;

(4) generating visual display signals in response to the input data from step (3);

(5) capturing signatures of the visual display signals from step (4) in a non-intrusive manner; and

,

(6) storing said signatures of the visual display signals from step (5).

28. The testing method of claim 27, wherein said step (5) comprises a step of generating a compact representation of the visual display signals.

29. The testing method of claim 28, wherein said step (3) comprises a step of using a linear feedback shift register for capturing said new signatures of visual display signals.

30. The testing method of claim 27, wherein said step (2) comprises a step of storing the input data on a host, and

said step (6) comprises a step of storing said signatures of visual display signals on said host.

31. The testing method of claim 27, wherein said step (5) comprises a step of using a micro-controller sub-system under firmware and hardware control.

32. A method for testing a system under test (SUT) having at least one of computer hardware and software applications residing thereon, the SUT having a software application responsive to input data and a visual display device responsive to the software application for generating visual display signals, the testing method being operable in a regeneration mode, said method comprising the steps of:

(1) sending the stored input data to the software application on the SUT,

(2) generating new signatures of visual display signals in response to the stored input data of step (1);

(3) capturing said new signatures of visual display signals of step (2) in a non-intrusive manner; and

(4) storing said new signatures of visual display signals.

33. A method for testing a system under test (SUT) having at least one of computer hardware and software applications residing thereon, the SUT having a software application responsive to input data and a visual display device responsive to the software application for generating visual display signals, the testing method being operable in a regeneration mode, said method comprising the steps of:

(1) sending the stored input data to the software application on the SUT,

(2) generating new signatures of visual display signals in response to the stored input data of step (1);

(3) capturing said new signatures of visual display signals from step (2);

(4) retrieving stored signatures of visual display signals; and

(5) comparing the stored signatures of the visual display signals and said new signatures of the visual display signals, so as to indicate whether or not the SUT is functioning properly.

34. The testing method of claim 33, wherein said step (4) further comprises a step of replacing previously stored signatures of visual display signals with said new signatures of visual display signals.

35. A system for testing a system under test (SUT) having at least one of computer hardware and software applications residing thereon, the SUT having a software application responsive to input data and a visual display device responsive to the software application to generate visual display signals, the testing system comprising:

(a) a host for receiving first input data from an input device, said host comprising,

(1) first means for storing said first input data as stored input data,

(2) second means for sending said first input data to an interface device during a record mode,

(3) third means for sending a take-signature command to said interface device during said record mode,

(4) fourth means for storing captured signatures of visual display signals as stored signatures of visual display signals, and

(5) software module means for causing said fourth means to store only said captured signatures of the visual display signals having a selected prevalence; and

(b) said interface device, being connected to said host and to the SUT for sending said first input data to the software application of the SUT, and comprising;

(1) fifth means for capturing in a non-intrusive manner signatures of the visual display signals of the visual display device in response to said take-signature command, and

(2) sixth means, responsive to said fifth means, for sending captured signatures of visual display signals to said fourth means for storage as said stored signatures of visual display signals.

36. The system of claim 35, wherein said software module means comprises histogram means for determining said selected prevalence.

37. A system for testing a system under test (SUT) having at least one of computer hardware and software applications residing thereon, the SUT having a software application responsive to input data and a visual display device responsive to the software application to generate visual display signals, the testing system comprising:

(a) a host for receiving first input data from an input device, said host comprising,

(1) first means for storing said first input data as stored input data,

(2) second means for sending said first input data to an interface device during a record mode,

(3) third means for sending a take-signature command to said interface device during said record mode, and

(4) fourth means for storing captured signatures of visual display signals as a stored visual signature of the display signals; and

(b) said interface device, being connected to said host and to the SUT for sending said first input data to the software application of the SUT, and comprising;

(1) fifth means for capturing in a non-intrusive manner a selected portion representative of a plane of the visual display signals of the visual display device in response to said take-signature command, and

(2) sixth means, responsive to said fifth means, for sending captured signature of the visual display signals to said fourth means for storage as a stored signature of visual display signals.

38. A system for testing a system under test (SUT) having at least one of computer hardware and software applications residing thereon, the SUT having a software application responsive to input data and a visual display device responsive to the software application to generate visual display signals, the testing system comprising:

(a) a host for receiving first input data from an input device, said host comprising,

(1) first means for storing said first input data as stored input data,

(2) second means for sending said first input data to an interface device during a record mode,

(3) third means for sending a take-signature command to said interface device during said record mode, and

(4) fourth means for storing captured signatures of visual display signals as a stored visual signature of the display signals; and

(b) said interface device, being connected to said host and to the SUT for sending said first input data to the software application of the SUT, and comprising;

(1) fifth means for capturing in a non-intrusive manner signatures of the visual display signals of the visual display device in response to said take-signature command, said fifth means comprising;

(i) a linear feedback shift register which captures said signatures of the visual display signals and

(ii) a hardware state machine for controlling said linear feedback shift register, and

(2) sixth means, responsive to said fifth means, for sending a captured signature of the visual display signals to said fourth means for storage as said stored signature of visual display signals.

39. The system of claim 38, wherein said hardware state machine is controlled by vertical synchronizing and horizontal synchronizing signals.

40. A system for testing a system under test (SUT) having at least one of computer hardware and software applications residing thereon, the SUT having a software application responsive to stored input data and a visual display device responsive to the software application and which generates visual display signals, the testing system comprising:

(a) a host for receiving first input data from an input device, said host comprising,

(1) first means for storing said first input data as stored input data,

(2) second means for sending said first input data to an interface device during a playback mode;

(3) third means for sending the first input data to said interface device during a record mode,

(4) fourth means for sending a take-signature command to said interface device during said record mode, and

(5) fifth means for storing captured signatures of the visual display signals as stored signatures of visual display signals during said record mode, and

(6) software module means for causing said fifth means to store only said captured signatures of the visual display signals having a selected prevalence; and

(b) said interface device, being connected to said host and to the system under test, which sends during said record mode the first input data to the software application of the system under test, which sends during said playback mode the stored input data, and which captures during said playback mode and said record mode the visual display signals from the visual display device, said interface device comprising,

(1) sixth means for capturing in a non-intrusive manner signatures of the visual display signals of the visual display device in response to said take-signature command during said record mode,

(2) seventh means for capturing in a non-intrusive manner signatures of the visual display signals of the visual display device during said playback mode,

(3) eighth means, responsive to said sixth means, for sending said captured signatures of the visual display signals to said fifth means for storage as the stored signatures of visual display signals, and

(4) ninth means, responsive to said seventh means and to said fifth means, for comparing the stored signatures of the visual display signals with captured signatures of the visual display signals, so as to provide an indication as to whether the system under test is functioning properly.

41. The system of claim 40, wherein said software module comprises histogram means for determining said selected prevalence.

42. A system for testing a system under test (SUT) having at least one of computer hardware and software applications residing thereon, the SUT having a software application responsive to stored input data and a visual display device responsive to the software application and which generates visual display signals, the testing system comprising:

(a) a host for receiving the first input data from an input device, said host comprising,

(1) first means for storing the first input data as the stored input data,

(2) second means for sending the stored input data to an interface device during a playback mode;

(3) third means for sending the first input data to said interface device during a record mode,

(4) fourth means for sending a take-signature command to said interface device during said record mode, and

(5) fifth means for storing captured signatures of the visual display signals as stored signatures of the visual display signals during said record mode; and

,

(b) said interface device, connected to said host and to the system under test, which sends during said record mode the first input data to the software application of the system under test, which sends during said playback mode the stored input data, and which captures during said playback mode and said record mode the visual display signals from the visual display device, said interface device comprising,

(1) sixth means for capturing in a non-intrusive manner signatures of the visual display signals of the visual display device in response to said take-signature command during said record mode,

(2) seventh means for capturing in a non-intrusive manner signatures of the visual display signals of the visual display device during said playback mode,

(3) eighth means, responsive to said sixth means, for sending said captured signatures of the visual display signals to said fifth means for storage as stored signatures of the visual display signals,

(4) ninth means, responsive to said seventh means and to said fifth means, for comparing said stored signatures of the visual display signals with captured signatures of the visual display signals, so as to provide an indication as to whether the system under test is functioning properly, and

(5) tenth means for capturing a selected portion of signatures of the visual display signals representative of a plane.

43. A method for testing a system under the test (SUT) having at least one of computer hardware and software applications residing thereon, the SUT having a software application responsive to input data stored input data and visual display device responsive to the software application and which generates visual display signals, the testing method being operable in a record mode and a playback mode, said method comprising the steps of:

in the record mode,

(1) receiving the input data from an input device,

(2) storing the input data from step (1) to thereby form stored input data,

(3) sending the input data from step (1) to the software application on the SUT,

(4) generating visual display signals in response to the input data from step (3),

(5) capturing in a non-intrusive manner signatures of the visual display signals from step (4); and

(6) storing in a memory on a host signatures of the visual display signals having a selected prevalence, and

in the playback mode,

(7) sending the stored input data from step (2) to the software application on the SUT,

(8) generating new signatures of the visual display signals in response to the stored input data from step (7),

(9) capturing in a non-intrusive manner said new signatures of the visual display signals from step (7), and

(10) comparing said signatures of the visual display signals from step (6) and said new signature of the visual display signals from step (9) so as to indicate whether or not the SUT is functioning properly.

44. The testing method of claim 43, further comprising a step of using a histogram for determining said selected prevalence.

45. A method for testing a system under the test (SUT) having at least one of computer hardware and software applications residing thereon, the SUT having a software application responsive to input data stored input data and visual display device responsive to the software application and which generates visual display signals, the testing method being operable in a record mode and a playback mode, said method comprising the steps of:

in the record mode,

(1) receiving the input data from an input device;

(2) storing the input data from step (1) to thereby form stored input data,

(3) sending the input data from step (1) to the software application on the system under test;

(4) generating visual display signals in response to the input data from step (3); and

(5) capturing in a non-intrusive manner a selected portion of the visual display signals from step (4) representative of a plane of the new visual display signal;

in the playback mode,

(6) sending the stored input data from step (2) to the software application on the system under test;

(7) generating new visual display signals in response to the stored input data from step (6);

(8) capturing in a non-intrusive manner said new visual display signals from step (7); and

(9) comparing the visual display signals from step (5) and said new visual display signals from step (8) so as to indicate whether or not the system under test is functioning properly.

46. A method for testing a system under the test (SUT) having at least one of computer hardware and software applications residing thereon, the SUT having a software application responsive to input data stored input data and visual display device responsive to the software application and which generates visual display signals, the testing method being operable in a record mode and a playback mode, said method comprising the steps of:

in the record mode,

(1) receiving the input data from an input device;

(2) storing the input data from step (1) to thereby form stored input data;

(3) sending the input data from step (1) to the software application on the system under test;

(4) generating visual display signals in response to the input data from step (3); and

(5) capturing in a non-intrusive manner signatures of the visual display signals from step (4);

in the playback mode,

(6) sending the stored input data from step

     2) to the software application on the system under test;

(7) generating new signatures of the visual display signals in response to the stored input data from step (6);

(8) capturing in a non-intrusive manner a selected portion of said new signatures of the visual display signals from step (7) representative of a plane of the new signatures of the visual display signals; and

,

(9) comparing signatures of the visual display signals from step (5) and said new signatures of the visual display signals from step (8) so as to indicate whether or not the SUT is functioning properly.