System and method for using model analysis to generate directed test vectors

• US 20070016394A1
• Filed: 06/30/2005
• Published: 01/18/2007
• Est. Priority Date: 06/30/2005
• Status: Active Grant

• Current Assignee: The MathWorks, Inc.
• Original Assignee: The MathWorks, Inc.
• Inventors: Gaudette, Thomas
• Granted Patent: US 7,970,594 B2
• Time in Patent Office: Days
• US Class Current: 703/19
• CPC Class Codes: G06F 11/261 by simulating additional hardware, e.g. fault simulation
• CPC Class Codes: G06F 30/33 Design verification, e.g. functional simulation or model checking

Abstract

A mechanism for exploiting the data gathered about a system model during the system design phase to aid the identification of errors subsequently detected in a deployed system based on the system model is disclosed. The present invention utilizes the coverage analysis from the design phase that is originally created to determine whether the system model as designed meets the specified system requirements. Included in the coverage analysis report is the analysis of which sets of test vectors utilized in simulating the system model excited individual components and sections of the system model. The present invention uses the information associated with the test vectors to select appropriate test vectors to use to perform directed testing of the deployed system so as to confirm a suspected fault.

Claims

1. A method of identifying errors in a deployed system, comprising;

testing in a technical computing environment a system model being designed with a plurality of sets of test vectors, the testing identifying a degree of model capture associated with each set of test vectors in the plurality of test vectors, the plurality of sets of test vectors saved following testing; and

identifying an error in the functioning of a deployed system based on the system model using the test vectors previously used during the testing of the system design.

2. The method of claim 1, further comprising:

presenting a testing results report to a user showing the degree of model capture associated with each set of test vectors.

3. The method of claim 2 wherein the testing results report further identifies whether an individual component in the system model is executed by a particular test vector.

4. The method of claim 2 wherein the testing results report indicates the percentage of time a particular component in the system model was active during the execution of the model with a given test vector.

5. The method of claim 1 wherein the step of identifying further comprises;

performing directed testing based on the identity of error reported, the degree of model capture, and the testing results report identifying the individual components executed by a given test vector.

6. The method of claim 1, further comprising:

providing a block diagram view of the system model being tested; and

tracing the execution path of the model during execution of a particular test vector.

7. The method of claim 6, further comprising:

providing a report linked to a block diagram view of the model.

8. The method of claim 1 wherein more than one of the plurality of sets of test vectors is executed in order to identify the reported error.

9. The method of claim 1, further comprising:

presenting, for each set of test vectors, a signal range analysis report to the user indicating the maximum and minimum signal values for each component in the model measured during simulation of the model.

10. The method of claim 1, further comprising:

presenting, for each set of test vectors, at least one of the group of a lookup table analysis report, a state transition analysis report, a decision analysis report, a condition analysis report, and a modified condition/decision coverage analysis report to a user.

11. The method of claim 1 wherein the testing is performed to determine compliance with a set of requirements for the system model.

12. The method of claim 1 wherein the error is identified using reliability data in addition to the plurality of test vectors.

13. In a computing device with a technical computing environment, a system for detecting errors in deployed systems based on system models, the system comprising:

a system model for a system being designed in the technical computing environment;

a test system for testing the system model, the test system simulating the execution of the system model with a plurality of sets of test vectors, at least some of the plurality of sets of test vectors being saved following testing; and

a coverage analysis report of the results of a simulation of the system model with the test system, the coverage analysis report subsequently used to select at least one of the plurality of sets of test vectors to perform directed testing to identify an error reported in a deployed system based on the system model.

14. The system of claim 13 wherein the technical computing environment generates a block diagram view of the system model.

15. The system of claim 13 wherein the coverage analysis report indicates at least one component in the system model executed by the use of each of the plurality of test vectors.

16. The system of claim 13 wherein the coverage analysis report indicates one of the percentage of the entire system model executed by the use of each of the plurality of test vectors and the percentage of time a particular component in the model was executed by each of the plurality of test vectors.

17. A method of identifying errors in a deployed system, comprising;

providing a model coverage analysis report for a model identifying a degree of model capture associated with each of a plurality of sets of vectors generated during the design of the model in a technical computing environment;

receiving a notification of an error in the functioning of a deployed system, the deployed system based on the system model; and

identifying an error in the functioning of a deployed system based on the system model using the model coverage analysis report and at least one of the plurality of sets of vectors.

18. The medium of claim 17, the instructions further comprising:

presenting a testing results report to a user showing the degree of model capture associated with each set of test vectors.

19. The medium of claim 18 wherein the testing results report further identifies whether an individual component in the system model is executed by a particular test vector.

20. The medium of claim 17 wherein the testing results report indicates the percentage of time a particular component in the system model was active during the execution of the model with a given test vector.

21. The medium of claim 17 wherein the instructions for the step of identifying further comprises;

performing directed testing based on the identity of error reported, the degree of model capture, and the testing results report identifying the individual components executed by a given test vector.

22. The medium of claim 17, the instructions further comprising:

providing a block diagram view of the system model being tested; and

tracing the execution path of the model during execution of a particular test vector.

23. The medium of claim 22, the instructions further comprising:

providing a report linked to a block diagram view of the model.

24. The medium of claim 17 wherein more than one of the plurality of sets of test vectors is executed in order to identify the reported error.

25. In a technical computing environment, a computer-readable medium holding computer-executable instructions for a method of identifying errors in a deployed system, the instructions comprising the steps of:

providing a set of requirements during the design of a system model;

testing the system model being designed with a plurality of sets of test vectors to determine compliance with the set of requirements, the testing identifying the degree of model capture associated with each set of test vectors in the plurality of test vectors, the plurality of sets of test vectors saved following testing;

receiving a notification of an error in the functioning of a deployed system, the deployed system based on the system model; and

identifying the error using the test vectors previously used to determine compliance with the set of requirements.

26. The method of claim 25, further comprising;

performing directed testing based on an identity of the error, and the model coverage analysis report, the model coverage analysis report identifying at least one individual component executed by at least one of the plurality of sets of vectors.