System and method for using model analysis to generate directed test vectors

• US 7,970,594 B2
• Filed: 06/30/2005
• Issued: 06/28/2011
• Est. Priority Date: 06/30/2005
• Status: Active Grant

• Current Assignee: The MathWorks, Inc.
• Original Assignee: The MathWorks, Inc.
• Inventors: Gaudette, Thomas
• Primary Examiner(s): Shah; Kamini S
• Assistant Examiner(s): GEBRESILASSIE, KIBROM K
• Application Number:
  US11/173,977
• Publication Number:
  US 20070016394A1
• Time in Patent Office: 2,189 Days
• Field of Search: 703/14, 703/22, 703/13, 703/20, 716/5, 716/4, 716/106, 714/100, 714/699, 714/738, 714/44, 714/715, 714/735, 714/736, 714/742
• US Class Current: 703/13
• CPC Class Codes: G06F 11/261 by simulating additional hardware, e.g. fault simulation
• CPC Class Codes: G06F 30/33 Design verification, e.g. functional simulation or model checking

Abstract

A mechanism for exploiting the data gathered about a system model during the system design phase to aid the identification of errors subsequently detected in a deployed system based on the system model is disclosed. The present invention utilizes the coverage analysis from the design phase that is originally created to determine whether the system model as designed meets the specified system requirements. Included in the coverage analysis report is the analysis of which sets of test vectors utilized in simulating the system model excited individual components and sections of the system model. The present invention uses the information associated with the test vectors to select appropriate test vectors to use to perform directed testing of the deployed system so as to confirm a suspected fault.

Claims

1. A method of identifying an error in a deployed system, the method comprising;

testing, in a technical computing environment operating on an electronic device having a processor, a system model using sets of test vectors, the system model having one or more components;

identifying, based on the testing, a degree of model capture associated with each set of test vectors used to test the system model, the degree of model capture indicating how many components of the system model are excited by each set of test vectors when used to test the system model;

saving in a memory coupled to the processor the sets of test vectors following the testing of the system model;

receiving a report of a fault in the deployed system;

based upon the reported fault in the deployed system, selecting from the sets of test vectors a set of test vectors for application to the deployed system, the set of test vectors that is selected being the set of test vectors that provided a greatest degree of model capture during the testing of the system model based on which set of test vectors excited the highest percentage of components of the system model with respect to other sets of test vectors; and

applying the selected set of test vectors to the deployed system to identify the error in the deployed system,

wherein

the deployed system is based on the system model and is a real-life system deployed outside of the technical computing environment.

2. The method of claim 1, further comprising:

presenting a testing results report to a user showing the degree of model capture associated with each set of test vectors.

3. The method of claim 2 wherein the testing results report further identifies whether an individual component in the system model is executed by a particular test vector.

4. The method of claim 2 wherein the testing results report indicates the percentage of time a particular component in the system model was active during the execution of the model with a given test vector.

5. The method of claim 3 wherein the step of selecting the set of test vectors to be applied to the deployed system is further based on an identity of the reported fault, the degree of model capture, and the testing results report identifying the individual components executed by a given test vector.

6. The method of claim 1, further comprising:

providing a block diagram view of the system model being tested; and

tracing the execution path of the model during execution of a particular test vector.

7. The method of claim 6, further comprising:

providing a report linked to the block diagram view of the model.

8. The method of claim 1 wherein more than one of the sets of test vectors is executed in order to identify the error.

9. The method of claim 1, further comprising:

presenting, for each set of test vectors, a signal range analysis report to a user indicating the maximum and minimum signal values for each component in the model measured during simulation of the model.

10. The method of claim 1, further comprising:

presenting, for each set of test vectors, at least one of the group of a lookup table analysis report, a state transition analysis report, a decision analysis report, a condition analysis report, and a modified condition/decision coverage analysis report to a user.

11. The method of claim 1 wherein the testing is performed to determine compliance with a set of requirements for the system model.

12. The method of claim 1 wherein the error is identified using reliability data in addition to the selected set of test vectors.

13. The method of claim 1 wherein

the technical computing environment runs on a first device, and

the deployed system is implemented on a second device separate from the first device.

14. The method of claim 1 further comprising:

generating a coverage analysis report, based on the testing,

wherein,

the coverage analysis report includes the degree of model capture associated with each set of test vectors used to test the system model.

15. The method of claims 1 wherein

the selecting of the set of test vectors to be applied to the deployed system is automated.

16. A method of identifying an error in a deployed system, the method comprising;

testing, in a technical computing environment operating on an electronic device having a processor, a system model using sets of test vectors, the system model has a plurality of model portions, and each set of test vectors used to test the system model provides a degree of activation of each model portion;

for each set of test vectors, identifying the degree of activation of each model portion, the degree of activation indicating how often each corresponding model portion is activated;

saving in a memory coupled to the processor the sets of test vectors following the testing of the system model;

receiving a report of a fault in the deployed system;

identifying a component of the deployed system suspected of causing the reported fault in the deployed system;

based upon the reported fault in the deployed system, selecting from the sets of test vectors a set of test vectors for application to the deployed system, the set of test vectors that is selected being the set of test vectors that provides a greatest degree of activation of the model portion that corresponds to the component of the deployed system suspected of causing the reported fault, the greatest degree of activation based on which set of test vectors most activated the component suspected of causing the reported fault with respect to other sets of test vectors; and

applying the selected set of test vectors to the deployed system to identify the error in the deployed system,

wherein

the deployed system is based on the system model and is a real-life system deployed outside of the technical computing environment.

17. A system for identifying an error in a deployed system, the system comprising:

a processor configured to implement a technical computing environment, the technical computing environment including,

a system model for a system being designed in the technical computing environment, the system model having one or more components,

a test system configured to;

test the system model using a plurality of sets of test vectors,

identify a degree of model capture associated with each set of test vectors used to test the system, the degree of model capture indicating how many components of the system model are excited by each set of test vectors when used to test the system model, and

save in a memory coupled to the processor the sets of test vectors following the testing of the system model, and a fault detector configured to;

select, based upon a reported fault in the deployed system, a set of test vectors, from the plurality of sets of test vectors, for application to the deployed system, the set of test vectors that is selected being the set of test vectors that provided a greatest degree of model capture during the testing of the system based on which set of test vectors excited the highest percentage of components of the system model with respect to other sets of test vectors, and

apply the selected set of test vectors to the deployed system to identify the error in the deployed system,

wherein

the deployed system is based on the system model and is a real-life system deployed outside of the technical computing environment.

18. The system of claim 17 wherein the technical computing environment generates a block diagram view of the system model.