Automated validation and verification of computer software

• US 6,071,316 A
• Filed: 09/29/1997
• Issued: 06/06/2000
• Est. Priority Date: 09/29/1997
• Status: Expired due to Term

• Current Assignee: Honeywell Incorporated (Honeywell International Inc.)
• Original Assignee: Honeywell Incorporated (Honeywell International Inc.)
• Inventors: Goossen, Emray R., Lippitt, Carl E., Shema, David K.
• Primary Examiner(s): Hafiz, Tariq R.
• Assistant Examiner(s): Khatri, Anil
• Application Number:
  US08/939,419
• Time in Patent Office: 981 Days
• Field of Search: 395/701, 395/702, 395/704, 395/706, 395/710, 395/705, 711/128, 364/491
• US Class Current: 717/126
• CPC Class Codes: G06F 11/3612 by runtime analysis performance monitoring G06F11/3466
• CPC Class Codes: G06F 11/3676 for coverage analysis
• CPC Class Codes: G06F 11/3688 for test execution, e.g. scheduling of test suites
• CPC Class Codes: G06F 11/3696 Methods or tools to render software testable

Abstract

A method and apparatus for automating validation and verification of computer software that confirms during a test execution of the software that all lines of code are executed and all branches in the software are taken or not taken at least once. The computer software to be tested is compiled and a link map is generated. After compilation of the code, it is run in a test fixture to test all the required functions. During this test execution, a monitoring process is performed which documents which lines of code have been executed and whether certain branches of the code were either taken or not taken. An execution record is generated which indicates what instruction branches were taken and were not taken. A comparison is then made between the link map originally generated and the instruction record generated to determine what lines of code were executed, whether each branch was taken at least once, and whether a branch was not taken at least once.

Claims

1. A method for automated validation and verification of computer software executing path and decision path, the computer software being validated and verified being the same computer software as used after validation and verification without being instrumented with other software, comprising the steps of:

compiling the computer software and generating a link map which includes a listing of all of lines of code in the computer software as well as links for all branches which are part of the computer software;

executing the computer software in a processor such that all functions of the computer software are executed;

monitoring the execution of the computer software in the processor of all lines of code in the computer software, storing a current address of a current line of the computer software which is being executed in the processor and a previous address of a previous line of the computer software being executed in the processor, and generating an execution record map with first and second fields, where the first field contains an indicator of all instruction addresses in the computer software where a branch is not taken, and the second field contains an indicator of all the instruction addresses in the software program where a branch is taken;

comparing the link map with the execution record map to determine that all lines of the computer software are executed, and to identify particular lines and branches of the computer software which were not executed; and

generating test coverage report which lists all the lines and branches of the computer software which were not executed and provides an evaluation of the test coverage of potential execution paths and decision execution paths;

wherein the executing and monitoring steps comprise the step of comparing the current address of a current line of the computer software which is being executed with the previous address of a previous line of the computer software that was executed, and if the current instruction is more than one address greater or less than the previous address, the branch taken field is filled with the indicator, and if the current address is no more than one greater from the previous address, the branch not taken field is filled with the indicator; and

wherein the step of comparing the link map with execution record map includes determining whether each of the addresses in the link map is or is not a branch instruction, and if it is, determining that the branch instruction has both the branch taken and the branch not taken indicators, and if the address is not a branch instruction determining the execution record map contains the branch not taken indicator.

2. An apparatus for automatic validation and verification of computer software execution path and decision path, the computer software being the same computer software as used after validation and verification without being instrumented with other software, the apparatus comprising:

means to compile the computer software;

means to generate a link map for the computer software which includes a listing of all of lines of code in the computer software as well as links for all branches which are part of the computer software;

means to execute the compiled computer software so that all functions of the computer software program are executed;

means to monitor the execution of the object code executing within the means to execute, and to store a current address of a current line of the computer software and a previous address of a previous line of the computer software, and to generate an execution record map with first and second fields, where the first field contains an indicator of all instruction addresses for execution of the computer software where a branch is not taken, and the second field contains an indicator of all the instruction addresses for execution of the computer software where a branch is taken;

means to compare the link map with the first and second fields of the execution record map to determine whether all lines of code in the software code have been executed, and whether all the branches in the computer software have been taken at least once and not been taken at least once; and

means to generate a test report which indicates which of the lines and the branches of the computer software have not been taken.

3. The apparatus for automated validation and verification of computer executing path and decision path software of claim 2 wherein the means to execute the computer software is a test fixture.

4. The apparatus for automated validation and verification of computer software executing path and decision path of claim 2 further including means to account for asynchronous interrupts during execution of the compiled computer software.

7. The apparatus of claim 2 wherein the validation and verification of the computer software is performed during requirements based testing.

8. The apparatus of claim 2 wherein the validation and verification of the computer software execution path and decision path is performed at the machine code execution level.

9. The apparatus of claim 2 wherein the validation and verification of the computer execution path and decision path is performed at the machine code execution level.

10. The apparatus of claim 2, wherein the means to compare the link map with the first and second fields compares the link map and the first and second fields post-processing to provide an analysis of the method in which the computer software was executed.

5. An apparatus for automatic validation and verification of computer software execution path and decision path comprising:

means to compile the computer software;

means to generate a link map for the computer software which includes a listing of all of lines of code in the computer software as well as links for all branches which are part of the computer software;

means to execute the compiled computer software so that all functions of the computer software are executed;

means to monitor the execution of the object code and to generate an execution record map with first and second fields, where the first field contains an indicator of all instruction addresses for execution of the computer software where a branch is not taken, and the second field contains an indicator of all the instruction addresses for execution of the computer software where a branch is taken;

means to compare the link map with the first and second fields to determine whether all lines of code in the computer software have been executed, and whether all the branches in the computer software have been taken at least once and not been taken at least once; and

means to generate a test report which indicates which of the lines and the branches of the computer software have not been taken;

wherein the validation and verification of the computer software execution path and decision path is performed while the computer software is embedded in the means to execute the compiled software.

6. The apparatus of claim 5 wherein the means to execute the compiled software is an avionics processor.