System and method enabling execution stop and restart of a test executive sequence(s)

• US 7,076,692 B2
• Filed: 08/31/2001
• Issued: 07/11/2006
• Est. Priority Date: 08/31/2001
• Status: Expired due to Term

• Current Assignee: National Instruments Corporation (Emerson Electric Company)
• Original Assignee: National Instruments Corporation (Emerson Electric Company)
• Inventors: Grey, James
• Primary Examiner(s): Beausoliel, Robert
• Assistant Examiner(s): DUNCAN, MARC M
• Application Number:
  US09/944,546
• Publication Number:
  US 20030046612A1
• Time in Patent Office: 1,775 Days
• Field of Search: 714/1, 714/48, 714/25, 714/26, 714/15, 714/16, 702/120
• US Class Current: 714/15
• CPC Class Codes: H04L 1/24 Testing correct operation

Abstract

A system and method for enabling execution stop and re-start of a test executive sequence or hierarchy of test executive sequences. Execution progress of a test executive sequence or test executive sequence hierarchy may be periodically stored. This may comprise performing or taking “snapshots” of the execution at various points during the execution. Performing a snapshot may comprise saving all data needed to restore and re-start the execution at the respective point. The criteria of when and where to perform the snapshots may be any of various criteria and may be specified in any of various ways.

Claims

1. A computer-implemented method for storing execution progress of a test executive sequence tree hierarchy, the method comprising:

executing the test executive sequence tree hierarchy on a computer system, wherein the test executive sequence tree hierarchy includes a plurality of test executive sequences related to each other according to a hierarchical tree relationship such that execution of each child test executive sequence in the tree hierarchy is invoked by a parent test executive sequence in the tree hierarchy, wherein each of the test executive sequences includes a plurality of steps;

performing one or more snapshots of the execution of the test executive sequence hierarchy, wherein each snapshot is performed at a particular point during execution of the test executive sequence hierarchy;

wherein, for each snapshot, performing the snapshot comprises storing information usable to re-start execution of the test executive sequence hierarchy from the point at which the snapshot was performed.

2. A computer-implemented method for executing a test executive sequence, the method comprising:

executing the test executive sequence on a computer system, wherein the test executive sequence comprises a plurality of steps, wherein each step in at least a subset of the steps calls an external code module;

wherein executing the test executive sequence comprises executing the test executive sequence under control of a test executive engine, wherein the test executive engine is operable to execute the steps in the test executive sequence according to an order of execution specified by the test executive sequence;

wherein for each step in the at least a subset of the steps, executing the step comprises the test executive engine invoking execution of the external code module called by the step;

wherein the test executive engine is further operable to perform one or more snapshots of the execution of the test executive sequence, wherein each snapshot is performed at a particular step during execution of the test executive sequence;

wherein, for each snapshot, performing the snapshot comprises the test executive engine storing information usable to re-start execution of the test executive sequence from the step at which the snapshot is performed;

wherein the method further comprises;

stopping execution of the test executive sequence after a particular snapshot is performed in response to user input requesting to stop execution of the test executive sequence; and

re-starting execution of the test executive sequence from the step at which the particular snapshot was performed in response to user input requesting to re-start execution of the test executive sequence, wherein re-starting execution of the test executive sequence comprises using the stored information of the particular snapshot to restore an execution environment of the computer system so that the test executive sequence can execute correctly from the step at which the particular snapshot was performed;

wherein the stored information of the particular snapshot enables a user to temporarily stop execution of the test executive sequence and then re-start execution of the test executive sequence without the entire test executive sequence being re-executed from the beginning.

3. The method of claim 2,

wherein the stored information of the particular snapshot aids the user in debugging the test executive sequence by enabling the user to temporarily stop execution of the test executive sequence and then re-start execution of the test executive sequence without the entire test executive sequence being re-executed from the beginning.

4. A computer-implemented method for executing a test executive sequence, the method comprising:

executing the test executive sequence on a computer system, wherein the test executive sequence comprises a plurality of steps, wherein each step in at least a subset of the steps calls an external code module;

wherein executing the test executive sequence comprises executing the test executive sequence under control of a test executive engine, wherein the test executive engine is operable to execute the steps in the test executive sequence according to an order of execution specified by the test executive sequence;

wherein for each step in the at least a subset of the steps, executing the step comprises the test executive engine invoking execution of the external code module called by the step;

wherein the test executive engine is further operable to perform a plurality of snapshots of the execution of the test executive sequence, wherein each snapshot is performed at a particular step during execution of the test executive sequence;

wherein, for each snapshot, performing the snapshot comprises the test executive engine storing information usable to re-start execution of the test executive sequence from the step at which the snapshot is performed;

wherein the method further comprises;

stopping execution of the test executive sequence after a particular snapshot is performed;

receiving user input selecting the particular snapshot from the plurality of snapshots; and

re-starting execution of the test executive sequence from the step at which the particular snapshot was performed in response to the user input selecting the particular snapshot, wherein re-starting execution of the test executive sequence comprises using the stored information of the particular snapshot to restore an execution environment of the computer system so that the test executive sequence can execute correctly from the step at which the particular snapshot was performed.

5. The method of claim 4, further comprising:

displaying a graphical user interface that visually indicates the plurality of snapshots;

wherein said receiving the user input selecting the particular snapshot from the plurality of snapshots comprises receiving user input to the graphical user interface selecting the particular snapshot from the plurality of visually indicated snapshots.

6. A computer-implemented method for executing a test executive sequence, the method comprising:

executing the test executive sequence on a computer system;

wherein the test executive sequence comprises a plurality of steps, wherein each step in at least a subset of the steps calls an external code module;

wherein said executing the test executive sequence comprises executing the test executive sequence under control of a test executive engine, wherein the test executive engine is operable to execute the steps in the test executive sequence according to an order of execution specified by the test executive sequence;

wherein for each step in the at least a subset of the steps, executing the step comprises the test executive engine invoking execution of the external code module called by the step;

wherein the plurality of steps in the test executive sequence includes a first particular step, wherein the first particular step is operable to cause the test executive engine to perform a snapshot when the first particular step is executed by the test executive engine, wherein performing the snapshot comprises the test executive engine storing information usable to re-start execution of the test executive sequence from the first particular step;

wherein the test executive engine performs the snapshot directly in response to executing the first particular step.

7. A computer-implemented method for re-starting execution of a test executive sequence, the method comprising:

executing the test executive sequence on a computer system, wherein the test executive sequence comprises a plurality of steps, wherein each step in at least a subset of the steps calls an external code module, wherein the steps are operable to control a hardware device to test a unit under test (UUT);

wherein executing the test executive sequence comprises;

executing a first subset of the steps in the test executive sequence, wherein one or more steps in the first subset of steps are operable to set the hardware device to a first state; and

performing a first snapshot at a first particular step in the test executive sequence, wherein the first particular step is executed after the first subset of steps, wherein performing the first snapshot comprises storing information usable to re-start execution of the test executive sequence from the first particular step;

wherein the method further comprises;

stopping execution of the test executive sequence after the first snapshot is performed; and

re-starting execution of the test executive sequence from the first particular step at which the first snapshot was performed, wherein re-starting execution of the test executive sequence from the first particular step at which the first snapshot was performed comprises re-executing one or more, but not all, of the steps in the first subset of steps that were executed before the first snapshot was performed, wherein the one or more steps are re-executed to reset the hardware device to the first state;

wherein the first snapshot enables execution of the test executive sequence to be re-started from the first particular step without re-executing all of the steps in the first subset of steps that were executed before the first snapshot was performed.

8. The method of claim 7,

wherein the test executive sequence is organized into step groups;

wherein said re-executing the one or more, but not all, of the steps in the first subset of steps that were executed before the first snapshot was performed comprises re-executing steps of a first step group, wherein the steps of the first step group comprise steps operable to set the hardware device to the first state.

9. A computer-implemented method for re-starting execution of a test executive sequence, the method comprising:

executing the test executive sequence on a computer system, wherein the test executive sequence comprises a plurality of steps, wherein each step in at least a subset of the steps calls an external code module;

wherein executing the test executive sequence comprises;

executing a first subset of the steps in the test executive sequence; and

performing a first snapshot at a first particular step in the test executive sequence, wherein the first particular step is executed after the first subset of steps, wherein performing the first snapshot comprises storing information usable to re-start execution of the test executive sequence from the first particular step;

wherein the method further comprises;

stopping execution of the test executive sequence after the first snapshot is performed; and

re-starting execution of the test executive sequence from the first particular step at which the first snapshot was performed, wherein re-starting execution of the test executive sequence from the first particular step at which the first snapshot was performed comprises re-executing one or more, but not all, of the steps in the first subset of steps that were executed before the first snapshot was performed;

wherein the first snapshot enables execution of the test executive sequence to be re-started from the first particular step without re-executing all of the steps in the first subset of steps that were executed before the first snapshot was performed.

10. A computer-implemented method for executing a test executive sequence, the method comprising:

executing the test executive sequence on a computer system, wherein the test executive sequence comprises a plurality of steps, wherein each step in at least a subset of the steps calls an external code module;

wherein said executing the test executive sequence comprises executing the test executive sequence under control of a test executive engine, wherein the test executive engine is operable to execute the steps in the test executive sequence according to an order of execution specified by the test executive sequence;

wherein for each step in the at least a subset of the steps, executing the step comprises the test executive engine invoking execution of the external code module called by the step;

wherein the test executive engine is operable to perform a first snapshot of the execution of the test executive sequence at a first particular step in the test executive sequence, wherein the test executive engine performs the first snapshot in response to evaluating one or more values set by execution of the test executive sequence, wherein performing the first snapshot at the first particular step comprises the test executive engine storing information usable to re-start execution of the test executive sequence from the first particular step.

11. The method of claim 10,

wherein said performing the first snapshot in response to evaluating the one or more values set by execution of the test executive sequence comprises performing the first snapshot in response to determining that a Boolean condition is True, wherein the Boolean condition is based on the one or more values set by execution of the test executive sequence.

12. A computer-readable memory medium comprising program instructions for executing a test executive sequence, wherein the program instructions are executable to implement:

executing the test executive sequence on a computer system, wherein the test executive sequence comprises a plurality of steps;

performing a first snapshot of the execution of the test executive sequence, wherein the first snapshot is performed at a particular step during execution of the test executive sequence;

stopping execution of the test executive sequence in response to first user input after the first snapshot is performed; and

re-starting execution of the test executive sequence from the step at which the first snapshot was performed in response to second user input, wherein said re-starting uses information from the first snapshot.

13. The memory medium of claim 12,

wherein the first snapshot enables a user to temporarily stop execution of the test executive sequence and then re-start execution of the test executive sequence without the entire test executive sequence being re-executed from the beginning.

14. The memory medium of claim 12,

wherein said re-starting execution of the test executive sequence comprises using the information from the first snapshot to restore an execution environment of the computer system so that the test executive sequence can execute starting from the step at which the first snapshot was performed.

15. The memory medium of claim 14,

wherein said restoring the execution environment of the computer system comprises using the information from the first snapshot to re-create a stack frame of the computer system.

16. The memory medium of claim 14,

wherein said restoring the execution environment of the computer system comprises using the information from the first snapshot to restore results of steps in the test executive sequence executed prior to the step at which the first snapshot was performed.

17. The memory medium of claim 16,

wherein said restoring the results of steps in the test executive sequence executed prior to the step at which the first snapshot was performed comprises restoring execution results of external code modules called by steps executed prior to the step at which the first snapshot was performed.

18. The memory medium of claim 12,

wherein performing the first snapshot comprises storing information representing execution results of steps in the test executive sequence executed prior to the step at which the first snapshot is performed.

19. The memory medium of claim 18,

wherein said storing the information representing the execution results of steps in the test executive sequence executed prior to the step at which the first snapshot is performed comprises storing information representing execution results of external code modules called by steps executed prior to the step at which the first snapshot is performed.

20. A computer-readable memory medium comprising program instructions for executing a test executive sequence, wherein the program instructions are executable to implement:

executing the test executive sequence on a computer system, wherein the test executive sequence comprises a plurality of steps;

performing a plurality of snapshots of the execution of the test executive sequence, wherein each snapshot is performed at a particular step during execution of the test executive sequence;

stopping execution of the test executive sequence after a particular snapshot is performed;

receiving user input selecting a first snapshot from the plurality of snapshots; and

re-starting execution of the test executive sequence from the step at which the first snapshot was performed in response to the user input selecting the first snapshot, wherein said re-starting uses information from the first snapshot.

21. The memory medium of claim 20, wherein the program instructions are further executable to implement:

displaying a graphical user interface that visually indicates the plurality of snapshots;

wherein said receiving the user input selecting the first snapshot from the plurality of snapshots comprises receiving user input to the graphical user interface selecting the first snapshot from the plurality of visually indicated snapshots.

22. The memory medium of claim 20,

wherein said re-starting execution of the test executive sequence comprises using the information from the first snapshot to restore an execution environment of the computer system so that the test executive sequence can execute starting from the step at which the first snapshot was performed.

23. The memory medium of claim 22,

wherein said restoring the execution environment of the computer system comprises using the information from the first snapshot to re-create a stack frame of the computer system.

24. A computer-readable memory medium comprising program instructions for executing a test executive sequence, wherein the program instructions are executable to implement:

executing the test executive sequence on a computer system, wherein the test executive sequence comprises a plurality of steps, wherein executing the test executive sequence comprises executing the plurality of steps;

wherein the plurality of steps includes a first step, wherein the first step is configured to cause a snapshot of the execution of the test executive sequence to be performed, wherein performing the snapshot comprises storing information usable to re-start execution of the test executive sequence from the first step;

wherein the program instructions are further executable to implement;

re-starting execution of the test executive sequence from the first step, wherein said re-starting uses the stored information from the snapshot.

25. The memory medium of claim 24,

wherein said re-starting execution of the test executive sequence comprises using the stored information from the snapshot to restore an execution environment of the computer system so that the test executive sequence can execute starting from the first step.

26. The memory medium of claim 25,

wherein said restoring the execution environment of the computer system comprises using the stored information from the snapshot to re-create a stack frame of the computer system.