System and method enabling hierarchical execution of a test executive subsequence

• US 6,868,508 B2
• Filed: 08/31/2001
• Issued: 03/15/2005
• Est. Priority Date: 08/31/2001
• Status: Expired due to Term

• Current Assignee: National Instruments Corporation (Emerson Electric Company)
• Original Assignee: National Instruments Corporation (Emerson Electric Company)
• Inventors: Grey, James
• Primary Examiner(s): Beausoliel, Robert
• Assistant Examiner(s): DUNCAN, MARC M
• Application Number:
  US09/943,988
• Publication Number:
  US 20030093736A1
• Time in Patent Office: 1,292 Days
• Field of Search: 714/1, 714/26, 714/25, 714/48, 702/120
• US Class Current: 714/25
• CPC Class Codes: G06F 11/263 Generation of test inputs, e.g. test vectors, patterns or sequences ; with adaptation of the tested hardware for testability with external testers

Abstract

Individual test executive sequences may correspond to individual sub-components of a hierarchical system. To test a system sub-component directly without testing “upper” sub-components in the hierarchy, a first test executive sequence corresponding to that system sub-component may be executed directly, without requiring execution of test executive sequences above the first test executive sequence in the hierarchy, or requiring only partial execution of test executive sequences above the first test executive sequence in the hierarchy. Another embodiment of the invention comprises a system and method for enabling propagation or inheritance of test executive sequence local variables.

Claims

1. A computer-implemented method for testing a sub-component of a system comprising a hierarchy of sub-components organized in multiple levels, the method comprising:

creating a plurality of test executive sequences in response to user input, wherein each test executive sequence maps to a sub-component of the system and is operable to test the respective sub-component, wherein the plurality of test executive sequences are configured to execute according to a hierarchy corresponding to the hierarchy of sub-components, wherein the hierarchy of subcomponents comprises a top level and a first level, wherein the first level is lower in the hierarchy than the top level;

calling a first test executive sequence to test a first sub-component of the first level in the hierarchy, wherein the first level in the hierarchy corresponds to only a subset of the plurality of test executive sequence; and

executing the first test executive sequence without executing test executive sequences that map to sub-components above the first sub component in the hierarchy;

wherein said executing the first test executive sequence tests the first sub-component.

2. The method of claim 1,

wherein said executing the first test executive sequence comprises, for test executive sequences that map to sub-components above the first sub-component in the hierarchy, executing only setup portions of the test executive sequences that map to sub-components above the first sub-component in the hierarchy.

3. The method of claim 2,

wherein the setup portions are operable to set variable values used in the first test executive sequence.

4. The method of claim 1, further comprising:

executing only a portion of one or more test executive sequences that map to sub-components above the first sub-component in the hierarchy, for purposes of setting up data used in the first test executive sequence.

5. The method of claim 4,

wherein said executing only a portion of one or more test executive sequences that map to sub-components above the first sub-component in the hierarchy does not include performing functional testing of the sub-components above the first sub-component in the hierarchy.

6. The method of claim 1, further comprising:

executing only a portion of one or more test executive sequences that map to sub-components above the first sub-component in the hierarchy, for purposes of initializing a hardware device.

7. The method of claim 1,

wherein each test executive sequence includes a setup group of step and a main group of steps;

wherein said executing the first test executive sequence without executing test executive sequences that map to sub-components above the first sub-component in the hierarchy comprises executing steps in the setup groups of the test executive sequences that map to sub-components above the first sub-component in the hierarchy but does not comprise executing steps in the main groups.

8. The method of claim 1, further comprising:

displaying the plurality of test executive sequences on a graphical use interface in a hierarchical manner corresponding to the sub-components of the system.

9. The method of claim 1,

wherein said calling the first test executive sequence is performed in response to user input requesting to call the first test executive sequence.

10. The method of claim 9, further comprising:

displaying the plurality of test executive sequences on a graphical user interface; and

receiving user input requesting to call the first test executive sequence from the displayed plurality of test executive sequences.

11. The method of claim 1,

wherein said calling the first test executive sequence is performed in response to an application programming interface (API) call to execute the first test executive sequence.

12. A computer-implemented method for testing a sub-component of a system comprising a hierarchy of sub-components organize in multiple levels, the method comprising:

creating a plurality of test executive sequences in response to user input, wherein each test executive sequence maps to a sub-component of the system and is operable to test the respective sub-component, wherein the plurality of test executive sequences are configured to execute according to a hierarchy corresponding to the hierarchy of sub-components, wherein the hierarchy of subcomponents comprises a top level and a first level, wherein the first level is lower in the hierarchy than the top level;

calling a first test executive sequence to test a first sub-component of the first level in the hierarchy, wherein the first level in the hierarchy corresponds to only a subset of the plurality of test executive sequences;

executing only a portion of one or more test executive sequences that map to sub-components above the first sub-component in the hierarchy, for purposes of setting up data used in the first test executive sequence, after said calling; and

executing the first test executive sequence after said executing only a portion of one or more test executive sequences that map to sub-components above the first sub-component in the hierarchy;

wherein said executing the first test executive sequence tests the first sub-component.

13. The method of claim 12,

wherein said executing only a portion of one or more test executive sequences that map to sub-components above the first sub-component in the hierarchy comprises executing only setup portions of the test executive sequences that map to sub components above the first sub-component in the hierarchy.

14. The method of claim 13,

wherein the setup portions are operable to set variable values used in the first test executive sequence.

15. The method of claim 12,

wherein said executing only a portion of one or more test executive sequences that map to sub-components above the first sub-component in the hierarchy does not include performing functional testing of the sub-components above first sub-component in the hierarchy.

16. The method of claim 12,

wherein said executing only a portion of one or more test executive sequences that map to sub-components above the first sub-component in the hierarchy comprises executing only the portion of the one or more test executive sequences for purposes of initializing a hardware device.

17. The method of claim 12,

wherein each test executive sequence includes a setup group of steps and a main group of steps;

wherein said executing only a portion of one or more test executive sequences that map to sub-components above the first sub-component in the hierarchy comprises executing steps in the setup groups of the test executive sequences that map to sub-components above the first sub-component in the hierarchy but does not comprise executing steps in the main groups of the test executive sequences that map to sub-components above the first sub-component in the hierarchy.

18. A computer-implemented method for testing a sub-component of a system, wherein the system comprises a plurality of sub-components, the method comprising:

creating a plurality of test executive sequences in response to user input, wherein each test executive sequence maps to a sub-component of the system and is operable to test the respective sub-component, wherein the plurality of test executive sequences are configured to execute according to a hierarchy, wherein the hierarchy of subcomponents comprises a top level and a first level, wherein the first level is lower in the hierarchy than the toil level;

calling a first test executive sequence at the first level in the hierarchy to test a first sub-component, wherein the first level in the hierarchy correspond to only a subset of the plurality of test executive sequences; and

executing the first test executive sequence without executing test executive sequences above the first level in the hierarchy,

wherein said executing the first test executive sequence tests the first sub-component.

19. The method of claim 18, further comprising:

executing only a portion of one or more test executive sequences above the first level in the hierarchy, for purposes of setting up data used in the first test executive sequence.

20. The method of claim 18,

wherein said executing the first test executive sequence without executing test executive sequences above the first level in the hierarchy comprises executing only setup portions of the test executive sequences that are above the first level in the hierarchy;

wherein the setup portions are operable to set variable values used in the first test executive sequence.

21. A computer-implemented method for testing a sub-component of a system, wherein the system comprises a plurality of sub-components, the method comprising:

creating a plurality of test executive sequences in response to user input, wherein each test executive sequence maps to a sub-component of the system and is operable to test the respective sub-component, wherein the plurality of test executive sequences are configured to execute according to a hierarchy, wherein the hierarchy of subcomponents comprises a top level and a first level, wherein the first level is lower in the hierarchy than the top level;

calling a first test executive sequence at a first level in the hierarchy to test a first sub-component, wherein the first level in the hierarchy corresponds to only subset of the plurality of test executive sequences;

executing only a portion of one or more test executive sequences above the first level in the hierarchy after said calling; and

executing the first test executive sequence after said executing only a portion of one or more test executive sequences above the first level in the hierarchy;

wherein said executing the first test executive sequence tests the first sub-component.

22. The method of claim 21,

wherein said executing only a portion of one or more test executive sequences above the first level in the hierarchy is performed for purposes of setting up data used in the first test executive sequence.

23. The method of claim 21,

wherein said executing only a portion of one or more test executive sequences above the first level in the hierarchy is performed for purposes of initializing hardware device.

24. The method of claim 21,

wherein said executing only a portion of one or more test executive sequences above the first level in the hierarchy comprises executing only setup portions of the test executive sequences that are above the first level in the hierarchy;

wherein the setup portions are operable to set variable values used in the first test executive sequence.

25. The method of claim 21,

wherein said executing only a portion of one or more test executive sequences above the first level in the hierarchy does not include performing functional testing of the sub-components above the first level in the hierarchy.

26. The method of claim 21,

wherein each test executive sequence includes a setup group of steps and a main group of steps;

wherein said executing only a portion of one or more test executive sequences above the first level in the hierarchy comprises executing steps in the setup groups of the test executive sequences above the first level in the hierarchy but does not comprise executing steps in the main groups above the first level in the hierarchy.

27. The method of claim 21, further comprising:

displaying the plurality of test executive sequences on a graphical user interface in a hierarchical manner corresponding to the sub-components of the system.

28. The method of claim 21,

wherein said calling the first test executive sequence is performed in response to user input requesting to call the first test executive sequence.

29. The method of claim 28, further comprising:

displaying the plurality of test executive sequences on a graphical user interface; and

receiving user input requesting to call the first test executive sequence from the displayed plurality of test executive sequences.

30. The method of claim 21,

wherein said calling the first test executive sequence is performed in response to an application programming interface (API) call to execute the first test executive sequence.

31. A computer-implemented method for executing a test executive sequence, the method comprising:

creating a plurality of test executive sequences in response to user input, wherein each test executive sequence comprises a plurality of test executive steps, wherein the plurality of test executive sequences are configured to execute according to hierarchy, wherein the hierarchy comprises a top level and a first level, wherein the first level is lower in the hierarchy than the top level;

hierarchically calling a first test executive sequence from the plurality of test executive sequences; and

hierarchically executing the first test executive sequence in response to said hierarchically calling the first test executive sequence.

32. The method of claim 31,

wherein said hierarchically calling the first test executive sequence comprises specifying a hierarchical path to the first test executive sequence, wherein the hierarchical path specifies a hierarchical path of one or more test executive sequences from the plurality of test executive sequences;

wherein said hierarchically executing the first test executive sequence comprises executing the first test executive sequence from the context of the hierarchical path of test executive sequences.

33. The method of claim 32,

wherein said executing the first test executive sequence from the context of the hierarchical path of test executive sequences comprises partially executing the test executive sequences in the hierarchical path to set up variable values used by the first test executive sequence, but does not comprise executing code modules reference by the test executive sequences in the hierarchical path.

34. The method of claim 32,

wherein one or more of the test executive sequences in the hierarchical path of test executive sequences includes:

a group of setup steps; and

a group of functional steps;

wherein said executing the first test executive sequence from the context of the hierarchical path of test executive sequences comprises executing the setup steps for the test executive sequences in the hierarchical path but does not comprise executing the functional steps for the test executive sequences in the hierarchical path.

35. The method of claim 32,

wherein the hierarchical path of test executive sequences includes a second test executive sequence operable to set a first variable to a first value, wherein the first test executive sequence utilizes the first variable;

wherein said executing the first test executive sequence from the context of the hierarchical path of test executive sequences comprises executing the first test executive sequence with the first variable set to the first value.

36. The method of claim 35,

wherein said executing the first test executive sequence from the context of the hierarchical path of test executive sequences does not comprise complete execution of the second test executive sequence.

37. The method of claim 32,

wherein the first test executive sequence utilizes a first variable;

wherein the hierarchical path of test executive sequences includes a second test executive sequence operable to;

set the first variable to a first value;

invoke one or more code modules;

wherein said executing the first test executive sequence from the context of the hierarchical path of test executive sequences comprises partially executing the second test executive sequence;

wherein said partially executing the second test executive sequence comprises setting the first variable the first value but does not comprise invoking the one or more code modules.

38. A computer-implemented method for testing a unit under test, the method comprising:

creating a test hierarchy including a plurality of test executive sequences for testing the unit under test, wherein the plurality of rest executive sequences are configured to execute according to a hierarchy, wherein the test hierarchy includes a first test executive sequence operable to call a second test executive sequence, and

executing the second test executive sequence without executing the first test executive sequence, wherein said executing the second test executive sequence comprises utilizing data from the first test executive sequence;

wherein the second test executive sequence is operable to test at least portion of the unit under test.

39. The method of claim 38, further comprising:

programmatically calling the second test executive sequence;

wherein said executing the second test executive sequence is performed in response to said programmatically calling the second test executive sequence.

40. The method of claim 39,

wherein said programmatically calling the second test executive sequence comprises specifying a hierarchical path to the second test executive sequence, wherein the hierarchical path includes the first test executive sequence.

41. A computer-implemented method for calling a test executive sequence, the method comprising:

creating a chain of test executive sequences in response to user input, wherein the chain of test executive sequences includes one top-level test executive sequence and one or more non-top-level test executive sequences; and

executing a first non-top-level test executive sequence directly without executing the top-level test executive sequence, wherein the first non-top-level test executive sequence in the hierarchy comprises a subset of the chain of test executive sequences;

wherein the first non-top-level test executive sequence is executable to test a portion of the system.

42. The method of claim 41, further comprising:

executing at least a portion of one or more test executive sequence(s) above the first non-top-level test executive sequence in the chain, for setting up data used by the first non-top-level test executive sequence.

43. A method for testing a portion of a hierarchical system, the method comprising:

displaying a hierarchical view of a test executive sequence hierarchy, wherein the test executive sequence hierarchy corresponds to only a top level and a first level, wherein the first level is lower in the hierarchy than the top level;

receiving user input specifying the first displayed test executive sequence, wherein the first level comprises the first displayed test executive sequence; and

executing the first test executive sequence in response to the user input;

wherein said executing the first test executive sequence does not include executing test executive sequences above the first test executive sequence in the hierarchy; and

wherein said executing the first test executive sequence comprises testing the portion of the hierarchical system.

44. A method for enabling propagation of a first local variable of a first test executive sequence, the method comprising;

receiving user input indicating a desire to propagate the first local variable to a subsequence of the first test executive sequence;

executing the first test executive sequence, wherein said executing the first test executive sequence comprises propagating the first local variable to the subsequence.

45. The method of claim 44,

wherein the subsequence has a second local variable named identically to the first local variable;

wherein the first local variable has a first value;

wherein the second local variable has a second value;

wherein said propagating the first local variable to the subsequence comprises overriding the second value with the first value.