Test executive system with memory leak detection for user code modules

• US 7,954,009 B2
• Filed: 03/06/2009
• Issued: 05/31/2011
• Est. Priority Date: 12/21/2004
• Status: Active Grant

• Current Assignee: National Instruments Corporation (Emerson Electric Company)
• Original Assignee: National Instruments Corporation (Emerson Electric Company)
• Inventors: Grey, James A.
• Primary Examiner(s): Chu; Gabriel L
• Application Number:
  US12/399,250
• Publication Number:
  US 20090172476A1
• Time in Patent Office: 816 Days
• Field of Search: None
• US Class Current: 717/131
• CPC Class Codes: G06F 11/3688 for test execution, e.g. scheduling of test suites

Abstract

A system and method for automatically detecting heap corruption errors and memory leak errors caused by user-supplied code modules that are called by steps of a test executive sequence. The test executive sequence may first be created by including a plurality of test executive steps in the test executive sequence and configuring at least a subset of the steps to call user-supplied code modules. The test executive sequence may then be executed on a host computer under control of a test executive engine. For each step that calls a user-supplied code module, the test executive engine may perform certain actions to automatically detect whether the user-supplied code module causes a heap corruption error and/or automatically detect whether the user-supplied code module causes a memory leak error.

Claims

1. A computer-readable memory medium comprising program instructions executable to:

include a plurality of test executive steps in a test executive sequence in response to user input;

configure each test executive step of the plurality of test executive steps to call a respective user-supplied code module in response to user input; and

execute the test executive sequence, wherein said executing comprises executing each test executive step of the plurality of test executive steps;

wherein for each test executive step of at least a subset of the plurality of test executive steps, said executing the test executive step comprises;

checking memory usage before invoking execution of the user-supplied code module called by the test executive step;

invoking execution of the user-supplied code module called by the test executive step; and

checking memory usage again after the user-supplied code module called by the test executive step finishes execution to determine whether the user-supplied code module causes a memory leak;

wherein one or more test executive steps of the subset have pre-functionality, wherein the pre-functionality for each of the one or more test executive steps comprises functionality to be performed before invoking execution of the user-supplied code module called by the test executive step;

wherein for each test executive step of the subset that has pre-functionality, said executing the test executive step further comprises executing the pre-functionality of the step before said checking the memory usage before invoking execution of the user-supplied code module called by the test executive step.

2. The computer-readable memory medium of claim 1,

wherein for each test executive step of the subset, said checking memory usage before invoking execution of the user-supplied code module called by the test executive step comprises checking an amount of memory in use for a heap;

wherein for each test executive step of the subset, said checking memory usage again after the user-supplied code module called by the test executive step finishes execution comprises checking the amount of memory in use for the heap again.

3. The computer-readable memory medium of claim 1,

wherein for each test executive step of the subset, said determining whether the user-supplied code module called by the test executive step causes a memory leak comprises determining whether memory usage after the user-supplied code module called by the test executive step finishes execution is greater than memory usage before said invoking execution of the user-supplied code module called by the test executive step.

4. The computer-readable memory medium of claim 1, wherein the program instructions are further executable to:

receive user input specifying a threshold memory amount for determining whether a first user-supplied code module called by a first test executive step in the test executive sequence causes a memory leak;

wherein determining whether the first user-supplied code module called by the first test executive step causes a memory leak comprises determining whether memory usage after the first user-supplied code module called by the first test executive step finishes execution is greater than memory usage before invoking execution of the first user-supplied code module called by the first test executive step by at least the threshold memory amount.

5. The computer-readable memory medium of claim 1, wherein the program instructions are further executable to:

receive user input specifying one or more test executive steps of the plurality of test executive steps for which memory leak checking is desired;

wherein said checking memory usage is performed when executing the specified one or more test executive steps but not when executing other test executive steps of the plurality of test executive steps.

6. The computer-readable memory medium of claim 1,

wherein one or more test executive steps of the subset have post-functionality, wherein the post-functionality for each of the one or more test executive steps comprises functionality to be performed after the user-supplied code module called by the test executive step finishes execution;

wherein for each test executive step of the subset that has post-functionality, said executing the test executive step further comprises executing the post-functionality of the step after said checking the memory usage again after the user-supplied code module called by the test executive step finishes execution.

7. The computer-readable memory medium of claim 1,

wherein the program instructions are further executable to stop execution of the test executive sequence in response to determining that a user-supplied code module called by a first test executive step causes a memory leak.

8. The computer-readable memory medium of claim 1,

wherein the program instructions are further executable to report an error in response to determining that a user-supplied code module called by a first test executive step causes a memory leak.

9. The computer-readable memory medium of claim 1,

wherein the program instructions are further executable to display a graphical user interface for creating the test executive sequence;

wherein said including the plurality of test executive steps in the test executive sequence is performed in response to user input received to the graphical user interface requesting inclusion of the test executive steps in the test executive sequence.

10. The computer-readable memory medium of claim 1,

wherein for each test executive step of the plurality of test executive steps, the program instructions are further executable to display a graphical user interface for configuring the test executive step;

wherein for each test executive step of the plurality of test executive steps, the test executive step is configured to call the respective user-supplied code module in response to user input received to the graphical user interface for configuring the test executive step, wherein the user input specifies the respective user-supplied code module.

11. The computer-readable memory medium of claim 1, wherein for a first test executive step in the subset, said invoking execution of the user-supplied code module called by the first test executive step comprises one of:

calling a function in a DLL;

invoking a method of an ActiveX object;

or

invoking execution of a graphical program.

12. The computer-readable memory medium of claim 1,

wherein said executing the test executive sequence comprises executing the test executive sequence under control of a test executive engine;

wherein the test executive engine is configured to perform said checking memory usage.

13. A method for discovering a memory leak error caused by a test executive sequence, the method comprising:

including a plurality of test executive steps in the test executive sequence in response to user input;

configuring each test executive step of the plurality of test executive steps to call a respective user-supplied code module in response to user input; and

executing the test executive sequence, wherein said executing comprises executing each test executive step of the plurality of test executive steps;

wherein for each test executive step of at least a subset of the plurality of test executive steps, said executing the test executive step comprises;

checking memory usage before invoking execution of the user-supplied code module called by the test executive step;

invoking execution of the user-supplied code module called by the test executive step; and

checking memory usage again after the user-supplied code module called by the test executive step finishes execution to determine whether the user-supplied code module causes a memory leak;

wherein one or more test executive steps of the subset have pre-functionality, wherein the pre-functionality for each of the one or more test executive steps comprises functionality to be performed before invoking execution of the user-supplied code module called by the test executive step;

wherein for each test executive step of the subset that has pre-functionality, said executing the test executive step further comprises executing the pre-functionality of the step before said checking the memory usage before invoking execution of the user-supplied code module called by the test executive step.

14. The method of claim 13,

wherein for each test executive step of the subset, said checking memory usage before invoking execution of the user-supplied code module called by the test executive step comprises checking an amount of memory in use for a heap;

wherein for each test executive step of the subset, said checking memory usage again after the user-supplied code module called by the test executive step finishes execution comprises checking the amount of memory in use for the heap again.

15. The method of claim 13,

wherein one or more test executive steps of the subset have post-functionality, wherein the post-functionality for each of the one or more test executive steps comprises functionality to be performed after the user-supplied code module called by the test executive step finishes execution;

wherein for each test executive step of the subset that has post-functionality, said executing the test executive step further comprises executing the post-functionality of the step after said checking the memory usage again after the user-supplied code module called by the test executive step finishes execution.

16. A system comprising:

a sequence editor;

a test executive engine;

a host computer configured to execute the sequence editor and the test executive engine; and

a unit under test (UUT) coupled to the host computer;

wherein the host computer is configured to execute the sequence editor to create a test executive sequence for testing the UUT, wherein said creating the test executive sequence comprises including a plurality of test executive steps in the test executive sequence in response to user input to the sequence editor and configuring each test executive step of the plurality of test executive steps to call a respective user-supplied code module in response to user input to the sequence editor;

wherein the host computer is further configured to execute the test executive engine, wherein the test executive engine is configured to control execution of the test executive sequence, wherein said controlling execution of the test executive sequence comprises causing each test executive step of the plurality of test executive steps to be executed on the host computer;

wherein for each test executive step of at least a subset of the plurality of test executive steps, the test executive engine is configured to;

check memory usage before invoking execution of the user-supplied code module called by the test executive step;

invoke the user-supplied code module called by the test executive step for execution on the host computer; and

check memory usage again after the user-supplied code module called by the test executive step finishes execution to determine whether the user-supplied code module causes a memory leak;

wherein one or more test executive steps of the subset have pre-functionality, wherein the pre-functionality for each of the one or more test executive steps comprises functionality to be performed before invoking execution of the user-supplied code module called by the test executive step, wherein the pre-functionality for each of the one or more test executive steps is implemented by the test executive engine;

wherein for each test executive step of the subset that has pre-functionality, the test executive engine is further configured to execute the pre-functionality of the step before said checking the memory usage before invoking execution of the user-supplied code module called by the test executive step.

17. The system of claim 16,

wherein one or more test executive steps of the subset have post-functionality, wherein the post-functionality for each of the one or more test executive steps comprises functionality to be performed after the user-supplied code module called by the test executive step finishes execution, wherein the post-functionality for each of the one or more test executive steps is implemented by the test executive engine;

wherein for each test executive step of the subset that has post-functionality, the test executive engine is further configured to execute the post-functionality of the step after said checking the memory usage again after the user-supplied code module called by the test executive step finishes execution.

18. A computer-readable memory medium comprising program instructions executable to:

include a plurality of test executive steps in a test executive sequence in response to user input;

configure each test executive step of the plurality of test executive steps to call a respective user-supplied code module in response to user input; and

execute the test executive sequence, wherein said executing comprises executing each test executive step of the plurality of test executive steps;

wherein for each test executive step of at least a subset of the plurality of test executive steps, said executing the test executive step comprises;

checking memory usage before invoking execution of the user-supplied code module called by the test executive step;

invoking execution of the user-supplied code module called by the test executive step; and

checking memory usage again after the user-supplied code module called by the test executive step finishes execution to determine whether the user-supplied code module causes a memory leak;

wherein one or more test executive steps of the subset have post-functionality, wherein the post-functionality for each of the one or more test executive steps comprises functionality to be performed after the user-supplied code module called by the test executive step finishes execution;

wherein for each test executive step of the subset that has post-functionality, said executing the test executive step further comprises executing the post-functionality of the step after said checking the memory usage again after the user-supplied code module called by the test executive step finishes execution.

19. A method for discovering a memory leak error caused by a test executive sequence, the method comprising:

including a plurality of test executive steps in the test executive sequence in response to user input;

configuring each test executive step of the plurality of test executive steps to call a respective user-supplied code module in response to user input; and

executing the test executive sequence, wherein said executing comprises executing each test executive step of the plurality of test executive steps;

wherein for each test executive step of at least a subset of the plurality of test executive steps, said executing the test executive step comprises;

checking memory usage before invoking execution of the user-supplied code module called by the test executive step;

invoking execution of the user-supplied code module called by the test executive step; and

checking memory usage again after the user-supplied code module called by the test executive step finishes execution to determine whether the user-supplied code module causes a memory leak;

wherein one or more test executive steps of the subset have post-functionality, wherein the post-functionality for each of the one or more test executive steps comprises functionality to be performed after the user-supplied code module called by the test executive step finishes execution;

wherein for each test executive step of the subset that has post-functionality, said executing the test executive step further comprises executing the post-functionality of the step after said checking the memory usage again after the user-supplied code module called by the test executive step finishes execution.

20. A system comprising:

a sequence editor;

a test executive engine;

a host computer configured to execute the sequence editor and the test executive engine; and

a unit under test (UUT) coupled to the host computer;

wherein the host computer is configured to execute the sequence editor to create a test executive sequence for testing the UUT, wherein said creating the test executive sequence comprises including a plurality of test executive steps in the test executive sequence in response to user input to the sequence editor and configuring each test executive step of the plurality of test executive steps to call a respective user-supplied code module in response to user input to the sequence editor;

wherein the host computer is further configured to execute the test executive engine, wherein the test executive engine is configured to control execution of the test executive sequence, wherein said controlling execution of the test executive sequence comprises causing each test executive step of the plurality of test executive steps to be executed on the host computer;

wherein for each test executive step of at least a subset of the plurality of test executive steps, the test executive engine is configured to;

check memory usage before invoking execution of the user-supplied code module called by the test executive step;

invoke the user-supplied code module called by the test executive step for execution on the host computer; and

check memory usage again after the user-supplied code module called by the test executive step finishes execution to determine whether the user-supplied code module causes a memory leak;

wherein one or more test executive steps of the subset have post-functionality, wherein the post-functionality for each of the one or more test executive steps comprises functionality to be performed after the user-supplied code module called by the test executive step finishes execution, wherein the post-functionality for each of the one or more test executive steps is implemented by the test executive engine;

wherein for each test executive step of the subset that has post-functionality, the test executive engine is further configured to execute the post-functionality of the step after said checking the memory usage again after the user-supplied code module called by the test executive step finishes execution.