Test executive which provides heap validity checking and memory leak detection for user code modules

• US 20060143537A1
• Filed: 04/07/2005
• Published: 06/29/2006
• Est. Priority Date: 12/21/2004
• Status: Active Grant

• Current Assignee: National Instruments Corporation (Emerson Electric Company)
• Original Assignee: National Instruments Corporation (Emerson Electric Company)
• Inventors: Grey, James A.
• Granted Patent: US 7,519,867 B2
• Time in Patent Office: Days
• US Class Current: 714/38
• CPC Class Codes: G06F 11/3688 for test execution, e.g. scheduling of test suites

Abstract

A system and method for automatically detecting heap corruption errors and memory leak errors caused by user-supplied code modules that are called by steps of a test executive sequence. The test executive sequence may first be created by including a plurality of test executive steps in the test executive sequence and configuring at least a subset of the steps to call user-supplied code modules. The test executive sequence may then be executed on a host computer under control of a test executive engine. For each step that calls a user-supplied code module, the test executive engine may perform certain actions to automatically detect whether the user-supplied code module causes a heap corruption error and/or automatically detect whether the user-supplied code module causes a memory leak error.

Claims

1. A carrier medium comprising program instructions executable to:

include a plurality of test executive steps in a test executive sequence in response to user input;

configure each of the test executive steps to call a user-supplied code module in response to user input; and

execute the test executive sequence, wherein said executing comprises executing each of the test executive steps in the test executive sequence;

wherein for each test executive step in the test executive sequence, said executing the test executive step comprises;

invoking execution of the user-supplied code module called by the test executive step; and

checking validity of a first heap after the user-supplied code module called by the test executive step finishes execution to determine whether the user-supplied code module caused the first heap to become corrupted.

2. The carrier medium of claim 1, wherein for each test executive step in the test executive sequence, said executing the test executive step further comprises:

checking validity of the first heap before said invoking execution of the user-supplied code module called by the test executive step to determine whether the first heap is corrupted before the user-supplied code module is executed.

3. The carrier medium of claim 1,

wherein one or more test executive steps in the test executive sequence have pre-functionality, wherein the pre-functionality for each of the one or more test executive steps comprises functionality to be performed before invoking execution of the user-supplied code module called by the test executive step;

wherein for each test executive step in the test executive sequence that has pre-functionality, said executing the test executive step further comprises executing the pre-functionality of the step before said invoking execution of the user-supplied code module called by the test executive step.

4. The carrier medium of claim 1,

wherein one or more test executive steps in the test executive sequence have post-functionality, wherein the post-functionality for each of the one or more test executive steps comprises functionality to be performed after the user-supplied code module called by the test executive step finishes execution;

wherein for each test executive step in the test executive sequence that has post-functionality, said executing the test executive step further comprises executing the post-functionality of the step after the user-supplied code module called by the test executive step finishes execution.

5. The carrier medium of claim 1,

wherein the program instructions are further executable to stop execution of the test executive sequence in response to determining that the user-supplied code module caused the first heap to become corrupted.

6. The carrier medium of claim 1,

wherein the program instructions are further executable to report an error in response to determining that the user-supplied code module caused the first heap to become corrupted.

7. The carrier medium of claim 1, wherein for each test executive step in the test executive sequence, said executing the test executive step further comprises:

checking validity of a second heap after the user-supplied code module called by the test executive step finishes execution to determine whether the user-supplied code module caused the second heap to become corrupted.

8. The carrier medium of claim 1,

wherein the program instructions are further executable to display a graphical user interface for creating the test executive sequence;

wherein said including the plurality of test executive steps in the test executive sequence is performed in response to user input received to the graphical user interface requesting inclusion of the test executive steps in the test executive sequence.

9. The carrier medium of claim 1,

wherein for each test executive step in the test executive sequence, the program instructions are further executable to display a graphical user interface for configuring the test executive step;

wherein for each test executive step in the test executive sequence, the test executive step is configured to call a user-supplied code module in response to user input received to the graphical user interface for configuring the test executive step, wherein the user input specifies the user-supplied code module.

10. The carrier medium of claim 1, wherein for a first test executive step in the test executive sequence, said invoking execution of the user-supplied code module called by the first test executive step comprises one of:

calling a function in a DLL;

invoking a method of an ActiveX object;

or

invoking execution of a graphical program.

11. The carrier medium of claim 1,

wherein said executing the test executive sequence comprises executing the test executive sequence under control of a test executive engine;

wherein the test executive engine is operable to perform said checking validity of the first heap.

12. A computer-implemented method for discovering a heap corruption error caused by a test executive sequence, the method comprising:

including a plurality of test executive steps in the test executive sequence in response to user input;

configuring each of the test executive steps to call a user-supplied code module in response to user input; and

executing the test executive sequence, wherein said executing comprises executing each of the test executive steps in the test executive sequence;

wherein for each test executive step in the test executive sequence, said executing the test executive step comprises;

invoking execution of the user-supplied code module called by the test executive step; and

checking validity of a first heap after the user-supplied code module called by the test executive step finishes execution to determine whether the user-supplied code module caused the first heap to become corrupted.

13. The method of claim 12, wherein for each test executive step in the test executive sequence, said executing the test executive step further comprises:

checking validity of the first heap before said invoking execution of the user-supplied code module called by the test executive step to determine whether the first heap is corrupted before the user-supplied code module is executed.

14. The method of claim 12,

wherein one or more test executive steps in the test executive sequence have pre-functionality, wherein the pre-functionality for each of the one or more test executive steps comprises functionality to be performed before invoking execution of the user-supplied code module called by the test executive step;

wherein for each test executive step in the test executive sequence that has pre-functionality, said executing the test executive step further comprises executing the pre-functionality of the step before said invoking execution of the user-supplied code module called by the test executive step.

15. The method of claim 12,

wherein one or more test executive steps in the test executive sequence have post-functionality, wherein the post-functionality for each of the one or more test executive steps comprises functionality to be performed after the user-supplied code module called by the test executive step finishes execution;

wherein for each test executive step in the test executive sequence that has post-functionality, said executing the test executive step further comprises executing the post-functionality of the step after the user-supplied code module called by the test executive step finishes execution.

16. The method of claim 12, further comprising:

stopping execution of the test executive sequence in response to determining that the user-supplied code module caused the first heap to become corrupted.

17. The method of claim 12, further comprising:

reporting an error in response to determining that the user-supplied code module caused the first heap to become corrupted.

18. The method of claim 12, wherein for each test executive step in the test executive sequence, said executing the test executive step further comprises:

checking validity of a second heap after the user-supplied code module called by the test executive step finishes execution to determine whether the user-supplied code module caused the second heap to become corrupted.

19. The method of claim 12,

wherein said executing the test executive sequence comprises executing the test executive sequence under control of a test executive engine;

wherein the test executive engine is operable to perform said checking validity of the first heap.

20. A system comprising:

a sequence editor;

a test executive engine;

a host computer operable to execute the sequence editor and the test executive engine; and

a unit under test (UUT) coupled to the host computer;

wherein the host computer is operable to execute the sequence editor to create a test executive sequence for testing the UUT, wherein said creating the test executive sequence comprises including a plurality of test executive steps in the test executive sequence in response to user input to the sequence editor and configuring each of the test executive steps to call a user-supplied code module in response to user input to the sequence editor;

wherein the host computer is further operable to execute the test executive engine, wherein the test executive engine is operable to control execution of the test executive sequence, wherein said controlling execution of the test executive sequence comprises causing each of the test executive steps in the test executive sequence to be executed on the host computer;

wherein for each test executive step in the test executive sequence, the test executive engine is operable to;

invoke the user-supplied code module called by the test executive step for execution on the host computer; and

check validity of a first heap after the user-supplied code module called by the test executive step finishes execution to determine whether the user-supplied code module caused the first heap to become corrupted.

21. The system of claim 20,

wherein for each test executive step in the test executive sequence, the test executive engine is further operable to check validity of the first heap before said invoking the user-supplied code module called by the test executive step to determine whether the first heap is corrupted before the user-supplied code module is executed.

22. The system of claim 20,

wherein one or more test executive steps in the test executive sequence have pre-functionality, wherein the pre-functionality for each of the one or more test executive steps comprises functionality to be performed before executing the user-supplied code module called by the test executive step, wherein the pre-functionality for each of the one or more test executive steps is implemented by the test executive engine;

wherein for each test executive step in the test executive sequence that has pre-functionality, the test executive engine is further operable to execute the pre-functionality of the step before said invoking the user-supplied code module called by the step.

23. The system of claim 20,

wherein one or more test executive steps in the test executive sequence have post-functionality, wherein the post-functionality for each of the one or more test executive steps comprises functionality to be performed after executing the user-supplied code module called by the test executive step, wherein the post-functionality for each of the one or more test executive steps is implemented by the test executive engine;

wherein for each test executive step in the test executive sequence that has post-functionality, the test executive engine is further operable to execute the post-functionality of the step after said invoking the user-supplied code module called by the step.

24. A carrier medium comprising program instructions executable to:

include a plurality of test executive steps in a test executive sequence in response to user input;

configure each of the test executive steps to call a user-supplied code module in response to user input; and

execute the test executive sequence, wherein said executing comprises executing each of the test executive steps in the test executive sequence;

wherein for each test executive step in at least a subset of the test executive steps in the test executive sequence, said executing the test executive step comprises;

checking memory usage before invoking execution of the user-supplied code module called by the test executive step;

invoking execution of the user-supplied code module called by the test executive step; and

checking memory usage again after the user-supplied code module called by the test executive step finishes execution to determine whether the user-supplied code module causes a memory leak.

25. The carrier medium of claim 24,

wherein for each test executive step in the subset, said checking memory usage before invoking execution of the user-supplied code module called by the test executive step comprises checking an amount of memory in use for a heap;

wherein for each test executive step in the subset, said checking memory usage again after the user-supplied code module called by the test executive step finishes execution comprises checking the amount of memory in use for the heap again.

26. The carrier medium of claim 24,

wherein for each test executive step in the subset, said determining whether the user-supplied code module called by the test executive step causes a memory leak comprises determining whether memory usage after the user-supplied code module called by the test executive step finishes execution is greater than memory usage before said invoking execution of the user-supplied code module called by the test executive step.

27. The carrier medium of claim 24, wherein the program instructions are further executable to:

receive user input specifying a threshold memory amount for determining whether a first user-supplied code module called by a first test executive step in the test executive sequence causes a memory leak;

wherein determining whether the first user-supplied code module called by the first test executive step causes a memory leak comprises determining whether memory usage after the first user-supplied code module called by the first test executive step finishes execution is greater than memory usage before invoking execution of the first user-supplied code module called by the first test executive step by at least the threshold memory amount.

28. The carrier medium of claim 24, wherein the program instructions are further executable to:

receive user input specifying one or more of the test executive steps in the test executive sequence for which memory leak checking is desired;

wherein said checking memory usage is performed when executing the specified one or more test executive steps but not when executing other test executive steps in the test executive sequence.

29. The carrier medium of claim 24,

wherein one or more test executive steps in the test executive sequence have pre-functionality, wherein the pre-functionality for each of the one or more test executive steps comprises functionality to be performed before invoking execution of the user-supplied code module called by the test executive step;

wherein for each test executive step in the subset that has pre-functionality, said executing the test executive step further comprises executing the pre-functionality of the step before said invoking execution of the user-supplied code module called by the test executive step.

30. The carrier medium of claim 24,

wherein one or more test executive steps in the test executive sequence have post-functionality, wherein the post-functionality for each of the one or more test executive steps comprises functionality to be performed after the user-supplied code module called by the test executive step finishes execution;

wherein for each test executive step in the subset that has post-functionality, said executing the test executive step further comprises executing the post-functionality of the step after the user-supplied code module called by the test executive step finishes execution.

31. The carrier medium of claim 24,

wherein the program instructions are further executable to stop execution of the test executive sequence in response to determining that a user-supplied code module called by a first test executive step causes a memory leak.

32. The carrier medium of claim 24,

wherein the program instructions are further executable to report an error in response to determining that a user-supplied code module called by a first test executive step causes a memory leak.

33. The carrier medium of claim 24,

wherein the program instructions are further executable to display a graphical user interface for creating the test executive sequence;

wherein said including the plurality of test executive steps in the test executive sequence is performed in response to user input received to the graphical user interface requesting inclusion of the test executive steps in the test executive sequence.

34. The carrier medium of claim 24,

wherein for each test executive step in the test executive sequence, the program instructions are further executable to display a graphical user interface for configuring the test executive step;

wherein for each test executive step in the test executive sequence, the test executive step is configured to call a user-supplied code module in response to user input received to the graphical user interface for configuring the test executive step, wherein the user input specifies the user-supplied code module.

35. The carrier medium of claim 24, wherein for a first test executive step in the subset, said invoking execution of the user-supplied code module called by the first test executive step comprises one of:

calling a function in a DLL;

invoking a method of an ActiveX object;

or

invoking execution of a graphical program.

36. The carrier medium of claim 24,

wherein said executing the test executive sequence comprises executing the test executive sequence under control of a test executive engine;

wherein the test executive engine is operable to perform said checking memory usage.

37. A method for discovering a memory leak error caused by a test executive sequence, the method comprising:

including a plurality of test executive steps in the test executive sequence in response to user input;

configuring each of the test executive steps to call a user-supplied code module in response to user input; and

executing the test executive sequence, wherein said executing comprises executing each of the test executive steps in the test executive sequence;

wherein for each test executive step in at least a subset of the test executive steps in the test executive sequence, said executing the test executive step comprises;

checking memory usage before invoking execution of the user-supplied code module called by the test executive step;

invoking execution of the user-supplied code module called by the test executive step; and

checking memory usage again after the user-supplied code module called by the test executive step finishes execution to determine whether the user-supplied code module causes a memory leak.

38. The method of claim 37,

wherein for each test executive step in the subset, said checking memory usage before invoking execution of the user-supplied code module called by the test executive step comprises checking an amount of memory in use for a heap;

wherein for each test executive step in the subset, said checking memory usage again after the user-supplied code module called by the test executive step finishes execution comprises checking the amount of memory in use for the heap again.

39. The method of claim 37,

wherein for each test executive step in the subset, said determining whether the user-supplied code module called by the test executive step causes a memory leak comprises determining whether memory usage after the user-supplied code module called by the test executive step finishes execution is greater than memory usage before said invoking execution of the user-supplied code module called by the test executive step.

40. The method of claim 37,

wherein one or more test executive steps in the test executive sequence have pre-functionality, wherein the pre-functionality for each of the one or more test executive steps comprises functionality to be performed before invoking execution of the user-supplied code module called by the test executive step;

wherein for each test executive step in the subset that has pre-functionality, said executing the test executive step further comprises executing the pre-functionality of the step before said invoking execution of the user-supplied code module called by the test executive step.

41. The method of claim 37,

wherein one or more test executive steps in the test executive sequence have post-functionality, wherein the post-functionality for each of the one or more test executive steps comprises functionality to be performed after the user-supplied code module called by the test executive step finishes execution;

wherein for each test executive step in the subset that has post-functionality, said executing the test executive step further comprises executing the post-functionality of the step after the user-supplied code module called by the test executive step finishes execution.

42. A system comprising:

a sequence editor;

a test executive engine;

a host computer operable to execute the sequence editor and the test executive engine; and

a unit under test (UUT) coupled to the host computer;

wherein the host computer is operable to execute the sequence editor to create a test executive sequence for testing the UUT, wherein said creating the test executive sequence comprises including a plurality of test executive steps in the test executive sequence in response to user input to the sequence editor and configuring each of the test executive steps to call a user-supplied code module in response to user input to the sequence editor;

wherein the host computer is further operable to execute the test executive engine, wherein the test executive engine is operable to control execution of the test executive sequence, wherein said controlling execution of the test executive sequence comprises causing each of the test executive steps in the test executive sequence to be executed on the host computer;

wherein for each test executive step in at least a subset of the test executive steps in the test executive sequence, the test executive engine is operable to;

check memory usage before invoking execution of the user-supplied code module called by the test executive step;

invoke the user-supplied code module called by the test executive step for execution on the host computer; and

check memory usage again after the user-supplied code module called by the test executive step finishes execution to determine whether the user-supplied code module causes a memory leak.

43. The system of claim 42,

wherein one or more test executive steps in the test executive sequence have pre-functionality, wherein the pre-functionality for each of the one or more test executive steps comprises functionality to be performed before invoking execution of the user-supplied code module called by the test executive step, wherein the pre-functionality for each of the one or more test executive steps is implemented by the test executive engine;

wherein for each test executive step in the test executive sequence that has pre-functionality, the test executive engine is further operable to execute the pre-functionality of the step before invoking execution of the user-supplied code module called by the step.

44. The system of claim 42,

wherein one or more test executive steps in the test executive sequence have post-functionality, wherein the post-functionality for each of the one or more test executive steps comprises functionality to be performed after the user-supplied code module called by the test executive step finishes execution, wherein the post-functionality for each of the one or more test executive steps is implemented by the test executive engine;

wherein for each test executive step in the test executive sequence that has post-functionality, the test executive engine is further operable to execute the post-functionality of the step after the user-supplied code module called by the step finishes execution.