Test executive system with processing of results data in chunks

• US 9,495,265 B2
• Filed: 08/13/2013
• Issued: 11/15/2016
• Est. Priority Date: 06/29/2012
• Status: Active Grant

• Current Assignee: National Instruments Corporation (Emerson Electric Company)
• Original Assignee: National Instruments Corporation (Emerson Electric Company)
• Inventors: Grey, James A.
• Primary Examiner(s): Bonzo, Bryce
• Assistant Examiner(s): Arcos, Jeison C
• Application Number:
  US13/965,462
• Publication Number:
  US 20140006860A1
• Time in Patent Office: 1,190 Days
• US Class Current: 1/1
• CPC Class Codes: G01R 31/31707 Test strategies methods for generation of test sequences G01R31/318371
• CPC Class Codes: G01R 31/318371 Methodologies therefor, e.g. algorithms, procedures
• CPC Class Codes: G06F 11/2273 Test methods
• CPC Class Codes: G06F 11/263 Generation of test inputs, e.g. test vectors, patterns or sequences ; with adaptation of the tested hardware for testability with external testers

Abstract

Test executive system and method of use. The system includes a test executive engine, configured to execute at least one test executive sequence to test at least one unit under test (UUT), a process model that specifies one or more function sequences for pre-test or post-test functionality for the test executive sequences, and a plug-in framework, configured to selectively incorporate one or more process model plug-in instances in the process model. Each process model plug-in instance specifies at least one respective function sequence for pre-test or post-test functionality for the test executive sequences.

Claims

1. A computer-implemented method comprising:

utilizing a computer to perform;

creating a test executive sequence, wherein the test executive sequence includes a first plurality of steps to be performed by a test executive engine to test a unit under test (UUT); and

configuring the test executive sequence with a process model, wherein the process model includes a second plurality of steps to be performed by the test executive engine before and/or after executing the first plurality of steps, wherein the process model is configured to accept process model plug-ins to define the second plurality of steps, wherein said configuring the test executive sequence with the process model comprises selecting one or more process model plug-ins for the process model, wherein each process model plug-in includes two or more of the second plurality of steps;

wherein the one or more process model plug-ins include a first process model plug-in configured to store and/or process test result data dispatched to the first process model plug-in in chunks, wherein chunk size is controllable by an auto-tuned threshold that is based on time measured for the dispatch and processing of prior results.

2. The method of claim 1, wherein chunk size is configurable by user input.

3. The method of claim 1, wherein chunk size is further controllable by one or more specified criteria, comprising:

one or more programmatic or graphical user interface (GUI) settable options.

4. The method of claim 3, wherein the one or more programmatic or graphical user interface (GUI) settable options comprise one or more of:

maximum number of results per dispatch;

or

maximum time between dispatches.

5. The method of claim 3, wherein the one or more policies that enforce implementation requirements of the first process model plug-in comprise one or more of:

upon entry to a subsequence;

or

upon exit from a subsequence.

6. The method of claim 1, wherein the auto-tuned threshold comprises a time threshold between chunks that automatically ensures that the dispatch and processing time does not exceed the time threshold the user specified for batching results.

7. The method of claim 1, wherein the auto-tuned threshold comprises a maximum threshold determined by explicit requirements and policy requirements of a union of all active plug-ins that process results as they are generated.

8. The method of claim 1, further comprising:

testing the UUT, wherein said testing comprises executing each of the process model plug-ins, comprising;

accumulating and dispatching test result data to the first process model plug-in in chunks; and

the first process model plug-in storing and/or processing the test result data in chunks.

9. The method of claim 1, further comprising:

receiving user input via a graphical user interface selecting each of the one or more process model plug-ins for the process model.

10. A non-transitory computer-accessible memory medium that stores program instructions executable by one or more processors to implement:

creating a test executive sequence, wherein the test executive sequence includes a first plurality of steps to be performed by a test executive engine to test a unit under test (UUT); and

configuring the test executive sequence with a process model, wherein the process model includes a second plurality of steps to be performed by the test executive engine before and/or after executing the first plurality of steps, wherein the process model is configured to accept process model plug-ins to define the second plurality of steps, wherein said configuring the test executive sequence with the process model comprises selecting one or more process model plug-ins for the process model, wherein each process model plug-in includes two or more of the second plurality of steps;

wherein the one or more process model plug-ins include a first process model plug-in configured to store and/or process test result data dispatched to the first process model plug-in in chunks, wherein chunk size is controllable by an auto-tuned threshold that is based on time measured for the dispatch and processing of prior results.

11. The non-transitory computer-accessible memory medium of claim 10, wherein chunk size is configurable by user input.

12. The non-transitory computer-accessible memory medium of claim 10, wherein chunk size is controllable by one or more specified criteria, comprising:

one or more programmatic or graphical user interface (GUI) settable options.

13. The non-transitory computer-accessible memory medium of claim 12, wherein the one or more programmatic or graphical user interface (GUI) settable options comprise one or more of:

maximum number of results per dispatch;

or

maximum time between dispatches.

14. The non-transitory computer-accessible memory medium of claim 12, wherein the one or more policies that enforce implementation requirements of the first process model plug-in comprise one or more of:

upon entry to a subsequence;

or

upon exit from a sub sequence.

15. The non-transitory computer-accessible memory medium of claim 10, wherein the auto-tuned threshold comprises a time threshold between chunks that automatically ensures that the dispatch and processing time does not exceed the time threshold the user specified for batching results.

16. The non-transitory computer-accessible memory medium of claim 10, wherein the auto-tuned threshold comprises a maximum threshold determined by explicit requirements and policy requirements of a union of all active plug-ins that process results as they are generated.

17. The non-transitory computer-accessible memory medium of claim 12, wherein the program instructions are further executable to:

test the UUT, including executing each of the process model plug-ins, comprising;

accumulating and dispatching test result data to the first process model plug-in in chunks; and

the first process model plug-in storing and/or processing the test result data in chunks.

18. The non-transitory computer-accessible memory medium of claim 12, wherein the program instructions are further executable to:

receive user input via a graphical user interface selecting each of the one or more process model plug-ins for the process model.

19. A system comprising:

one or more processors; and

memory storing program instructions, wherein the program instructions are executable by the one or more processors to implement;

creating a test executive sequence, wherein the test executive sequence includes a first plurality of steps to be performed by a test executive engine to test a unit under test (UUT); and

configuring the test executive sequence with a process model, wherein the process model includes a second plurality of steps to be performed by the test executive engine before and/or after executing the first plurality of steps, wherein the process model is configured to accept process model plug-ins to define the second plurality of steps, wherein said configuring the test executive sequence with the process model comprises selecting one or more process model plug-ins for the process model, wherein each process model plug-in includes two or more of the second plurality of steps;

wherein the one or more process model plug-ins include a first process model plug-in configured to store and/or process test result data dispatched to the first process model plug-in in chunks, wherein chunk size is controllable by an auto-tuned threshold that is based on time measured for the dispatch and processing of prior results.