Mechanism for graphical test exclusion

• US 6,980,916 B1
• Filed: 04/29/2004
• Issued: 12/27/2005
• Est. Priority Date: 04/29/2004
• Status: Active Grant

• Current Assignee: Oracle America, Inc. (Oracle Corporation)
• Original Assignee: Sun Microsystems Incorporated (Oracle Corporation)
• Inventors: Katz, Ron, Kuturianu, Olga, Kinsbrunner, Eran
• Primary Examiner(s): Nghiem, Michael
• Assistant Examiner(s): LE, JOHN H
• Application Number:
  US10/834,488
• Time in Patent Office: 607 Days
• Field of Search: 702/117, 702/119, 702/120, 702/123, 714/25, 714/26, 714/738, 717/131, 717/175, 434/365
• US Class Current: 702/119
• CPC Class Codes: G06F 11/3664 Environments for testing or debugging software
• CPC Class Codes: G06F 11/3688 for test execution, e.g. scheduling of test suites

Abstract

Methods and systems for administratively qualifying and disqualifying test programs of a suite as excludable from performance by a client are disclosed in conjunction with a test framework. Using a graphical user interface that represents the suite as a hierarchy or tree, an administrator is able to select test programs or entire divisions of a tree in order to disqualify these elements from exclusion by clients. The results of the operation are compactly stored in a table of excludable records, as pointers to a node of the tree, rather than individual elements of the subtree headed by the node. Using the same graphical user interface, the client is allowed to see only those elements of the tree that have not been disqualified by the administrator. The client can select visible elements of the tree to prevent one or more test programs of the suite from being executed.

Claims

1. A method for managing the composition and execution of test suites in testing a computing product, comprising the steps of:

storing a suite of test programs for execution using said computing product, said suite being organized in a hierarchy;

hierarchically displaying said suite;

selecting qualified test programs from said suite, said qualified test programs being excludable from performance;

transmitting a pointer to a node of said hierarchy to a client, wherein all of said test programs that are located in a subtree headed by said node are included in said qualified test programs; and

executing at least a portion of said test programs using said computing product responsively to said pointer.

2. The method according to claim 1, further comprising the step of hierarchically displaying only said qualified test programs to said client, said qualified test programs being selectable by said client.

3. The method according to claim 2, wherein said step of hierarchically displaying only said qualified test programs to said client is performed by:

responsively to said pointer disposing a representation of a subtree of said suite on a graphical user interface.

4. The method according to claim 3, further comprising the steps of:

selecting one of said qualified test programs via said graphical user interface as an excluded test program; and

while executing said test programs, avoiding performance of said excluded test program.

5. The method according to claim 3, further comprising the steps of:

selecting one of said qualified test programs via a master exclude list as an excluded test program; and

while executing said test programs, avoiding performance of said excluded test program.

6. A computer software product, comprising a computer-readable medium in which computer program instructions are stored, which instructions, when read by a computer, cause the computer to perform a method for managing the composition and execution of test suites in testing a computing product, comprising the steps of:

storing a suite of test programs for execution using said computing product, said suite being organized in a hierarchy;

hierarchically displaying said suite;

memorizing a selection of qualified test programs from said suite, said qualified test programs being excludable from performance;

transmitting a table of said qualified test programs to a client, wherein said table of said qualified test programs comprises a pointer to a node of said hierarchy, wherein all of said test programs that are located in a subtree headed by said node are included in said qualified test programs; and

executing at least a portion of said test programs using said computing product responsively to said pointer.

7. The computer software product according to claim 6, wherein said computer is further instructed to perform the step of hierarchically displaying only said qualified test programs to said client, said qualified test programs being selectable by said client as excludable from performance.

8. The computer software product according to claim 7, wherein said step of hierarchically displaying only said qualified test programs to said client is performed by:

responsively to said pointer displaying a representation of a subtree of said suite via a graphical user interface.

9. The computer software product according to claim 8, further comprising the steps of:

responsively to a selection of one of said qualified test programs via said graphical user interface as an excluded test program;

while executing said test programs, avoiding performance of said excluded test program.

10. The computer software product according to claim 8, further comprising the steps of:

accepting a selection of one of said qualified test programs via a master exclude list as an excluded test program; and

while executing said test programs, avoiding performance of said excluded test program.

11. A verification system for managing the composition and execution of test suites in testing a computing product, comprising:

a server;

a memory in said server for storing a suite of test programs for execution using said computing product, said suite being organized in a hierarchy;

a terminal for hierarchically displaying said suite;

an editor adapted for selection of qualified test programs from said suite, said qualified test programs being excludable from performance; and

a communication facility for transmitting a table of said qualified test programs to a client, wherein said table of said qualified test programs comprises a pointer to a node of said hierarchy, and wherein all of said test programs that are located in a subtree headed by said node are included in said qualified test programs.

12. The verification system according to claim 11, further comprising a client terminal that is controlled by said server, said client terminal hierarchically displaying only said qualified test programs to said client, said qualified test programs being selectable by said client via said client terminal.

13. The verification system according to claim 12, wherein said client terminal is instructed via said server to display a representation of a subtree of said suite on a graphical user interface responsively to said pointer.

14. The verification system according to claim 13, said server being further adapted, responsively to a selection of one of said qualified test programs via said graphical user interface as an excluded test program, to control execution of said suite using said computing product and to avoid performance of said excluded test program.

15. A method for managing the composition and execution of test suites in testing a computing product, comprising the steps of:

receiving a table comprising qualified programs of a suite of test programs, said qualified programs being qualified for exclusion from performance, wherein said suite is organized as a hierarchy of said test programs, and said table comprises a pointer to a node of said hierarchy, and wherein all of said test programs that are located in a subtree headed by said node are included in said qualified programs;

hierarchically displaying said qualified programs;

selecting one of said qualified programs to be excluded from performance to define an excluded test program;

executing said suite of test programs using said computing product; and

while executing said suite of test programs, avoiding performance of said excluded test program.

16. The method according to claim 15, wherein said step of executing said suite of test programs includes executing unqualified test programs of said suite.

17. A method for managing the composition and execution of test suites in testing a computing product, comprising the steps of:

providing a central repository holding a suite of test programs, an execution test harness, an installer, and a graphical user interface for modifying said suite;

displaying said suite as a hierarchy on said graphical user interface;

selecting qualified test programs from said suite via said graphical user interface, said qualified test programs being excludable from performance by a client of said execution test harness; and

transmitting a record of said qualified test programs to said client.

18. The method according to claim 17, wherein said record of said qualified test programs comprises a pointer to a node of said hierarchy, wherein all of said test programs in a subtree headed by said node are included in said qualified test programs.

19. The method according to claim 17, further comprising the step of hierarchically displaying only said qualified test programs to said client, said qualified test programs being selectable by said client as excluded test programs.

20. The method according to claim 19, further comprising the steps of:

downloading said installer to said client;

responsively to an execution of said installer in said client;

downloading and installing at least a portion of said test programs from said central repository to said client for use by said client in testing said computing product; and

avoiding performance of any of said excluded test programs.

21. The method according to claim 17, further comprising the step of selecting one of said qualified test programs via a master exclude list as an excluded test program.

22. A computer software product, comprising a computer-readable medium in which computer program instructions are stored, which instructions, when read by a computer, cause the computer to perform a method for managing the composition and execution of test suites in testing a computing product, comprising the steps of:

receiving a table comprising qualified programs of a suite of test programs, said qualified programs being excludable from performance, wherein said suite is organized as a hierarchy of said test programs, and said table comprises a pointer to a node of said hierarchy, and wherein all of said test programs that are located in a subtree headed by said node are included in said qualified programs;

hierarchically displaying said qualified programs;

selecting one of said qualified programs to be excluded from performance to define an excluded test program;

executing said suite of test programs using said computing product; and

while executing said suite of test programs, avoiding performance of said excluded test program.

23. The computer software product according to claim 22, wherein said step of executing said suite of test programs includes executing unqualified test programs of said suite.

24. A computer software product, comprising a computer-readable medium in which computer program instructions are stored, which instructions, when read by a computer, cause the computer to perform a method for testing a computing product, comprising the steps of:

providing a central repository holding a suite of test programs, an execution test harness, an installer, and a graphical user interface for modifying said suite;

displaying said suite as a hierarchy on said graphical user interface;

using said graphical user interface, selecting qualified test programs from said suite, said qualified test programs being excludable from performance by a client of said execution test harness; and

transmitting a record of said qualified test programs to said client.

25. The computer software product according to claim 24, wherein said record of said qualified test programs comprises a pointer to a node of said hierarchy, wherein all of said test programs in a subtree headed by said node are included in said qualified test programs.

26. The computer software product according to claim 24, wherein said computer is further instructed to perform the step of hierarchically displaying only said qualified test programs to said client, said qualified test programs being selectable by said client as excluded test programs.

27. The computer software product according to claim 26, wherein said computer is further instructed to perform the steps of:

downloading said installer to said client;

responsively to an execution of said installer in said client;

downloading and installing at least a portion of said test programs from said central repository to said client for use by said client in testing said computing product;

executing said portion of said test programs using said computing product; and

while executing said suite of test programs, avoiding performance of said excluded test programs.

28. The computer software product according to claim 24, wherein said computer is further instructed to perform the step of accepting a selection of one of said qualified test programs via a master exclude list as an excluded test program.

29. A verification system for managing the composition and execution of test suites in testing a computing product, comprising:

a client terminal receiving a table comprising qualified programs of a suite of test programs from a server, said qualified programs being qualified for exclusion from performance, wherein said suite is organized as a hierarchy of said test programs, and said table comprises a pointer to a node of said hierarchy, and wherein all of said test programs that are located in a subtree headed by said node are included in said qualified programs; and

a graphical user interface in said client terminal that hierarchically displays said qualified programs for selection thereof, said client terminal being adapted, responsively to a selection via said graphical user interface of an excluded test program, to execute said suite of test programs using said computing product; and

to avoid performance of said excluded test program.

30. A verification system for managing the composition and execution of test suites in testing a computing product, comprising:

a server;

a memory in said server that stores a central repository holding a suite of test programs, an execution test harness, an installer, and a graphical user interface for modifying said suite;

a terminal displaying said suite as a hierarchy on said graphical user interface, qualified test programs from said suite being selectable via said graphical user interface;

a communications facility for transmitting a record of said qualified test programs; and

a client terminal linked to said server for hierarchically displaying only said qualified test programs responsively to receiving said record from said server, said qualified test programs being selectable as excluded test programs.

31. The verification system according to claim 30, wherein said record of said qualified test programs comprises a pointer to a node of said hierarchy, wherein all of said test programs in a subtree headed by said node are included in said qualified test programs.

32. The verification system according to claim 30, wherein said server is adapted to download said installer to said client terminal;

and wherein responsively to an execution of said installer said server is adapted to download and install at least a portion of said test programs from said central repository to said client terminal for execution using said computing product while avoiding execution of any of said excluded test programs.