WHITE-BOX TESTING SYSTEMS AND/OR METHODS FOR USE IN CONNECTION WITH GRAPHICAL USER INTERFACES

• US 20140101640A1
• Filed: 10/05/2012
• Published: 04/10/2014
• Est. Priority Date: 10/05/2012
• Status: Active Grant

• Current Assignee: Software AG
• Original Assignee: Software AG
• Inventors: ROMDHANE, Taoufik
• Granted Patent: US 9,047,413 B2
• Time in Patent Office: Days
• US Class Current: 717/125
• CPC Class Codes: G06F 11/3664 Environments for testing or debugging software
• CPC Class Codes: G06F 11/3684 for test design, e.g. generating new test cases
• CPC Class Codes: G06F 11/3688 for test execution, e.g. scheduling of test suites

Abstract

Certain example embodiments relate to white-box testing techniques, e.g., for graphical user interfaces (GUIs). A test case to be run on the application includes interactions between the application and external components that were gathered as the application ran and received user input from a user until a desired test state was reached, and assertions to be made to the application once that state is reached. A test environment replaces a normal environment in which the application runs. An adapter executes interactions on the test environment'"'"'s GUI dedicated thread(s). The application is fed with interactions, through the adapter, until the application reaches the desired state. Once the application reaches that state, the application is fed with assertions. Expected data associated with the assertions is compared with data actually being generated by the application. A determination is made regarding whether the test case resulted in success or failure.

Claims

1. A method of testing an application in connection with a predefined test case, the method comprising:

retrieving the test case to be run on the application, the test case including (a) interactions between the application and components external to the test case'"'"'s scope that were gathered as the application ran and received user input from a user until a desired test state was reached, and (b) assertions to be made to the application once the application reaches the desired test state;

setting up a test environment in connection with a test runner, the test environment replacing a normal environment in which the application runs, the test environment including an adapter configured to execute interactions on the test environment'"'"'s GUI thread;

feeding the application with interactions included in the test case, through the adapter, until the application reaches the desired test state;

once the application reaches the desired state, feeding the application with assertions included in the test case through the adapter;

comparing expected data associated with the assertions with data actually being generated by the application; and

determining, from the comparison, whether the test case resulted in a success or failure.

2. The method of claim 1, further comprising:

sending the comparison to the test runner, the success/failure determination being made by the test runner and reported to the user through a unit tester module connected to the test runner.

3. The method of claim 1, wherein the test case further comprises user-defined test logic to be performed on the application once it reaches the desired test state, and further comprising:

once the application reaches the desired test state, evaluating the test logic in the application under test;

further comparing expected results from the test logic with results actually being generated by the application; and

including this further comparison in the determination as to whether the test case resulted in a success or failure.

4. The method of claim 1, further comprising, in the setting up of the test environment, initializing mock objects that simulate behaviors of communication layer entities that operate when the application is running in the normal environment.

5. The method of claim 4, wherein during said feeding processes, the adapter is in communication with the mock objects rather than components external to the test case'"'"'s scope.

6. The method of claim 1, wherein each said interaction includes a message identifier, a message name, a timestamp associated with the interaction, and one or more data fields.

7. The method of claim 6, wherein the interactions are chronologically orderable by virtue of the identifiers associated therewith.

8. The method of claim 1, further comprising attaching a runtime framework to the application, the framework having access to data and classes of the application while the application is running.

9. The method of claim 8, further comprising executing, within the application, test code developed and maintained outside of the application in connection with the framework.

10. The method of claim 8, further comprising:

executing a plurality of test cases on the application; and

deriving test coverage metrics for the test cases, in connection with the framework.

11. A non-transitory computer readable storage medium tangibly storing a program that, when executed by at least one processor of a computer system, implements a method according to claim 1.

12. A method of generating a test case for an application to be tested, the method comprising:

starting the application to be tested in an environment in which the application is normally run;

storing, in connection with a recorder attached to the application, interactions between the application and components external to the test case'"'"'s scope as the application runs and receives user input from a user;

when a desired test state is reached;

determining which components of the environment are currently active;

determining, from a store including a plurality of different assertion sets, which of those assertions sets apply to the currently active components;

feeding corresponding component state data to each said applicable assertion set to generate a corresponding assertion that holds current component state data; and

generating the test case, the test case including the generated assertions and the stored interactions.

13. The method of claim 12, further comprising:

enabling the user to build custom test logic in connection with user input test logic and/or test logic from a custom test logic store and/or a builder module; and

adding the custom test logic to the test case.

14. The method of claim 12, wherein the recorder operates in a distinct layer between the application and the components external to the test case'"'"'s scope.

15. The method of claim 14, wherein the distinct layer in which the recorder operates is between a communication layer and a layer including the application'"'"'s classes.

16. A computer system for testing a computer application in connection with a predefined test case, comprising:

processing resources including at least one processor and a memory;

a non-transitory computer readable storage medium storing a plurality of test cases, each said test case including (a) interactions between the application and components external to the test case'"'"'s scope that were gathered as the application ran and received user input from a user until a desired test state was reached, and (b) assertions to be made to the application once the application reaches the desired test state; and

a test runner configured to cooperate with the processing resources to set up a test environment in connection that replaces a normal environment in which the application runs, the test environment including an adapter configured to execute interactions on the test environment'"'"'s GUI thread;

wherein, in connection with a selected one of said test cases, the adapter and/or test runner is/are configured to;

feed the application with interactions included in the selected test case until the application reaches the desired test state,

once the application reaches the desired state, feed the application with assertions included in the selected test case,

compare expected data associated with the assertions with data actually being generated by the application, and

determine, from the comparison, whether the selected test case resulted in a success or failure.

17. The system of claim 16, wherein:

at least one of the test cases further comprises user-defined test logic to be performed on the application once it reaches the desired test state;

the adapter is further configured to facilitate evaluation of test logic from a selected test case in the application under test once the application reaches the desired test state; and

the test runner is further configured to;

further compare expected results from the test logic with results actually being generated by the application; and

include this further comparison in the determination as to whether the test case resulted in a success or failure.

18. The system of claim 16, wherein the test runner is further configured to initialize mock objects that simulate behaviors of communication layer entities that operate when the application is running in the normal environment such that, during said feeding processes, the adapter is in communication with the mock objects rather than components external to the test case'"'"'s scope.

19. The system of claim 16, wherein each said interaction includes a message identifier, a message name, a timestamp associated with the interaction, and one or more data fields, the interactions being chronologically orderable by virtue of the identifiers associated therewith.

20. The system of claim 16, further comprising a runtime framework attachable to the application, the framework having access to data and classes of the application while the application is running and being configured to execute, within the application, test code developed and maintained outside of the application in connection with the framework.