Program test system

• US 7,958,495 B2
• Filed: 03/08/2007
• Issued: 06/07/2011
• Est. Priority Date: 03/08/2007
• Status: Expired due to Fees

• Current Assignee: Systemware, Inc.
• Original Assignee: Systemware, Inc.
• Inventors: Kelso, Tim
• Primary Examiner(s): Zhen; Wei Y
• Assistant Examiner(s): Wu; Junchun
• Application Number:
  US11/683,908
• Publication Number:
  US 20080222454A1
• Time in Patent Office: 1,552 Days
• Field of Search: 717124-126
• US Class Current: 717/124
• CPC Class Codes: G06F 11/3684 for test design, e.g. generating new test cases
• CPC Class Codes: G06F 11/3688 for test execution, e.g. scheduling of test suites

Abstract

An improved automated software testing system provides the ability to generate and reuse test cases over multiple platforms. Keywords and natural language are used in test case creation, simplifying the process for non-technical business users. Business users can write test cases without scripts. Test cases can be generated even before the application to be tested is available. Data substitution provides ability for test cases to adapt to changing data. Abstraction allows use of all third-party and custom software test tools to be incorporated. Persistent data handling allows capture of data generated during test execution for later use. Testing can be performed entirely automatically or can incorporate some manual interaction. Test results, screen captures of the system tested, along with environment and machine variables are saved in results logs for later review.

Claims

1. An improved system for testing complex software driven applications, the system comprising:

at least one computer processing device providing a user interface device for creating and maintaining a plurality of test cases and associated test data for the application, wherein the test cases and test data are reusable, the interface allowing a user to perform the method steps comprising;

creating an object map which represents objects of the application to be tested;

creating a test case hierarchy, the hierarchy including at least one project data structure to establish a top-level project node, at least one phase data structure associated with a project data structure, wherein the phase data structure is a child node of the project data structure, at least one suite data structure associated with a phase data structure;

creating a plurality of test cases with associated test data, wherein a test case includes at least one group data structure associated with a suite data structure, wherein the group data structure is a child node of the suite data structure, at least one navigation data structure associated with a group data structure, wherein the navigation data structure is a child node of the group data structure, at least one task data structure associated with a navigation data structure, wherein the task data structure is a child node of the navigation data structure;

initiating test case execution;

a scripting device operable on the computer processing device for abstracting the complexity associated with third-party software testing tools, wherein the scripting device translates natural language calls into software test tool specific calls for utilizing the software testing tools; and

a storage device for maintaining system settings, system data, test cases, or test data.

2. The system of claim 1 wherein the interface device presents the user with a view allowing manipulation of the hierarchical structure of each test case.

3. The system of claim 1 wherein the tasks are organized into reusable groups of windows scenarios.

4. The system of claim 1 wherein the plurality of test cases can be created before the application is available to test.

5. The system of claim 1 further comprising a report generating device, wherein the report generating device captures screenshots from the application while a test case is running and stores them in the storage device, and, wherein the user interface device can display the screenshots.

6. The system of claim 1 further comprising a report generating device, wherein the report generating device captures test coverage data representing the extent to which each of the objects of the application have been tested and stores this data in the storage device.

7. The system of claim 1 wherein the scripting device can run on a computer different from that which the user interface device is running, and, wherein the scripting device displays the current test case status as the test case is executing.

8. The system of claim 1 further comprising a plurality of object maps for use in test case creation, wherein an object map abstracts the complex physical name of an object to be tested in the application to a logical name representing the object, and, wherein the logical name may either be terse or natural language.

9. The system of claim 1 further comprising a plurality of object maps for use in test case creation, wherein an object map abstracts the complex physical name of an object to be tested in the application to a logical name representing the object, and, wherein the logical name may either be terse or natural language, and, wherein the object map can be created, modified, and displayed within either the interface device or an external spreadsheet program.

10. The system of claim 1 further comprising a plurality of object maps for use in test case creation, wherein an object map abstracts the complex physical name of an object to be tested in the application to a logical name representing the object, and, wherein the logical name may either be terse or natural language, and, wherein the interface device selectively filters the object maps available for test case creation.

11. The system of claim 1 further comprising a plurality of object maps for use in test case creation, wherein an object map abstracts the complex physical name of an object to be tested in the application to a logical name representing the object, and, wherein the logical name may either be terse or natural language, and, wherein the physical name of an object can be overridden during execution of a test case using the interface device.

12. The system of claim 1 further comprising a plurality of object maps for use in test case creation, wherein an object map abstracts the complex physical name of an object to be tested in the application to a logical name representing the object, and, wherein the logical name may either be terse or natural language, and, wherein natural language keywords represent an action to be taken on an object in the object maps.

13. The system of claim 1 further comprising a plurality of object maps for use in test case creation, wherein an object map abstracts the complex physical name of an object to be tested in the application to a logical name representing the object, and, wherein the logical name may either be terse or natural language, and, wherein natural language keywords represent an action to be taken on an object in the object maps, and, wherein at least one keyword allows for manual action to be taken during execution of a test case.

14. The system of claim 1 wherein a portion of the test data can be replaced with substitute data whose type is chosen from the group consisting of constant, defined, persistent, prefetch, and runtime.

15. The system of claim 1 wherein a portion of the test data can be substituted with persistent data, and, wherein the persistent data is dynamically generated and captured during test case execution and is stored in the storage device for later use in the same or subsequent test cases.

16. The system of claim 1 wherein a portion of the test data can be substituted with prefetch data, and, wherein the prefetch data is original application data that is obtained before test execution and subsequently retained in the storage device for later access during test case execution, and, wherein the retained prefetch data is accessed during test case execution instead of accessing the original application data.

17. An improved method for testing complex software driven applications, the method comprising the following steps:

determining the system requirements of the application;

creating an object map which represents objects of the application to be tested;

creating a test case hierarchy, the hierarchy including at least one project data structure to establish a top-level project node, at least one phase data structure associated with a project data structure, wherein the phase data structure is a child node of the project data structure, at least one suite data structure associated with a phase data structure;

creating a plurality of test cases with associated test data, wherein a test case includes at least one group data structure associated with a suite data structure, wherein the group data structure is a child node of the suite data structure, at least one navigation data structure associated with a group data structure, wherein the navigation data structure is a child node of the group data structure, at least one task data structure associated with a navigation data structure, wherein the task data structure is a child node of the navigation data structure;

initiating test case execution;

generating results based on test case execution;

verifying the results generated during test case execution;

correcting the application errors; and

repeating each step as needed until there are no further errors.

18. The method of claim 17 wherein the steps for test cases creation comprise:

selecting a window in which an object resides;

selecting an action keyword representing the action to be taken in the selected window; and

selecting an object that corresponds to the selected action, wherein the objects available for selection are dependent upon the action selected, and, wherein the actions available for selection are dependent upon the selected window.

19. The method of claim 17 wherein the test cases are created in natural language and are reusable for subsequent application tests.

20. The method of claim 17 wherein the object map is created and maintained using a spreadsheet program.

21. The method of claim 17 further comprising modifying the test data with substitute data whose type is chosen from the group consisting of constant, defined, persistent, prefetch, and runtime.

22. The method of claim 17 further comprising verifying manual test case steps.

23. The method of claim 17 wherein generating results comprises capturing screenshots of the application under test;

and monitoring which objects are tested; and

, wherein verifying the results comprises verifying that each object was thoroughly tested.

24. The method of claim 17 wherein the object map physical name can be overridden during test case execution without manually generating additional scripts, and, wherein the overridden physical name takes on the attributes of the data requested.

25. The method of claim 17 further comprising capturing test case generated data;

storing the test case generated data; and

wherein the test case generated data is used for subsequent test cases.

26. The method of claim 17 wherein each test case of the plurality of test cases is written in natural language.

27. The method of claim 17 wherein the test case execution includes the use of a software test-tool and, wherein each test case of the plurality of test cases is developed independently and without regard to the software test-tool.

28. A computer software program tangibly embodied in a non-transitory computer readable medium, the program including machine-readable instructions executable by a computer processor to perform a method for testing complex software driven applications, the program steps comprising:

creating an object map which represents objects of the application to be tested;

creating a test case hierarchy, the hierarchy including at least one project data structure to establish a top-level project node, at least one phase data structure associated with a project data structure, wherein the phase data structure is a child node of the project data structure, at least one suite data structure associated with a phase data structure;

creating a plurality of test cases with associated test data, wherein a test case includes at least one group data structure associated with a suite data structure, wherein the group data structure is a child node of the suite data structure, at least one navigation data structure associated with a group data structure, wherein the navigation data structure is a child node of the group data structure, at least one task data structure associated with a navigation data structure, wherein the task data structure is a child node of the navigation data structure;

initiating test case execution;

generating results based on test case execution;

verifying the results generated during test case execution;

correcting the application errors; and

repeating each step as needed until there are no further errors.