Automated test system for testing an application running in a windows-based environment and related methods

• US 7,398,469 B2
• Filed: 03/12/2004
• Issued: 07/08/2008
• Est. Priority Date: 03/12/2004
• Status: Active Grant

• Current Assignee: United Parcel Service Of America Incorporated (United Parcel Service Incorporated)
• Original Assignee: United Parcel of America Incorporated
• Inventors: Kisamore, Michael T., Emmel, Donald L.
• Primary Examiner(s): Luu; Sy D.
• Assistant Examiner(s): Takele; Meseker
• Application Number:
  US10/799,904
• Publication Number:
  US 20050204343A1
• Time in Patent Office: 1,579 Days
• Field of Search: 715/700, 715/764, 715/751, 715/753, 715/755, 715/756, 715/757, 715/758, 715/704, 715/763, 345/156, 345/300, 719/328, 719/329, 717/124
• US Class Current: 715/704
• CPC Class Codes: G06F 11/0793 Remedial or corrective actions recovery from an exception in an instruction pipeline G06F9/3861; by retry G06F11/1402; for recovering from a failure of a protocol instance or entity H04L69/40
• CPC Class Codes: G06F 11/3688 for test execution, e.g. scheduling of test suites

Abstract

Methods and systems disclosed herein can be used for testing an application'"'"'s rendering of output to individual controls of a Graphical User Interface (GUI), displayed on a target device executing the application. A system according to the present invention includes a test tool running on a development computer that communicates with an agent executing on the target device. Testing is performed by using the test tool to execute test scripts, which cause test input to be injected via the agent into the application on the target device. The test tool can validate whether actual output on the target device matches expected output known to the test tool. The present invention includes a variety of key components, such as a flexible trap manager for handling unexpected screens that appear during an automated test, and a configuration manger for testing against multiple languages and platform configurations.

Claims

1. A system for testing an application running on a target device, the system comprising:

a target device storing and executing a software test agent, wherein said application under test is also stored and executed on said target device;

a test development computer storing and executing a software test tool for testing and validating said application'"'"'s rendering of output to a Graphical User Interface (GUI), said test tool configured to communicate with said agent on said target device;

said software test tool executing a test script for testing and validating one or more aspects of said application'"'"'s rendering of output to said GUI, said test tool operable to generate requests to said agent to obtain information from and send events to controls associated with said GUI;

said software test tool comprising a configuration manager for handling testing of said application against multiple languages and platform configurations;

said configuration manager comprising a configuration table having a plurality of user-defined configurations, each said configuration comprising a collection of value sets corresponding to respective configuration item groups, each said value set being one of a plurality of possible value sets that are selectable in association with said respective item group, each said value set comprising a collection of one or more related configuration items and corresponding values for said related items; and

said configuration table being stored in a spread sheet format for ease of editing by said user, said spread sheet being configured for allowing said user to simultaneously view in a tabular format said plurality of possible value sets that are selectable in association with each said configuration item group wherein,

in response to a user changing from a first value set to a second value set during execution of said test script, said software test tool is further configured to;

automatically delete all configuration item values associated with said first value set and reload said configuration items with corresponding values associated with said second value set; and

continue executing said test script using said configuration item values associated with said second value set.

2. The system of claim 1, wherein said configuration manager comprises a capability that allows said user to get said value of a particular configuration item, and a capability that allows said user to set said value of a particular configuration item, during execution of said test script.

3. The system of claim 1, wherein said test tool comprises a capability that allows said user to assign an input-event name to a grouping of multiple key sequences, said grouping of multiple key sequences representing an input event that occurs on said target device during execution of said test script, wherein said input-event name can be used in place of said grouping each time said grouping is to be written into said test script.

4. A method of testing an application that is operable to execute in multiple languages and platform configurations, the method comprising:

storing a configuration table in a spread sheet format for ease of editing by a user having a plurality of user-defined configurations, each configuration comprising a collection of value sets corresponding to respective configuration item groups, each said value set being one of a plurality of possible value sets that are selectable in association with said respective item group, each said value set comprising a collection of one or more related configuration items and corresponding values for said related items, and said spread sheet being configured for allowing said user to simultaneously view in a tabular format said plurality of possible value sets that are selectable in association with each said configuration item group;

writing a test script that executes differently based on which user-defined configuration is loaded from said table;

loading a user-defined configuration from said configuration table prior to execution of said test script; and

executing said test script in accordance with said configuration loaded from said configuration table wherein,

in response to changing from a first value set to a second value set during execution of said test script;

automatically deleting all configuration item values associated with said first value set;

automatically reloading said configuration items with corresponding values associated with said second value set; and

continue executing said test script using said configuration item values associated with said second value set.

5. A method performed by a test development computer storing and executing a test tool operable to communicate with a target device storing and executing a test agent to interrogate a particular Graphical User Interface (GUI) control that is associated with an application under test running on said target device, the method comprising:

storing a first table that comprises a list of entries accessible by said test tool, each said entry containing information corresponding to a particular GUI control associated with said application under test, said information associated with said particular GUI control comprising;

a user-defined name for uniquely referring to said control;

a plurality of properties which define said control, including a class name that indicates a class type to which said control belongs; and

a data field for optionally storing a control-specific identify flag to be used by said test tool for indicating to said test agent that a specific set of properties are to be used in identifying said control on said target device; and

storing a second table that comprises a list of entries accessible by said test tool, each said entry containing information related to a particular class of GUI controls associated with said application under test, said information associated with each said particular class of GUI controls comprising;

a class name for uniquely referring to said particular class of controls; and

a default identify flag to be used by said test tool for indicating to said test agent that a default set of properties are to be used for identifying controls of said particular class of controls on said target device that do not have a control-specific identify flag associated with them, as defined in said first table.

6. The method of claim 5, further comprising:

locating in said first table said user defined name of said control to be interrogated;

reading from said first table said properties associated with said control to be interrogated;

checking to see whether said the control-specific identify-flag data field associated with said control is empty;

loading said the identify flag from said control-specific identify flag data field, if said data field is not empty;

if said control-specific identify-flag field is empty, loading sai default identify flag from said second table using said class name associated with said control to be interrogated;

generating a request to said agent to locate a control on said target device that matches said control to be interrogated, said agent locating a matching control by searching for a control which has a set of properties indicated by said identify flag obtained in said steps above that match said corresponding set of properties for said control to be interrogated, said matching control having a handle for accessing said control associated therewith; and

receiving said handle for said matching control from said agent.

7. The method of claim 6, wherein said entry associated with each said control listed in said first table further comprises a data field for optionally storing a control-specific verify flag, said control-specific verify flag used for indicating that a specific set of properties are to be used when verifying that an occurrence of said control on said target device matches an expected state;

and

wherein said entry associated with each said class of controls listed in said second table further comprises a default verify flag, said default verify flag used for indicating that a default set of properties are to be used for verifying controls of said particular class of controls on said target device that do not have a control-specific verify flag associated with them, as defined in said first table.

8. The method of claim 7, further comprising:

checking to see whether said control-specific verify-flag data field associated with said control to be interrogated is empty;

loading said verify flag from said control-specific verify flag data field, if said data field is not empty;

if said control-specific verify-flag field is empty, loading said default verify flag from said second table using said class name associated with said control to be interrogated;

interpreting said verify flag obtained in said steps above to determine which properties are to be used in verifying that said matching control matches said control to be interrogated;

generating a request to said agent to retrieve actual property values associated with said matching control, said values retrieved comprising values for at least those properties indicated in said verify flag obtained in said steps above; and

comparing said actual property values associated with said matching control, for only those properties indicated in said verify flag obtained above, to said corresponding property values defined in said first table for said control to be interrogated.

9. The method of claim 8, further comprising:

upon detecting a mismatch in any of said properties compared, generating an entry in a log file detailing said mismatch.