Automating interactions with software user interfaces

• US 9,740,506 B2
• Filed: 04/30/2010
• Issued: 08/22/2017
• Est. Priority Date: 11/21/2002
• Status: Expired due to Term

• Current Assignee: Siemens Industry Software Inc. (Siemens AG)
• Original Assignee: Mentor Graphics Corporation (Siemens AG)
• Inventors: Ren, Bing
• Primary Examiner(s): DAO, THUY CHAN
• Application Number:
  US12/772,013
• Publication Number:
  US 20100281457A1
• Time in Patent Office: 2,671 Days
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces

Abstract

A method and apparatus applies an action to a software application by determining a target object for the input action. The determination of the target object is performed by identifying the target object through socially identifying object information relative to a reference object. Then, the input action is applied to the target object.

Claims

1. A computer-implemented method comprising:

determining a target object in a graphical user interface (GUI) for a software application input action based upon socially identifying information of the target object, the socially identifying information describing the target objects relationship to one or more peer objects of a same type as the target object and/or descendent objects in the GUI and comprising;

a type of the target object'"'"'s parent,

a rank of the target object relative to the one or more peer objects of the same type as the target object, wherein the rank indicates a location of the target object in the GUI relative to the one or more peer objects of the same type as the target object, and

a location of a label of the target object, wherein the label is separate from the target object and the location of the label describes a position of the label in the GUI relative to the target object; and

applying the input action to the target object.

2. The method of claim 1, wherein the target object is further determined using name information.

3. The method of claim 2, wherein the name information comprises an object name.

4. The method of claim 1, wherein the socially identifying information further comprises information about a parent object to the target object.

5. The method of claim 1, wherein the socially identifying information includes an identifier of which of a number of unnamed objects the target object is.

6. The method of claim 1, wherein the input action is provided by a script file, the method further comprising generating the script file by recording one or more user actions with respect to the GUI.

7. The method of claim 1, wherein the socially identifying information does not identify a name of the target object.

8. The method of claim 1, wherein the socially identifying information further comprises a spatial location, the spatial location comprising a numerical value indicating that the one or more objects are spatially located to the left or right of the target object.

9. The method of claim 1, wherein the socially identifying information further comprises a spatial location, the spatial location comprising a numerical value indicating that the one or more objects are spatially located above or below the target object.

10. One or more tangible computer-readable storage media storing computer-readable instructions that when executed by a computer, cause the computer to perform a method, the method comprising:

determining a target object in a graphical user interface (GUI) for an action, the determining being based upon data defined in a script file and describing the target object'"'"'s relationship to one or more peer objects in the GUI and comprising;

a type of the target object'"'"'s parent,

a rank of the target object relative to the one or more peer objects of a same type as the target object, wherein the rank is determined by signature information defined in the script file and is based on a relative location of the one or more peer objects with respect to the target object in the GUI, and

a location of a label of the target object, wherein the label is separate from the target object and the location of the label describes a position of the label in the GUI relative to the target object; and

applying the action to the target object, the action being defined in the script file.

11. The one or more tangible computer-readable storage media of claim 10, wherein the target object is further determined using name information.

12. The one or more tangible computer-readable storage media of claim 11, wherein the name information comprises an object name.

13. The one or more tangible computer-readable storage media of claim 11, wherein the name information comprises an identifier of a parent object to the target object.

14. The one or more tangible computer-readable storage media of claim 10, wherein the data further comprises information about a parent object to the target object.

15. The one or more tangible computer-readable storage media of claim 10, wherein the data further comprises an identifier of which of a number of unnamed objects the target object is.

16. The one or more tangible computer-readable storage media of claim 10, wherein determining a target object is performed by interacting with a java virtual machine.

17. The one or more tangible computer-readable storage media of claim 10 wherein the target object is a java component.

18. The one or more tangible computer-readable storage media of claim 17, wherein the java component comprises a top level container name and a component name.

19. An apparatus comprising:

a processor; and

a computer-readable medium having stored thereon a plurality of instructions which, when executed, cause the processor to;

determine a target object in a graphical user interface (GUI) for the input action based upon socially identifying information of the target object, the socially identifying information describing hierarchy information about the target object'"'"'s relationship to one or more peer objects in the GUI, wherein the one or more peer objects have a same type as the target object, and a rank of the target object relative to the one or more peer objects having the same type as the target object, wherein the rank is based on a relative location of the one or more peer objects with respect to the target object in the GUI, wherein the socially identifying information further comprises label signature information, the label signature information describing a location of a label of the target object, wherein the label is separate from the target object and the location of the label describes a position of the label in the GUI relative to the target object; and

apply the input action to the target object, the input action being defined in a script file including the hierarchy information.

20. The apparatus of claim 19, wherein the target object is further determined using name information.

21. The apparatus of claim 20, wherein the name information comprises one or more of an object name or an identifier of a parent object to the target object.

22. The apparatus of claim 19, wherein the socially identifying information comprises information about a parent object to the target object.

23. The apparatus of claim 19, wherein the socially identifying information further comprises an identifier of which of a number of unnamed objects the target object is.

24. One or more tangible computer-readable storage media storing computer-readable instructions that when executed by a computer, cause the computer to perform a method, the method comprising:

determining a target object in a graphical user interface (GUI) for an action, the determining being based upon socially identifying information of the target object, the socially identifying information being defined in a script file and describing hierarchy information about the target object'"'"'s relationship to one or more peer and/or descendent objects in the GUI and comprising a name of the target objects parent and rank of the target object relative to the one or more peer objects and/or descendent objects in the GUI, wherein the rank is based on a relative location of a label component to the target object, wherein the label component is one of the peer objects and/or descendent objects, wherein the relative location is determined using signature information defined in the script file, wherein the signature information is in an Extensible Markup Language (XML) format; and

applying the action to the target object, the action being defined in the script file including the hierarchy information.

25. The computer-readable storage media of claim 24, wherein the signature information comprises the name of the target objects parent.

26. The computer-readable storage media of claim 24, wherein the script file includes test domain information specifying the target object to which the action is to be applied.

27. The computer-readable storage media of claim 24, wherein the script file includes hierarchical test domain information specifying the target object to which the action is to be applied, and wherein a hierarchy for the target object includes at least one hierarchy level not defined in the hierarchical test domain information.

28. A computer-implemented method comprising:

determining a target object in a graphical user interface (GUI) for a software application input action based upon socially identifying information of the target object, the socially identifying information describing the target objects relationship to one or more peer objects in the GUI and comprising a name of the target object'"'"'s parent and a rank of the target object relative to the one or more peer objects of a same type as the target object, wherein the rank is a number indicating a spatial arrangement of the target object with respect to the one or more peer objects in the GUI, and a location of a label of the target object, wherein the label is separate from the target object and the location of the label describes a position of the label in the GUI relative to the target object; and

applying the input action to the target object, wherein the input action is provided by a script file.

29. The method of claim 28, wherein the script file is in a database format.