Capture and Playback for GUI-Based Tasks

• US 20120131456A1
• Filed: 11/22/2010
• Published: 05/24/2012
• Est. Priority Date: 11/22/2010
• Status: Abandoned Application

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Lin, Qingwei, Li, Fan, Li, Jiang
• Application Number:
  US12/952,010
• Publication Number:
  US 20120131456A1
• Time in Patent Office: Days
• US Class Current: 715/704
• CPC Class Codes: G06F 11/3672 Test management
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces

Abstract

Described herein are techniques for capture and playback of user-performed GUI-based (Graphical User Interface) tasks across multiple GUI-based applications. The described techniques include performing the playback of such tasks without depending upon the playback environmental conditions matching the original capture conditions.

Claims

1. A method that facilitates capture and playback of Graphical User Interface (GUI) based tasks in a GUI-based environment, the method comprising:

capturing a series of GUI-based tasks input by a user within a context of the GUI-based environment;

setting conditions which will trigger a replay of the captured series of GUI-based tasks;

determining whether triggering conditions have occurred;

in response to the determining;

triggering the replay of the captured series of GUI-based tasks;

locating GUI-based targets of the captured series of GUI-based tasks;

based upon the located targets, regenerating user input for the replay of the captured series of GUI-based tasks;

based upon the regenerated user input, replaying the captured series of GUI-based tasks, wherein the replaying includes visually simulating a user providing user input in performing the captured series of GUI-based tasks.

2. A method as recited in claim 1 further comprising storing the captured series of GUI-based tasks using a defined format for storing GUI-based tasks.

3. A method as recited in claim 1, wherein the captured series of GUI-based tasks includes tasks performed within a context of multiple GUI-based applications.

4. A method as recited in claim 1, wherein the capturing includes collecting information about user input from multiple GUI-based applications.

5. A method as recited in claim 1, wherein the triggering, locating, regenerating, and replaying occur in response to a determination that triggering conditions have occurred.

6. A method as recited in claim 1, wherein the triggering, locating, regenerating, and replaying occur in response to the user manually initiating the performance of the triggering, locating, regenerating, and replaying.

7. A method as recited in claim 1, wherein the triggering conditions include time-based or event-based conditions.

8. A method as recited in claim 1, wherein the locating comprises, for a particular task of the captured series of GUI-based tasks, restoring environmental conditions of the particular task during the replaying of the particular task to the environmental conditions that existed when the particular task was captured.

9. A method as recited in claim 8, wherein the environmental conditions are selected from a group consisting of a process name, a main window, a frame window, an element path, an element, an absolute position, a frame window rectangle, a status and value, and a time stamp.

10. One or more computer-readable media storing processor-executable instructions that, when executed, cause one or more processors to perform operations that facilitate capture and playback of Graphical User Interface (GUI) based tasks in a GUI-based environment, the operations comprising:

capturing a series of GUI-based tasks received from a user within a context of the GUI-based environment, wherein a set of environmental conditions exists that describes the context of the GUI-based environment during the capturing;

replaying the captured series of GUI-based tasks, wherein another set of environmental conditions exists that describes a context of the GUI-based environment at the initiation of the replaying and the environmental conditions of the sets are unmatching.

11. One or more computer-readable media as recited in claim 10, wherein the capturing includes collecting information about user input from multiple GUI-based applications.

12. One or more computer-readable media as recited in claim 10, wherein the operations further comprise:

setting conditions which will trigger a replay of the captured series of GUI-based tasks;

determining whether triggering conditions have occurred;

in response to the determining, triggering the replaying.

13. One or more computer-readable media as recited in claim 10, wherein the replaying includes visually simulating a user providing user input in performing the captured series of GUI-based tasks.

14. One or more computer-readable media as recited in claim 10, wherein the environmental conditions are selected from a group consisting of a process name, a main window, a frame window, an element path, an element, an absolute position, a frame window rectangle, a status and value, and a time stamp.

15. One or more computer-readable media as recited in claim 10, wherein the replaying operation includes:

locating GUI-based targets of the captured series of GUI-based tasks;

based upon the located targets, regenerating user input for the replay of the captured series of GUI-based tasks;

based upon the regenerated user input, replaying the captured series of GUI-based tasks.

16. One or more computer-readable media as recited in claim 10, wherein the environmental conditions that describes a context of the GUI-based environment are selected from a list consisting of window position, window size, target element.

17. One or more computer-readable media storing processor-executable instructions that, when executed, cause one or more processors to perform operations that facilitate capture and playback of Graphical User Interface (GUI) based tasks in a GUI-based environment, the operations comprising:

triggering a playback of a captured series of GUI-based tasks input by a user within a context of the GUI-based environment;

in response to the triggering, playing back the captured series of GUI-based tasks, wherein the playing back includes a replay of tasks captured through multiple GUI-based applications.

18. One or more computer-readable media as recited in claim 17, wherein the playing back operation includes visually simulating a user providing user input in performing the captured series of GUI-based tasks.

19. One or more computer-readable media as recited in claim 17, wherein the playing back operation comprises:

locating GUI-based targets of the captured series of GUI-based tasks;

based upon the located targets, regenerating user input for the replay of the captured series of GUI-based tasks.

20. One or more computer-readable media as recited in claim 17, wherein the playing-back operation comprises:

obtaining the captured series of GUI-based tasks from a memory;

formatting each of the tasks of the obtained series of GUI-based tasks to specify a GUI-based operation target of each task;

locating GUI-based operation targets of each of the tasks of the captured series of GUI-based tasks;

based upon the located GUI-based operation targets, regenerating user input for the replay of the captured series of GUI-based tasks;

simulating a user providing user input in the playing back of each task of the captured series of GUI-based tasks.