Automatic batching of GUI-based tasks

• US 8,924,329 B2
• Filed: 01/31/2013
• Issued: 12/30/2014
• Est. Priority Date: 11/17/2010
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Lin, Qingwei, Li, Fan, Li, Jiang
• Primary Examiner(s): Holmes, Michael B
• Application Number:
  US13/756,237
• Publication Number:
  US 20140068618A1
• Time in Patent Office: 698 Days
• Field of Search: None
• US Class Current: 706/45
• CPC Class Codes: G06F 3/0484 for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces
• CPC Class Codes: G06F 9/453 Help systems
• CPC Class Codes: G06F 9/46 Multiprogramming arrangements
• CPC Class Codes: G06F 9/4881 Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
• CPC Class Codes: G06N 5/04 Inference or reasoning models

Abstract

Described herein are techniques for automatically batching GUI-based (Graphical User Interface) tasks. The described techniques include automatically determining whether a user is performing batchable tasks in a GUI-based environment. Once detected, the described techniques include predicting the next tasks of a batch based upon those detected batchable tasks. With the described techniques, the user may be asked to verify and/or correct the predicted next tasks. Furthermore, the described techniques may include performing a batch and doing so without user interaction.

Claims

1. A method comprising:

identifying batchable graphical user interface (GUI)-based tasks performed by a user;

generating a batch of not-yet-performed GUI-based tasks based, at least in part, upon the identified batchable GUI-based tasks; and

performing the generated batch of GUI-based tasks.

2. A method as recited in claim 1, wherein identifying batchable GUI-based tasks performed by a user comprises:

tracking GUI-based tasks performed by the user within a context of a GUI-based operating system environment;

parsing each of the tracked GUI-based tasks into an invariant portion and one or more variant portions, wherein the invariant portion of like tasks does not differ between like tasks and the variant portions of like tasks differ between like tasks;

examining the invariant portion of each task of a sequence of GUI-based tasks; and

detecting, in the sequence, a repeated sub-sequence of tasks having a common pattern of invariant portions.

3. A method as recited in claim 2, wherein generating the batch of not-yet-performed GUI-based tasks comprises:

selecting one or more defined sequences of values of variant portions of GUI-based tasks; and

setting values of variant portions of anticipated not-yet-performed GUI-based tasks based upon the selected one or more defined sequences.

4. A method as recited in claim 1, wherein the batchable GUI-based tasks that are identified occur across multiple GUI-based applications in a GUI-based operating system environment.

5. A system comprising:

a processor;

a memory;

an operating system executed at least in part by the processor, wherein the operating system provides a graphical user interface (GUI)-based environment; and

an auto-batcher module stored in the memory and executed at least in part by the processor to automatically batch GUI-based tasks for execution within the GUI-based environment.

6. A system as recited in claim 5, wherein the auto-batcher module comprises a GUI-based task tracker configured to record GUI-based tasks performed by a user.

7. A system as recited in claim 6, wherein the GUI-based task tracker comprises:

a user-operation capturer configured to record, for a GUI-based task performed by the user, an operation record that identifies an operation type and an operation element; and

a GUI-based task parser configured to;

receive a series of operation records; and

analyze the series of operation records to parse invariant portions of each GUI-based task represented in the series of operation records from variant portions of each GUI-based task represented in the series of operation records.

8. A system as recited in claim 5, wherein the auto-batcher module comprises a batchable task determiner configured to:

detect batchable GUI-based tasks; and

anticipate GUI-based tasks that a user might perform.

9. A system as recited in claim 8, wherein the batchable task determiner comprises:

a task pattern detector configured to detect a repeated pattern of sub-sequences within a sequence of recorded GUI-based tasks; and

a parameter projector configured to predict task parameters in an anticipated batch of GUI-based tasks.

10. A system as recited in claim 5, wherein the auto-batcher module comprises a batchable task confirmer configured to confirm with a user a predicted batch of GUI-based tasks.

11. A system as recited in claim 10, wherein confirming with the user the predicted batch of GUI-based tasks includes correcting the predicted batch of GUI-based tasks based on input from the user.

12. A system as recited in claim 10, wherein the batchable task confirmer comprises:

a projection verifier configured to;

notify the user that a predicted batch of GUI-based tasks is available for verification; and

visually simulate the predicted batch of GUI-based tasks; and

a projection corrector configured to receive user input to correct any incorrectly anticipated GUI-based tasks in the predicted batch.

13. A system as recited in claim 5, wherein the auto-batcher module comprises a batch performer to perform a batch of GUI-based tasks.

14. A system as recited in claim 13, wherein the batch performer comprises:

a parameter range determiner configured to determine a range of parameters for a particular GUI-based task within the batch of GUI-based tasks;

an operation target locator configured to locate a target element of a particular GUI-based task within the batch of GUI-based tasks;

a batch generator configured to generate a predicted operation sequence for executing the batch of GUI-based tasks; and

a user-input emulator configured to mimic the appearance of a user controlling an input while executing the batch of GUI-based tasks.

15. A system as recited in claim 14, wherein the parameter range determiner is configured to determine the range of parameters based on input from a user.

16. A system as recited in claim 14, wherein the parameter range determiner is configured to determine the range of parameters by detecting a range based on parameters of previously recorded GUI-based tasks.

17. A system as recited in claim 14, wherein the operation target locator is configured to locate the target element of a particular GUI-based task within a relative context.

18. A method comprising:

recording graphical user interface (GUI)-based tasks performed by a user within a context of a GUI-based environment;

identifying as batchable, at least a subset of the GUI-based tasks that are recorded;

generating a sequence of GUI-based tasks for future execution based, at least in part, on the identified batchable GUI-based tasks;

executing the sequence of GUI-based tasks while visually simulating a user providing input in performing the sequence of GUI-based tasks.

19. A method as recited in claim 18, further comprising verifying with the user that the generated sequence of GUI-based tasks for future execution represents additional tasks that the user intends to perform.

20. A method as recited in claim 18, further comprising determining a parameter value for a particular GUI-based task within the sequence of the GUI-based tasks.