MULTI-THREADED CONDITIONAL PROCESSING OF USER INTERACTIONS

• US 20120284663A1
• Filed: 05/03/2011
• Published: 11/08/2012
• Est. Priority Date: 05/03/2011
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Driver, David Owen, Lobo, Philomena Francisca, Liu, Xun
• Granted Patent: US 9,069,459 B2
• Time in Patent Office: Days
• US Class Current: 715/784
• CPC Class Codes: G06F 2209/501 Performance criteria
• CPC Class Codes: G06F 2209/5018 Thread allocation
• CPC Class Codes: G06F 3/0485 Scrolling or panning
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces
• CPC Class Codes: G06F 9/505 considering the load

Abstract

Embodiments isolate gesture processing for improved user interactivity with content. A first thread detects user interaction with a document via a touch sensor associated with a computing device. A latency associated with a second thread is calculated. The calculated latency is compared to a threshold latency. Based on the comparison, the first thread determines whether to transfer processing of the detected user interaction from the first thread to the second thread. The detected user interaction is processed by the first thread or by the second thread based on the determination. In some embodiments, users are provided with smooth interactivity such as panning and scrolling when browsing web content with a browser executing on a mobile computing device.

Claims

1. A system for separating gesture processing from rendering processing, said system comprising:

a memory area associated with a mobile computing device, said memory area storing at least a portion of a web page and one or more of a plurality of scrollable elements associated therewith, said memory area further storing a threshold latency;

a processor programmed to;

detect, by a gesture processing thread via a touch sensor associated with the mobile computing device, a request to scroll the web page stored in the memory area;

request, from a rendering thread, touch point data associated with the detected request, said touch point data identifying one of the scrollable elements associated with the detected request;

scroll, by the rendering thread, said one of the scrollable elements if the requested touch point data is received by the gesture processing thread from the rendering thread within the threshold latency stored in the memory area; and

scroll, by the gesture processing thread, the web page if the requested touch point data is not received by the gesture processing thread from the rendering thread within the threshold latency stored in the memory area.

2. The system of claim 1, wherein the processor is further programmed to calculate an elapsed time since requesting the touch point data.

3. The system of claim 1, wherein the processor is further programmed to request the touch point data by requesting one or more of the following:

whether a touch point is scrollable, whether a touch point is editable, and a shape of a touch point.

4. The system of claim 1, wherein the plurality of scrollable elements comprises one or more of the following:

a frame, a subpane, a panel, and a popup.

5. The system of claim 1, further comprising means for separating gesture processing from rendering processing.

6. The system of claim 1, further comprising means for latency-based conditional transfer of gesture processing from one thread to another.

7. A method comprising:

detecting, by a first thread, user interaction with a document via a touch sensor associated with a computing device, said first thread handling gesture processing;

calculating a latency associated with a second thread, said second thread handling rendering of the document;

comparing the calculated latency to a threshold latency;

determining, based on the comparison, whether to transfer processing of the detected user interaction from the first thread to the second thread; and

processing the detected user interaction by the first thread or by the second thread based on said determining.

8. The method of claim 7, wherein calculating the latency comprises:

requesting, from the second thread, touch point data associated with the detected user interaction;

calculating an elapsed time since said requesting; and

determining the latency based at least on the calculated elapsed time.

9. The method of claim 8, wherein processing the detected user interaction comprises processing, by the first thread, the detected user interaction if the second thread fails to provide the requested touch point data.

10. The method of claim 8, further comprising:

receiving, by the first thread, the requested touch point data from the second thread; and

determining an operation associated with the received touch point data.

11. The method of claim 10, wherein determining the operation comprises identifying a gesture associated with the received touch point data.

12. The method of claim 7, wherein processing the detected user interaction comprises processing, by the first thread, the detected user interaction if the calculated latency is greater than the threshold latency.

13. The method of claim 7, wherein processing the detected user interaction comprises performing, by the second thread, the operation if the calculated latency is less than the threshold latency.

14. The method of claim 7, wherein the detected user interaction corresponds to at least one operation, and wherein processing the detected user interaction by the first thread comprises applying, by the first thread, the operation to a version of the document cached by the computing device.

15. The method of claim 7, wherein the detected user interaction corresponds to at least one operation, and wherein processing the detected user interaction by the second thread comprises applying, by the second thread, the operation to a version of the document stored by another computing device.

16. The method of claim 7, wherein detecting the user interaction comprises detecting user interaction with a web page.

17. One or more computer readable storage media having computer-executable components, said components comprising:

a user interface component that when executed causes at least one processor to detect, by a fast response thread, user interaction with a document via a touch sensor associated with a computing device, said fast response thread handling gesture processing;

a touch point component that when executed causes at least one processor to request, from a slow response thread by the fast response thread, touch point data corresponding to the user interaction detected by the user interface component, said slow response thread handling display of the document, said requested touch point data being associated with an operation to be applied to at least a portion of the document;

a measurement component that when executed causes at least one processor to compare, by the fast response thread, a latency associated with the slow response thread to a threshold latency; and

an overlay component that when executed causes at least one processor to apply, based on the comparison performed by the measurement component, the operation by the fast response thread to the entire document or by the slow response thread to only said portion of the document.

18. The computer readable storage media of claim 17, wherein the user interaction corresponds to a gesture and the operation includes a scrolling operation.

19. The computer readable storage media of claim 17, wherein the overlay component applies the operation by the fast response thread to the entire document if the latency violates the threshold latency.

20. The computer readable storage media of claim 17, wherein the overlay component applies the operation by the slow response thread to only said portion of the document if the latency satisfies the threshold latency.