RECOGNIZING GESTURES AND UPDATING DISPLAY BY COORDINATOR

• US 20170147176A1
• Filed: 11/23/2015
• Published: 05/25/2017
• Est. Priority Date: 11/23/2015
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Fischer, Ian, Garde, Chinmay Sunil
• Granted Patent: US 10,761,714 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 3/04817 using icons graphical or visual programming using iconic symbols G06F8/34
• CPC Class Codes: G06F 3/0484 for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces

Abstract

A non-transitory computer-readable storage medium may comprise instructions stored thereon. When executed by at least one processor, the instructions may be configured to cause a computing device to implement at least a user interface module and a coordinator module. The user interface module may be configured to pass layout constraints to the coordinator module. The coordinator module may be configured to process input received by the computing device, recognize gestures based on the input, update a display based on the recognized gestures and the layout constraints passed to the coordinator module by the user interface module, and in response to recognizing an application-specific gesture, pass the application-specific gesture to the user interface module.

Claims

1. A non-transitory computer-readable storage medium comprising instructions stored thereon that, when executed by at least one processor, are configured to cause a computing device to implement at least:

a user interface module configured to pass layout constraints to a coordinator module; and

the coordinator module configured to;

process input received by the computing device;

recognize gestures based on the input;

update a display based on the recognized gestures and the layout constraints passed to the coordinator module by the user interface module; and

in response to recognizing an application-specific gesture, pass the application-specific gesture to the user interface module.

2. The non-transitory computer-readable storage medium of claim 1, wherein the layout constraints define at least a location of a first user interface element on the display with respect to a location of a second user interface element on the display.

3. The non-transitory computer-readable storage medium of claim 1, wherein the layout constraints define constraints between user interface elements on the display without describing a control flow.

4. The non-transitory computer-readable storage medium of claim 1, wherein the user interface module is further configured to construct the layout constraints based on at least one hardware specification of the display.

5. The non-transitory computer-readable storage medium of claim 1, wherein the user interface module is further configured to construct the layout constraints based on a pixel width of the display and a pixel length of the display.

6. The non-transitory computer-readable storage medium of claim 1, wherein the user interface module is further configured to pass a definition of the application-specific gesture to the coordinator module.

7. The non-transitory computer-readable storage medium of claim 1, wherein the input is received via a touchscreen included in the computing device.

8. The non-transitory computer-readable storage medium of claim 1, wherein the updating the display includes determining a location on the display of at least one user interface element.

9. The non-transitory computer-readable storage medium of claim 1, wherein the updating the display includes solving layout and gesture constraints.

10. The non-transitory computer-readable storage medium of claim 1, wherein the updating the display includes solving layout and gesture constraints based on a Cassowary linear arithmetic constraint solving algorithm.

11. The non-transitory computer-readable storage medium of claim 1, wherein the application-specific gesture includes a gesture that prompts a response from an application running on the computing device.

12. The non-transitory computer-readable storage medium of claim 1, wherein the application-specific gesture includes a click on a button associated with an application running on the computing device.

13. The non-transitory computer-readable storage medium of claim 1, wherein the user interface module is further configured to prompt an application to perform an action in response to receiving the application-specific gesture from the coordinator module.

14. The non-transitory computer-readable storage medium of claim 1, wherein:

the user interface module is implemented on a first thread; and

the coordinator module is implemented on a second thread.

15. A non-transitory computer-readable storage medium comprising instructions stored thereon that, when executed by at least one processor, are configured to cause a computing device to launch at least:

a user interface thread configured to pass layout constraints to a coordinator thread; and

the coordinator thread configured to;

process input received by the computing device;

recognize gestures based on the input;

update a display based on the recognized gestures and the layout constraints passed to the coordinator thread by the user interface thread; and

in response to recognizing an application-specific gesture, pass the application-specific gesture to the user interface thread.

16. The non-transitory computer-readable storage medium of claim 15, wherein the user interface thread is further configured to prompt an application process to perform an action in response to receiving the application-specific gesture from the coordinator thread.

17. A computing device comprising:

a touchscreen configured to display user interface elements and receive touch input;

at least one processor; and

at least one memory device comprising instructions stored thereon that, when executed by the at least one processor, are configured to cause the computing device to implement;

a user interface module configured to pass layout constraints to a coordinator module; and

the coordinator module configured to;

process the touch input received by the touchscreen;

recognize gestures based on the touch input;

update user interface elements displayed by the touchscreen based on the recognized gestures and the layout constraints passed to the coordinator module by the user interface module; and

in response to recognizing an application-specific gesture, pass the application-specific gesture to the user interface module.

18. The computing device of claim 17, wherein the layout constraints define at least a location of a first user interface element on the display with respect to a location of a second user interface element on the touchscreen.

19. The computing device of claim 17, wherein the user interface module is further configured to construct the layout constraints based on a pixel width and a pixel length of the touchscreen.

20. The computing device of claim 17, wherein the updating the touchscreen includes determining a location on the touchscreen of at least one user interface element.

21. The computing device of claim 17, wherein the application-specific gesture includes a gesture that prompts a response from an application running on the computing device.

22. The computing device of claim 17, wherein the application-specific gesture includes a click on a button displayed by the touchscreen, the button being associated with an application running on the computing device.

23. The computing device of claim 17, wherein the user interface module is further configured to prompt an application to perform an action in response to receiving the application-specific gesture from the coordinator module.