TOUCH GESTURES FOR REMOTE CONTROL OPERATIONS

• US 20120216154A1
• Filed: 09/30/2011
• Published: 08/23/2012
• Est. Priority Date: 02/23/2011
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Li, Yang
• Granted Patent: US 8,271,908 B2
• Time in Patent Office: Days
• US Class Current: 715/863
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: H04N 21/41265 having a remote control device for bidirectional communication between the remote control device and client device unidirectional remote control device H04N21/42204
• CPC Class Codes: H04N 21/42224 Touch pad or touch panel provided on the remote control

Abstract

In general, this disclosure describes techniques for providing a user of a first computing device (e.g., a mobile device) with the ability to utilize the first computing device to control a second computing device (e.g., a television). Specifically, the techniques of this disclosure may, in some examples, allow the user to use drawing gestures on a mobile computing device to remotely control and operate the second computing device. Using a presence-sensitive user interface device (e.g., a touch screen), the user may use drawing gestures to indicate characters associated with operations and commands to control the second computing device.

Claims

1. A method comprising:

receiving, at a first device, via a presence-sensitive display of the first device, a first user input comprising a first drawing gesture, the first drawing gesture representing a drawing corresponding to a first character;

transmitting, by the first device to a second device wirelessly coupled to the first device, first data representing the first drawing gesture, wherein the second device includes a display displaying one or more selectable elements, and wherein at least a first one of the selectable elements includes the first character and is visually indicated on the display of the second device based upon the first data;

receiving, at the first device, via the presence-sensitive display, a second user input comprising a second drawing gesture, the second drawing gesture representing a drawing corresponding to a second character;

transmitting, by the first device to the second device, second data representing the second drawing gesture, wherein at least a second one of the selectable elements includes both the first character and the second character and is visually indicated on the display of the second device based upon the second data;

after transmitting the second data, receiving, at the first device, via the presence-sensitive display, a third user input comprising a motion gesture; and

transmitting, by the first device to the second device, third data representing the motion gesture, wherein at least a third one of the selectable elements includes the first character and is visually indicated on the display of the second device based upon the third data.

3. The method of claim 1, wherein the display of the second device outputs a representation of the first drawing gesture based on the first data.

4. The method of claim 1, further comprising:

receiving, at the first device, via the first device, a second motion gesture that defines a direction of movement;

transmitting, by the first device to the second device, motion data representing the second motion gesture, wherein at least a fourth one of the selectable elements is visually indicated on the display of the second device based upon the motion data representing the second motion gesture, according to a defined direction relative to the at least third one of the selectable elements.

5. The method of claim 1, further comprising:

receiving, at the first device, via the presence-sensitive display of the first device, a fourth user input comprising a selection gesture; and

transmitting, by the first device to the second device, fourth data representing the selection gesture, wherein the at least third one of the selectable elements is selected in response to the fourth data,

wherein the display of the second device outputs a new list of one or more selectable elements in response to the fourth data.

6. The method of claim 1, further comprising:

receiving, via the presence-sensitive display of the first device, a fourth user input comprising a selection gesture; and

transmitting, by the first device to the second device, fourth data representing the selection gesture, wherein the at least the third one of the selectable elements is selected in response to the fourth data,

wherein an operation associated with the at least the third one of the selectable elements is executed by the second device in response to the fourth data.

7. The method of claim 1, wherein the one or more selectable elements comprises at least one of:

one or more words, one or more graphical user interface elements, or one or more operations of the second device.

9. The method of claim 1, wherein the first data comprises data indicative of the first drawing gesture, and wherein the second device determines the first character based on the first data.

10. The method of claim 1, further comprising determining, by the first device, the first character based on the first drawing gesture, wherein the first data comprises the first character.

2. (canceled)

8. (canceled)

11. (canceled)

12. A computer-readable storage medium encoded with instructions that, when executed, cause one or more processors of a first device to perform operations comprising:

receiving, at a first device, via a presence-sensitive display of the first device, a first user input comprising a first drawing gesture, the first drawing gesture representing a drawing corresponding to a first character;

transmitting, by the first device to a second device wirelessly coupled to the first device, first data representing the first drawing gesture, wherein the second device includes a display outputting one or more selectable elements, and wherein at least a first one of the selectable elements includes the first character and is visually indicated on the display of the second device based upon the first data;

receiving, at the first device, via the presence-sensitive display of the first device, a second user input comprising a second drawing gesture, the second drawing gesture representing a drawing representing to a second character; and

transmitting, by the first device to the second device, data representing the second drawing gesture, wherein the at least one of a second of the selectable elements includes both the first character and the second character and is visually indicated on the display of the second device based upon the second data;

after transmitting the second data, receiving, at the first device, via the presence-sensitive display of the first device, a third user input comprising a motion gesture; and

transmitting, by the first device to the second device, third data representing the motion gesture, wherein at least a third one of the selectable elements includes the first character and is visually indicated on the display of the second device based upon the third data.

14. The computer-readable storage medium of claim 12, wherein the display of the second device outputs a representation of the first drawing gesture based on the first data.

13. (canceled)

15. A first device, comprising:

one or more processors;

a presence-sensitive display;

a gesture module operable by the one or more processors to receive, via the presence-sensitive, a first user input comprising a first drawing gesture and a second user input comprising a second drawing gesture, wherein the first drawing gesture represents a drawing corresponding to a first character, and wherein the second drawing gesture represents a drawing corresponding to a second character;

a network interface configured to transmit, to a second device wirelessly coupled to the first device, first data representing the first drawing gesture, wherein the second device includes a display outputting one or more selectable elements, and wherein at least a first one of the selectable elements includes the first character and is visually indicated on the display of the second device based upon the first data;

the network interface being further configured to transmit second data representing the second drawing gesture to the second device wirelessly coupled to the first device, wherein at least a second one of the selectable elements includes both the first character and the second character and is visually indicated on the display of the second device based upon the second data;

a user interface controller operable by the one or more processors to receive, via the presence-sensitive user interface device, a third user input comprising a motion gesture; and

the network interface being further configured to transmit third data representing the motion gesture to the second device, wherein at least a third one of the selectable elements includes the first character and is visually indicated on the display of the second device based upon the third data.

16. A system comprising:

a first device comprising;

a presence-sensitive display configured to receive user input;

a network interface configured to transmit data; and

a second device coupled to the first device, the second device comprising a display configured to output one or more selectable elements;

wherein the presence-sensitive display receives a first user input comprising a first drawing gesture, the first drawing gesture representing a drawing corresponding to a first character,

wherein the network interface transmits to the second device first data representing the first drawing gesture,

wherein at least a first one of the selectable elements includes the first character is visually indicated on the display of the second device based upon the first data,

wherein the presence-sensitive user interface device receives a second user input comprising a second drawing gesture, the second drawing gesture representing a drawing corresponding to a second character,

wherein the network interface transmits, to the second device, second data representing the second drawing gesture,

wherein at least a second one of the selectable elements includes both the first character and the second character and is visually indicated on the display of the second device based upon the second data,

wherein after the second data is transmitted, the presence-sensitive display receives a third user input comprising a motion gesture

wherein the network interface transmits to the second device third data representing the motion gesture, and

wherein a third at least one of the selectable elements includes the first character and is

visually indicated on the display of the second device based upon the third data.

17-20. -20. (canceled)