Gesture-keyboard decoding using gesture path deviation

• US 9,304,595 B2
• Filed: 05/07/2013
• Issued: 04/05/2016
• Est. Priority Date: 10/19/2012
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Kataoka, Satoshi, Kuroyanagi, Keisuke
• Primary Examiner(s): Ell, Matthew
• Application Number:
  US13/889,146
• Publication Number:
  US 20140115522A1
• Time in Patent Office: 1,064 Days
• Field of Search: 715/773, 715/863, 345/168, 345/173
• US Class Current: 1/1
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0237 using prediction or retrieval techniques
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus

Abstract

In one example, a method includes outputting, by a computing device and for display, a graphical keyboard comprising a group of keys. The method may further include receiving an indication of a first portion of a gesture to select a first key, and determining, based at least in part on the first key, one or more predicted next characters. The method may further include determining a path from a first location of the graphical keyboard to a second location of the graphical keyboard, the first location corresponding to the first key, the second location corresponding to one of the one or more predicted next characters. The method may further include comparing a second portion of the gesture with the determined path, and selecting, based at least in part on the comparison, the one of the one or more predicted next characters.

Claims

1. A method comprising:

outputting, by a computing device and for display, a graphical keyboard comprising a group of keys;

receiving, by the computing device, an indication of a first portion of a gesture detected at a presence-sensitive input device;

selecting, by the computing device and based at least in part on the first portion of the gesture, a first key from the group of keys of the graphical keyboard;

determining, by the computing device and based at least in part on the first key, one or more predicted next characters;

determining, by the computing device, a straight-line path from a first location of the graphical keyboard to a second location of the graphical keyboard, the first location corresponding to the first key, the second location corresponding to a key associated with one of the one or more predicted next characters;

receiving, by the computing device, an indication of a second portion of the gesture detected at the presence-sensitive input device;

comparing, by the computing device, the second portion of the gesture with the straight-line path from the first location to the second location to determine an indication of an angle between the second portion of the gesture and the straight-line path; and

selecting, by the computing device and based at least in part on the angle between the second portion of the gesture and the straight-line path from the first location to the second location, the one of the one or more predicted next characters.

2. The method of claim 1, further comprising:

outputting, by the computing device and for display, at least one candidate word, the at least one candidate word based at least in part on a character associated with the one of the one or more predicted keys.

3. The method of claim 1, wherein determining the indication of the angle between the second portion of the gesture and the straight-line path from the first location to the second location comprises determining a cosine of an angle between the second portion of the gesture and the straight-line path from the first location to the second location.

4. The method of claim 1, further comprising:

determining, by the computing device, that the determined indication of the angle satisfies a threshold value,

wherein selecting the one of the group of predicted next characters comprises selecting the one of the group of predicted next characters based at least in part on the determination that the indication of the angle satisfies the threshold value.

5. The method of claim 1, wherein the first location comprises a centroid of the first key, and wherein the second location comprises a centroid of the key of the one of the one or more predicted next characters.

6. The method of claim 1, further comprising:

determining, by the computing device and based at least in part on the straight-line path from the first location to the second location, a calculation region of the presence-sensitive input device,

wherein comparing the second portion of the gesture with the straight-line path from the first location to the second location comprises comparing a segment of the second portion of the gesture that is detected within the calculation region with the straight-line path from the first location to the second location.

7. The method of claim 6, wherein the calculation region comprises a region of the presence-sensitive input device that bounds, by at least one threshold distance, a segment of the straight-line path from the first location to the second location.

8. The method of claim 1, wherein determining the one or more predicted next characters further comprises determining the group of predicted next characters based at least in part on a language model.

9. The method of claim 8, wherein the language model comprises an n-gram language model.

10. The method of claim 1, further comprising:

determining, by the computing device and based at least in part on a plurality of features associated with the gesture, a group of alignment points traversed by the gesture;

determining, by the computing device, respective cost values for each of at least the first key, the second key, and a third key, wherein each of the respective cost values represents a probability that an alignment point of the group of alignment points indicates a key of the group of keys of the graphical keyboard;

determining, by the computing device, a first combined cost value based at least in part on the determined cost value for the first key and the determined cost value for the second key;

determining, by the computing device, a second combined cost value based at least in part on the determined cost value for the first key and the determined cost value for the third key; and

adjusting, by the computing device and based at least in part on the comparison of the second portion of the gesture with the straight-line path from the first location to the second location, at least one of the first combined cost value and the second combined cost value,

wherein selecting the one of the one or more predicted next characters further comprises selecting the one of the group of predicted next characters based at least in part on the at least one adjusted first combined cost value and second combined cost value.

12. The method of claim 10, wherein determining the respective cost values for each of at least the first key, the second key, and the third key comprises:

determining, by the computing device, respective physical cost values for each of at least the first key, the second key, and the third key, wherein each of the respective physical cost values represents a probability that at least one physical feature of an alignment point of the group of alignment points indicates at least one physical feature of a key of the group of keys of the graphical keyboard;

determining, by the computing device, respective lexical cost values for each of at least the first key, the second key, and the third key, wherein each of the respective lexical cost values represents a probability that a letter represented by a key of the group of keys of the graphical keyboard is included in a candidate word; and

determining, by the computing device and based at least in part on the respective physical cost values and the respective lexical cost values for each of at least the first key, the second key, and the third key, the respective cost values for each of at least the first key, the second key, and the third key.

13. The method of claim 12, wherein determining the respective physical cost values for each of at least the first key, the second key, and the third key comprises comparing key regions of each of at least the first key, the second key, and the third key with at least one of the plurality of features associated with the gesture, wherein the key regions comprise locations of the presence-sensitive input device that output the respective keys.

14. The method of claim 12, wherein determining the respective lexical cost values for each of at least the first key, the second key, and the third key comprises comparing each of at least the first key, the second key, and the third key with a language model.

11. The method of 10, wherein the plurality of features associated with the gesture comprises at least one of:

a length of a segment of the gesture, wherein the segment comprises a path traversed by the gesture at the presence-sensitive input device;

a direction of the segment of the gesture;

a curvature of the segment of the gesture;

a local speed that represents a rate at which the segment of the gesture was detected; and

a global speed that represents a rate at which the gesture was detected.

15. A non-transitory, non-signal computer-readable storage medium encoded with instructions that, when executed, cause at least one processor of a computing device to:

output, for display, a graphical keyboard comprising a group of keys;

receive an indication of a first portion of a gesture detected at a presence-sensitive input device;

select, based at least in part on the first portion of the gesture, a first key from the group of keys of the graphical keyboard;

determine, based at least in part on the first key, one or more predicted next characters;

determine a straight-line path from a first location of the graphical keyboard to a second location of the graphical keyboard, the first location corresponding to the first key, the second location corresponding to a key associated with one of the one or more predicted next characters;

receive an indication of a second portion of the gesture detected at the presence-sensitive input device;

compare the second portion of the gesture with the straight-line path from the first location to the second location to determine an indication of an angle between the second portion of the gesture and the determined straight-line path; and

select, based at least in part on the angle between the second portion of the gesture and the straight-line path from the first location to the second location, the one of the one or more predicted next characters.

16. A computing system comprising:

at least one processor;

a presence-sensitive input device operatively coupled to the at least one processor; and

at least one module operable by the at least one processor to;

output, for display, a graphical keyboard comprising a group of keys;

receive an indication of a first portion of a gesture detected at a presence-sensitive input device;

select, based at least in part on the first portion of the gesture, a first key from the group of keys of the graphical keyboard;

determine, based at least in part on the first key, one or more predicted next characters;

determine a straight-line path from a first location of the graphical keyboard to a second location of the graphical keyboard, the first location corresponding to the first key, the second location corresponding to a key associated with one of the one or more predicted next characters;

receive an indication of a second portion of the gesture detected at the presence-sensitive input device;

compare the second portion of the gesture with the straight-line path from the first location to the second location to determine an indication of an angle between the second portion of the gesture and the determined straight-line path; and

select, based at least in part on the angle between the second portion of the gesture and the straight-line path from the first location to the second location, the one of the one or more predicted next characters.

17. The computing system of claim 16,

wherein the at least one module is further operable by the at least one processor to determine that the determined indication of the angle satisfies a threshold value, and

wherein the at least one module is operable to select the one of the group of predicted next characters by at least being operable to select the one of the group of predicted next characters based at least in part on the determination that the indication of the angle satisfies the threshold value.