Incremental feature-based gesture-keyboard decoding

• US 8,782,549 B2
• Filed: 01/04/2013
• Issued: 07/15/2014
• Est. Priority Date: 10/05/2012
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Ouyang, Yu, Zhai, Shumin, Chelba, Ciprian Ioan, Bi, Xiaojun, Kataoka, Satoshi, Wakasa, Ken, Kuroyanagi, Keisuke
• Primary Examiner(s): Abdul-Ali, Omar
• Assistant Examiner(s): BHARGAVA, ANIL K
• Application Number:
  US13/734,810
• Publication Number:
  US 20140101594A1
• Time in Patent Office: 557 Days
• Field of Search: 715/773
• US Class Current: 715/773
• CPC Class Codes: G06F 3/0236 using selection techniques to select from displayed items
• CPC Class Codes: G06F 3/0237 using prediction or retrieval techniques
• CPC Class Codes: G06F 3/0484 for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: G06F 40/284 Lexical analysis, e.g. tokenisation or collocates
• CPC Class Codes: G06F 40/44 Statistical methods, e.g. probability models

Abstract

In one example, a method includes outputting, at a presence-sensitive display operatively coupled to a computing device, a graphical keyboard comprising a plurality of keys and receiving an indication of a gesture to select a group of keys of the plurality of keys. The method may further include determining, in response to the indication, a candidate word based at least in part on the group of keys. The method may further include determining a group of alignment points traversed by the gesture and respective cost values for each of at least two keys of the plurality of keys. The method may further include comparing the respective cost values for at least each of at least two keys of the plurality of keys to determine a combination of keys having a combined cost value and outputting the candidate word that is based on the respective cost values.

Claims

1. A method comprising:

outputting, by a computing device and for display, a graphical keyboard comprising a plurality of keys;

receiving an indication of a portion of a gesture entered at a presence-sensitive display to select at least one key of the plurality of keys, the gesture comprising a plurality of portions;

maintaining, by the computing device and based at least in part on the gesture, a plurality of letter chains associated with respective combined cost values, wherein each letter chain corresponds to a different group of one or more keys of the graphical keyboard;

incrementally determining for the portion of the gesture, by the computing device and in response to receiving the indication of the portion of the gesture, a candidate word, wherein the determining comprises;

determining, by the computing device, a group of one or more alignment points traversed by the portion of the gesture;

determining, by the computing device, respective cost values for each of at least two keys of the plurality of keys, wherein each of the respective cost values represents a probability that an alignment point of the group of alignment points indicates a key of the plurality of keys;

updating, by the computing device and based at least in part on at least one of the respective cost values for one or more of the at least two keys, at least one of the respective combined cost values associated with at least one of the plurality of letter chains;

comparing, by the computing device, the respective combined cost values respectively associated with the plurality of letter chains; and

outputting, by the computing device for display and based at least in part on the comparison of the respective combined cost values, the candidate word.

2. The method of claim 1, wherein determining the respective cost values for each of the at least two keys of the plurality of keys comprises:

determining, by the computing device, a first cost value for a first key of the plurality of keys based on a first alignment point of the group of alignment points and a second cost value for a second key of the plurality of keys based on a second alignment point of the group of alignment points.

3. The method of claim 1, wherein determining the respective cost values for each of the at least two keys of the plurality of keys comprises:

determining respective physical cost values for each of the at least two keys of the plurality of keys, wherein each of the respective physical cost values represents a probability that at least one physical feature of an alignment point of the group of alignment points indicates at least one physical feature of a key of the plurality of keys;

determining respective lexical cost values for each of the at least two keys of the plurality of keys, wherein each of the respective lexical cost values represents a probability that a letter represented by a key of the plurality of keys is included in the candidate word; and

comparing the respective physical cost values with the respective lexical cost values to determine the respective cost values for each of the at least two keys of the plurality of keys.

4. The method of claim 3, wherein determining the respective physical cost values for each of the at least two keys comprises comparing key regions of each of the at least two keys of the plurality of keys with at least one of the plurality of features associated with the gesture, wherein the key regions comprise locations of the presence-sensitive display that output the respective keys.

5. The method of claim 3, wherein determining the respective lexical cost values for each of the at least two keys comprises comparing each of the at least two keys of the plurality of keys with a language model.

6. The method of claim 5, wherein the language model comprises an n-gram language model.

7. The method of claim 3, further comprising:

weighting, by the computing device, the respective physical cost values differently than the respective lexical cost values.

8. The method of claim 7, further comprising:

in response to determining that the physical cost values satisfy one or more thresholds, weighting the lexical cost values with a first weighting value, and weighting the physical cost values with a second weighting value, wherein the first weighting value is greater than the second weighting value.

9. The method of claim 1, further comprising:

determining, by the computing device and based at least in part on a plurality of features associated with the gesture, the group of alignment points traversed by the gesture.

10. The method of claim 9, wherein the plurality of features associated with the gesture comprises at least one of:

a length of a segment of the gesture, wherein the segment comprises a path traversed by the gesture at the presence-sensitive display;

a direction of the segment of the gesture;

a curvature of the segment of the gesture;

a local speed that represents a rate at which the segment of the gesture was detected; and

a global speed that represents a rate at which the gesture was detected.

11. The method of claim 1, wherein the candidate word is determined contemporaneously with the detection of the gesture to select the at least one key of the plurality of keys.

12. The method of claim 1, further comprising:

copying, in response to the detecting portion of the gesture, a token from a first node of a lexicon to a second node of the lexicon, wherein the second node is a child node of the first node;

determining, based on a plurality of features associated with the portion of the gesture, an alignment point traversed by the portion of the gesture;

determining whether the second node is a terminal node, wherein each terminal node represents the candidate word;

copying, in response to determining that the second node is a terminal node, the candidate word represented by the second node to a list of output predictions;

determining whether the portion of the gesture is a final portion of the gesture, wherein the final portion of the gesture comprises the portion of the gesture detected prior to detecting that the gesture is complete; and

outputting, for display and in response to determining that the portion of the gesture is the final portion of the gesture, at least a portion of the list of output predictions.

13. The method of claim 12, wherein determining the alignment point traversed by the portion of the gesture further comprises:

determining a cost value for the alignment point, wherein the cost value represents a probability that the alignment point indicates the second node.

14. The method of claim 13, wherein determining the cost value for the alignment point comprises:

determining a physical cost value for the alignment point;

determining a lexical cost value for the alignment point; and

comparing the physical cost value and the lexical cost value to determine the cost value for the alignment point.

15. The method of claim 14, wherein determining the lexical cost value for the alignment point comprises comparing the alignment point with a language model.

16. The method of claim 15, wherein the lexicon comprises the language model.

17. The method of claim 12, further comprising:

determining a combined cost value for the candidate word, wherein the combined cost value comprises a probability that the gesture indicates the candidate word; and

removing, from the list output predictions, candidate words having combined cost values which fail to satisfy a threshold.

18. The method of claim 12, wherein the lexicon is implemented, by the computing device, as a trie data structure.

19. The method of claim 12, wherein each node of the lexicon corresponds to at least one key of the plurality of keys.

20. The method of claim 1, wherein outputting the candidate word that is based on the combination of keys comprises outputting, for display, the candidate word in response to determining, by the computing device, that combined cost value of the combination of keys satisfies a threshold.

21. The method of claim 1, further comprising:

determining, by the computing device, a physical distance of the gesture performed at the presence-sensitive display;

determining, by the computing device, a weight based at least in part on the physical distance of the gesture; and

applying, by the computing device, the weight to at least one of a lexical cost value of a spatial model and a physical cost value of a language model.

22. A computing device, comprising:

at least one processor; and

at least one module operable by the at least one processor to;

output, for display a graphical keyboard comprising a plurality of keys;

receive an indication of a portion of a gesture to select at least one key of the plurality of keys, the gesture comprising a plurality of portions;

maintain, based at least in part on the gesture, a plurality of letter chains associated with respective combined cost values, wherein each letter chain corresponds to a different group of one or more keys of the graphical keyboard;

incrementally determine, for the portion of the gesture and in response to receiving the indication of the portion of the gesture, a candidate word;

determine a group of one or more alignment points traversed by the portion of the gesture;

determine respective cost values for each of at least two of the plurality of keys, wherein each of the respective cost values represents a probability that an alignment point of the group of alignment point indicates a key of the plurality of keys;

update, based at least in part on at least one of the respective cost values for one or more of the at least two keys, at least one of the respective combined cost values associated with at least one of the plurality of letter chains;

compare the respective combined cost values respectively associated with the plurality of letter chains; and

determine the candidate word based at least in part on the comparison of the respective combined cost values.

23. A non-transitory computer-readable storage medium encoded with instructions that, when executed, cause at least one processor of a computing device to:

output, for display, a graphical keyboard comprising a plurality of keys;

receive an indication of a portion of a gesture entered at a presence-sensitive display to select at least one key of the plurality of keys, the gesture comprising a plurality of portions;

maintain, based at least in part on the gesture, a plurality of letter chains associated with respective combined cost values, wherein each letter chain corresponds to a different group of one or more keys of the graphical keyboard;

incrementally determine, for the portion of the gesture and in response to receiving the indication of the portion of the gesture, a candidate word, wherein the determining comprises;

determine, by the computing device, a group of one or more alignment points traversed by the portion of the gesture;

determine, by the computing device, respective cost values for each of at least two keys of the plurality of keys, wherein each of the respective cost values represents a probability that an alignment point of the group of alignment points indicates a key of the plurality of keys;

update, based at least in part on at least one of the respective cost values for one or more of the at least two keys, at least one of the respective combined cost values associated with at least one of the plurality of letter chains;

compare, by the computing device, the respective combined cost values respectively associated with the plurality of letter chains; and

output, for display and based at least in part on the comparison of the respective combined cost values, the candidate word.