Incremental multi-word recognition

• US 8,701,032 B1
• Filed: 03/06/2013
• Issued: 04/15/2014
• Est. Priority Date: 10/16/2012
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Zhai, Shumin, Ouyang, Yu, Wakasa, Ken, Kataoka, Satoshi
• Primary Examiner(s): Nguyen, Cao Kevin
• Application Number:
  US13/787,513
• Publication Number:
  US 20140108991A1
• Time in Patent Office: 405 Days
• Field of Search: 715/705, 715/715, 715/781, 715762-763, 715/825, 715/835, 715/773, 715/780, 715/702, 715/816, 715/817, 715863-864
• US Class Current: 715/773
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0237 using prediction or retrieval techniques
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: G06F 40/274 Converting codes to words; Guess-ahead of partial word inputs
• CPC Class Codes: G06F 40/284 Lexical analysis, e.g. tokenisation or collocates

Abstract

In one example, a computing device includes at least one processor that is operatively coupled to a presence-sensitive display and a gesture module operable by the at least one processor. The gesture module may be operable by the at least one processor to output, for display at the presence-sensitive display, a graphical keyboard comprising a plurality of keys and receive an indication of a continuous gesture detected at the presence-sensitive display, the continuous gesture to select a group of keys of the plurality of keys. The gesture module may be further operable to determine, in response to receiving the indication of the continuous gesture and based at least in part on the group of keys of the plurality of keys, a candidate phrase comprising a group of candidate words.

Claims

1. A method comprising:

outputting, by a computing device and for display, a graphical keyboard comprising a plurality of keys;

receiving, by the computing device, an indication of a gesture detected at a presence-sensitive input device, a first portion of the gesture to select a first key of the plurality of keys and a second portion of the gesture to select a second key of the plurality of keys;

determining, by the computing device and based at least in part on the first key, a word-level token comprising a single string of a plurality of predicted characters;

determining, by the computing device, that the word-level token represents a candidate word included in a lexicon;

determining, by the computing device and in response to determining that the word-level token represents the candidate word in the lexicon, a phrase-level token based at least in part on the word-level token and the second key, wherein the phrase-level token comprises a plurality of character strings; and

outputting for display, by the computing device and based at least in part on the plurality of character strings, a candidate phrase comprising the candidate word in the lexicon and a second character string, wherein the second character string begins with a character associated with the second key.

2. The method of claim 1, wherein the plurality of character strings of the phrase-level token comprises a first character string and the second character string, wherein the word-level token comprises the first character string, and wherein the second character string comprises the character associated with the second key.

3. The method of claim 1, wherein the word-level token comprises a first word-level token comprising a first single string of the plurality of predicted characters, the method further comprising:

determining, by the computing device and based at least in part on the first key and the second key, a second word-level token comprising a second single string of the plurality of predicted characters.

4. The method of claim 1, wherein the word-level token is a first word-level token comprising a first single string of the plurality of predicted characters, and wherein determining the phrase-level token comprises:

generating, in response to determining that the first word-level token represents the candidate word in the lexicon, a next-word token that indicates that a next selected key of the plurality of keys is a prefix of a second word-level token;

determining, in response to determining the next-word token, a second word-level token comprising a second single string of the plurality of predicted characters, wherein the second key is a prefix of the second word-level token; and

determining the phrase-level token as a combination of the first word-level token and the second word-level token.

5. The method of claim 4, wherein determining the phrase-level token further comprises:

determining, based on a plurality of features associated with the gesture, a group of alignment points traversed by the gesture;

determining respective cost values for each of at least the first key, the second key, and a third key, wherein each of the respective cost values represents a probability that an alignment point of the group of alignment points indicates a key of the plurality of keys;

determining a first combined cost value based at least in part on the determined cost value for the first key and the determined cost value for the second key;

determining a second combined cost value based at least in part on the determined cost value for the first key and the determined cost value for the third key;

comparing the first combined cost value and the second combined cost value; and

determining the second word-level token based on the comparison of the first combined cost value and the second combined cost value.

6. The method of claim 5, wherein the plurality of features associated with the gesture comprises at least one of:

a length of a segment of the gesture, wherein the segment comprises a path traversed by the gesture at the presence-sensitive display;

a direction of the segment of the gesture;

a curvature of the segment of the gesture;

a local speed that represents a rate at which the segment of the gesture was detected; and

a global speed that represents a rate at which the gesture was detected.

7. The method of claim 5, wherein determining the respective cost values for each of at least the first key, the second key, and the third key comprises:

determining respective physical cost values for each of at least the first key, the second key, and the third key, wherein each of the respective physical cost values represents a probability that at least one physical feature of an alignment point of the group of alignment points indicates at least one physical feature of a key of the plurality of keys;

determining respective lexical cost values for each of at least the first key, the second key, and the third key, wherein each of the respective lexical cost values represents a probability that a letter represented by a key of the plurality of keys is included in the candidate word; and

determining the respective cost values for each of at least the first key, the second key, and the third key based on the respective physical cost values and the respective lexical cost values for each of at least the first key, the second key, and the third key.

8. The method of claim 7, wherein determining the respective physical cost values for each of at least the first key, the second key, and the third key comprises comparing key regions of each of at least the first key, the second key, and the third key with at least one of the plurality of features associated with the gesture, wherein the key regions comprise locations of the presence-sensitive display that output the respective keys.

9. The method of claim 7, wherein determining the respective lexical cost values for each of at least the first key, the second key, and the third key comprises comparing each of at least the first key, the second key, and the third key with a language model.

10. The method of claim 9, wherein the language model comprises an n-gram language model.

11. The method of claim 9, wherein the language model comprises a group of predefined phrases.

12. The method of claim 1, wherein the phrase-level token is a first phrase-level token comprising a first plurality of character strings, the method further comprising:

determining, by the computing device, that the first phrase-level token satisfies a threshold;

responsive to determining that the first phrase-level token satisfies the threshold, selecting, by the computing device, a character string of the first plurality of character strings of the first phrase-level token as a committed character string; and

removing, by the computing device, the committed character string from the first plurality of character strings of the phrase-level token to determine a second phrase-level token comprising a second plurality of character strings.

13. The method of claim 12, further comprising:

outputting, by the computing device, the first phrase-level token for display at a text-suggestion region of the presence-sensitive display; and

responsive to determining that the first phrase-level token satisfies the threshold;

outputting, by the computing device, the committed character string for display at a committed-text region of the presence-sensitive display, wherein the committed-text region is different than the text-suggestion region; and

outputting, by the computing device, the second plurality of character strings for display at the text-suggestion region.

14. The method of claim 12, wherein determining that the first phrase-level token satisfies the threshold comprises determining that a number of character strings of the first plurality of character strings satisfies a threshold number of character strings.

15. The method of claim 1, wherein the lexicon is implemented, by the computing device, as a trie data structure.

16. The method of claim 1, wherein receiving the indication of the gesture comprises receiving an indication of a motion of an input unit from a first location of the presence-sensitive input device to a second location of the presence-sensitive input device with substantially constant contact between the input unit and the presence-sensitive input device.

17. The method of claim 1, wherein determining the phrase-level token further comprises determining that the phrase-level token includes at least one delimiter character, the at least one delimiter character being positioned between a first character string and a second character string of the plurality of character strings.

18. A non-transitory computer-readable storage medium encoded with instructions that, when executed, cause at least one processor of a computing device to:

output, for display at a presence-sensitive display operatively coupled to the computing device, a graphical keyboard comprising a plurality of keys;

receive an indication of a continuous gesture detected at the presence-sensitive display, the continuous gesture to select a group of keys of the plurality of keys; and

determine, in response to receiving the indication of the continuous gesture to select the group of keys, a phrase-level token representing a plurality of candidate words, wherein the phrase-level token comprises a plurality of character strings; and

output for display, based at least in part on the plurality of candidate words, a candidate phrase comprising a first candidate word in the lexicon and a second candidate word, wherein the second candidate word begins with a character associated with the second key.

19. A device comprising:

at least one processor that is operatively coupled to a presence-sensitive display; and

at least one module operable by the at least one processor to;

output, for display at the presence-sensitive display, a graphical keyboard comprising a plurality of keys;

receive an indication of a continuous gesture detected at the presence-sensitive display, the continuous gesture to select a first key and a second key in a group of keys of the plurality of keys; and

determine, in response to receiving the indication of the continuous gesture and based at least in part on the group of keys of the plurality of keys, a candidate phrase comprising a group of candidate words; and

output for display, based at least in part on the group of candidate words, a candidate phrase comprising a first candidate word and a second candidate word, wherein the first candidate word includes a first character associated with the first key and the second candidate word begins with a second character associated with the second key.