Multi-gesture text input prediction

• US 10,977,440 B2
• Filed: 07/12/2017
• Issued: 04/13/2021
• Est. Priority Date: 10/16/2012
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Ouyang, Yu, Zhai, Shumin, Bi, Xiaojun
• Primary Examiner(s): Tran, Tuyetlien T
• Application Number:
  US15/647,887
• Publication Number:
  US 20170308522A1
• Time in Patent Office: 1,371 Days
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0237 using prediction or retrieval techniques
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: G06F 40/274 Converting codes to words; Guess-ahead of partial word inputs

Abstract

A computing device outputs a keyboard for display, receives an indication of a first gesture to select a first sequence of one or more keys, determines a set of candidate strings based in part on the first sequence of keys, and outputs for display at least one of the set of candidate strings. The computing device receives an indication of a second gesture to select a second sequence of one or more keys, and determines that characters associated with the second sequence of keys are included in a first candidate word based at least in part on the set of candidate strings, or are included in a second candidate word not based on the first sequence of keys. The computing device modifies the set of candidate strings based at least in part on the determination and outputs for display at least one of the modified candidate strings.

Claims

1. A method comprising:

displaying, by a computing device, a graphical keyboard comprising a plurality of alphabet character keys and a terminating key;

detecting, by the computing device, one or more first gestures that select, without selecting the terminating key, a first set of alphabet character keys from the plurality of alphabet character keys;

determining, by the computing device, based on the first set of alphabet character keys, a first candidate string;

after displaying the first candidate string in a word suggestion region of a graphical user interface, and prior to detecting any gestures that select the terminating key, detecting, by the computing device, one or more second gestures that select, without selecting the terminating key, a second set of alphabet character keys from the plurality of alphabet character keys, wherein the one or more second gestures are disjoint from the one or more first gestures;

determining, by the computing device, whether alphabet characters associated with the second set of alphabet character keys are included or not included in the first candidate string;

responsive to determining that the alphabet characters associated with the second set of alphabet character keys are not included in the first candidate string, displaying, by the computing device, in the word suggestion region, a second candidate string that includes the alphabet characters associated with the second set of alphabet character keys and one or more space characters that separate the first candidate string from the second candidate string;

after displaying the second candidate string in the word suggestion region, and prior to detecting any gestures that select the terminating key, detecting, by the computing device, one or more third gestures that select, without selecting the terminating key, a third set of alphabet character keys from the plurality of alphabet character keys, wherein the one or more third gestures are disjoint from the one or more first gestures and the one or more second gestures;

determining, by the computing device, whether alphabet characters associated with the third set of alphabet character keys are included or not included in the first candidate string or the second candidate string; and

responsive to determining that the alphabet characters associated with the third set of alphabet character keys are included in the first candidate string or the second candidate string, continuing, by the computing device, the displaying of the first candidate string or the second candidate string, respectively, in the word suggestion region.

2. The method of claim 1, further comprising:

responsive to determining that the alphabet characters associated with the third set of alphabet character keys are not included in the first candidate string or the second candidate string, displaying, by the computing device, in the word suggestion region, a third candidate string that includes the alphabet characters associated with the third set of alphabet character keys and one or more space characters that separate the third candidate string from the second candidate string and the first candidate string.

3. The method of claim 1, wherein the one or more first gestures, the one or more second gestures, and the one or more third gestures are tap gestures.

4. The method of claim 1, wherein the one or more first gestures, the one or more second gestures, and the one or more third gestures are continuous gestures.

5. The method of claim 1, wherein the one or more first gestures, the one or more second gestures, and the one or more third gestures are a combination of tap or continuous gestures.

6. The method of claim 2, further comprising:

detecting, by the computing device, a fourth gesture that selects the word suggestion region; and

displaying, by the computing device, in a text display region of the graphical user interface, the first candidate string, the second candidate string, and the third candidate string each separated by the one or more space characters.

7. The method of claim 1, wherein the terminating key is a spacebar key of the graphical keyboard.

8. A non-transitory computer-readable storage medium encoded with instructions that, when executed, cause one or more processors of a computing device to:

display a graphical keyboard comprising a plurality of alphabet character keys and a terminating key;

detect one or more first gestures that select, without selecting the terminating key, a first set of alphabet character keys from the plurality of alphabet character keys;

determine, based on the first set of alphabet character keys, a first candidate string;

after a display of the first candidate string in a word suggestion region of a graphical user interface, and prior to a detection of any gestures that select the terminating key, detect one or more second gestures that select, without selecting the terminating key, a second set of alphabet character keys from the plurality of alphabet character keys, wherein the one or more second gestures are disjoint from the one or more first gestures;

determine whether alphabet characters associated with the second set of alphabet character keys are included or not included in the first candidate string;

responsive to a determination that the alphabet characters associated with the second set of alphabet character keys are not included in the first candidate string, display, in the word suggestion region, a second candidate string that includes the alphabet characters associated with the second set of alphabet character keys and one or more space characters that separate the first candidate string from the second candidate string;

after a display of the second candidate string in the word suggestion region, and prior to the detection of any gestures that select the terminating key, detect one or more third gestures that select, without selecting the terminating key, a third set of alphabet character keys from the plurality of alphabet character keys, wherein the one or more third gestures are disjoint from the one or more first gestures and the one or more second gestures;

determine whether alphabet characters associated with the third set of alphabet character keys are included or not included in the first candidate string or the second candidate string; and

responsive to a determination that the alphabet characters associated with the third set of alphabet character keys are included in the first candidate string or the second candidate string, continue to display the first candidate string or the second candidate string, respectively, in the word suggestion region.

9. The non-transitory computer-readable storage medium of claim 8, wherein the instructions, when executed, further cause the one or more processors to:

responsive to a determination that the alphabet characters associated with the third set of alphabet character keys are not included in the first candidate string or the second candidate string, display, in the word suggestion region, a third candidate string that includes the alphabet characters associated with the third set of alphabet character keys and one or more space characters that separate the third candidate string from the second candidate string and the first candidate string.

10. The non-transitory computer-readable storage medium of claim 8, wherein the one or more first gestures, the one or more second gestures, and the one or more third gestures are tap gestures.

11. The non-transitory computer-readable storage medium of claim 8, wherein the one or more first gestures, the one or more second gestures, and the one or more third gestures are continuous gestures.

12. The non-transitory computer-readable storage medium of claim 8, wherein the one or more first gestures, the one or more second gestures, and the one or more third gestures are a combination of tap or continuous gestures.

13. The non-transitory computer-readable storage medium of claim 9, wherein the instructions, when executed, further cause the one or more processors to:

detect a fourth gesture that selects the word suggestion region; and

display, in a text display region of the graphical user interface, the first candidate string, the second candidate string, and the third candidate string each separated by the one or more space characters.

14. The non-transitory computer-readable storage medium of claim 8, wherein the terminating key is a spacebar key of the graphical keyboard.

15. A computing device, comprising one or more processors configured to:

display a graphical keyboard comprising a plurality of alphabet character keys and a terminating key;

detect one or more first gestures that select, without selecting the terminating key, a first set of alphabet character keys from the plurality of alphabet character keys;

determine, based on the first set of alphabet character keys, a first candidate string;

after a display of the first candidate string in a word suggestion region of a graphical user interface, and prior to a detection of any gestures that select the terminating key, detect one or more second gestures that select, without selecting the terminating key, a second set of alphabet character keys from the plurality of alphabet character keys, wherein the one or more second gestures are disjoint from the one or more first gestures;

determine whether alphabet characters associated with the second set of alphabet character keys are included or not included in the first candidate string;

responsive to a determination that the alphabet characters associated with the second set of alphabet character keys are not included in the first candidate string, display, in the word suggestion region, a second candidate string that includes the alphabet characters associated with the second set of alphabet character keys and one or more space characters that separate the first candidate string from the second candidate string;

after a display of the second candidate string in the word suggestion region, and prior to the detection of any gestures that select the terminating key, detect one or more third gestures that select, without selecting the terminating key, a third set of alphabet character keys from the plurality of alphabet character keys, wherein the one or more third gestures are disjoint from the one or more first gestures and the one or more second gestures;

determine whether alphabet characters associated with the third set of alphabet character keys are included or not included in the first candidate string or the second candidate string; and

responsive to a determination that the alphabet characters associated with the third set of alphabet character keys are included in the first candidate string or the second candidate string, continue to display the first candidate string or the second candidate string, respectively, in the word suggestion region.

16. The computing device of claim 15, wherein the one or more processors are further configured to:

responsive to a determination that the alphabet characters associated with the third set of alphabet character keys are not included in the first candidate string or the second candidate string, display, in the word suggestion region, a third candidate string that includes the alphabet characters associated with the third set of alphabet character keys and one or more space characters that separate the third candidate string from the second candidate string and the first candidate string.

17. The computing device of claim 15, wherein the one or more first gestures, the one or more second gestures, and the one or more third gestures are tap gestures.

18. The computing device of claim 15, wherein the one or more first gestures, the one or more second gestures, and the one or more third gestures are continuous gestures.

19. The computing device of claim 15, wherein the one or more first gestures, the one or more second gestures, and the one or more third gestures are a combination of tap or continuous gestures.

20. The computing device of claim 16, wherein the one or more processors are further configured to:

detect a fourth gesture that selects the word suggestion region; and

display, in a text display region of the graphical user interface, the first candidate string, the second candidate string, and the third candidate string each separated by the one or more space characters.