Touch keyboard using language and spatial models

• US 10,528,663 B2
• Filed: 10/27/2017
• Issued: 01/07/2020
• Est. Priority Date: 01/15/2013
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Zhai, Shumin, Chelba, Ciprian Ioan
• Primary Examiner(s): Ell, Matthew
• Application Number:
  US15/795,955
• Publication Number:
  US 20180046612A1
• Time in Patent Office: 802 Days
• CPC Class Codes: G06F 3/0237 using prediction or retrieval techniques
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: G06F 40/242 Dictionaries
• CPC Class Codes: G06F 40/263 Language identification
• CPC Class Codes: G06F 40/284 Lexical analysis, e.g. tokenisation or collocates
• CPC Class Codes: H04L 51/02 using automatic reactions or user delegation, e.g. automatic replies or chatbot-generated messages
• CPC Class Codes: H04L 51/063 Content adaptation, e.g. replacement of unsuitable content

Abstract

A computing device outputs for display at a presence-sensitive display, a graphical keyboard comprising a plurality of keys, receives an indication of at least one gesture to select a group of keys of the plurality of keys, and determines at least one characteristic associated with the at least one gesture to select the group of keys of the plurality of keys. The computing device modifies a spatial model based at least in part on the at least one characteristic and determines a candidate word based at least in part on data provided by the spatial model and a language model, wherein the spatial model provides data based at least in part on the indication of the at least one gesture and wherein the language model provides data based at least in part on a lexicon. The computing device outputs for display at the presence-sensitive display, the candidate word.

Claims

1. A method comprising:

presenting, via a display, a graphical keyboard, each key of the graphical keyboard is defined by a respective touch-point distribution pattern for that key, the respective touch-point distribution pattern for each key of the graphical keyboard specifies a variability in expected touch points for the key, and a respective standard deviation value associated with each key of the graphical keyboard indicates an amount of variability in the expected touch points for the key;

receiving, by a computing device and from a presence-sensitive input device that is operatively coupled to the display, a plurality of indications of input, each indication of input from the plurality of indications of input corresponding to a respective location of the display that presents the graphical keyboard;

determining, based on the respective locations of the plurality of indications of input, a first likelihood that the plurality of indications of input corresponds to a set of characters from the graphical keyboard;

determining a second likelihood that the set of characters from the graphical keyboard is included in a word in a lexicon;

improving accuracy of the first likelihood by modifying the first likelihood based on the second likelihood by increasing or decreasing a respective standard deviation value associated with a key of the graphical keyboard that is located at the respective location of the display corresponding to at least one of the plurality of indications of input;

determining, by the computing device, based on a product of the modified first likelihood and the second likelihood, at least one predicted current word; and

outputting, by the computing device and for display, the at least one predicted current word as a soft commit word in a text entry area of a graphical user interface.

2. The method of claim 1, wherein determining the first likelihood that the plurality of indications of input corresponds to the set of characters from the graphical keyboard comprises:

for each respective indication of input from the plurality of indications of input, incrementally modifying, by the computing device and based on the second likelihood, a respective spatial model score associated with each character from the set of characters; and

determining the first likelihood from the respective spatial model scores associated with the characters from the set of characters.

3. The method of claim 1, wherein modifying the first likelihood based on the second likelihood further comprises multiplying the second likelihood with the first likelihood to compute the product.

4. The method of claim 1, wherein increasing the respective standard deviation value associated with the key of the graphical keyboard that is located at the respective location of the display corresponding to the at least one of the plurality of indications of input decreases the first likelihood.

5. The method of claim 1, wherein decreasing the respective standard deviation value associated with the key of the graphical keyboard that is located at the respective location of the display corresponding to the at least one of the plurality of indications of input increases the first likelihood.

6. The method of claim 1, wherein modifying the first likelihood based on the second likelihood occurs incrementally while each of the plurality of indications of input is received.

7. The method of claim 1, further comprising:

determining, by the computing device and based at least in part on the at least one predicted current word, at least one predicted next word that follows the at least one predicted current word, wherein the at least one predicted next word that follows the at least one predicted current word and the at least one predicted current word are output for display in the text entry area of the graphical user interface.

8. A computing device comprising:

at least one processor; and

memory configured to store instructions that, when executed, cause the at least one processor to;

present, via a display, a graphical keyboard, each key of the graphical keyboard is defined by a respective touch-point distribution pattern for that key, the respective touch-point distribution pattern for each key of the graphical keyboard specifies a variability in expected touch points for the key, and a respective standard deviation value associated with each key of the graphical keyboard indicates an amount of variability in the expected touch points for the key;

receive, from a presence-sensitive input device that is operatively coupled to the display, a plurality of indications of input, each respective indication of input from the plurality of indications of input corresponding to a respective location of the display that presents the graphical keyboard;

determine, based on the respective locations of the plurality of indications of input, a first likelihood that the plurality of indications of input corresponds to a set of characters from the graphical keyboard;

determine a second likelihood that the set of characters from the graphical keyboard is included in a word in a lexicon;

improve accuracy of the first likelihood by modifying the first likelihood based on the second likelihood by increasing or decreasing a respective standard deviation value associated with a key of the graphical keyboard that is located at the respective location of the display corresponding to at least one of the plurality of indications of input;

determine, based on a product of the modified first likelihood and the second likelihood, at least one predicted current word; and

output, for display, the at least one predicted current word as a soft commit word in a text entry area of a graphical user interface.

9. The computing device of claim 8, wherein the instructions, when executed, further cause the at least one processor to determine the first likelihood that the plurality of indications of input corresponds to the set of characters from the graphical keyboard by:

for each respective indication of input from the plurality of indications of input, incrementally modifying, based on second likelihood, a respective spatial model score associated with each character from the set of characters; and

determining the first likelihood from the respective spatial model scores associated with the characters from the set of characters.

10. The computing device of claim 8, wherein the instructions, when executed, cause the at least one processor to further modify the first likelihood based on the second likelihood by multiplying the second likelihood with the first likelihood to compute the product.

11. The computing device of claim 8, wherein the instructions, when executed, further cause the at least one processor to increase the standard deviation value associated with the key of the graphical keyboard that is located at the respective location of the display corresponding to the at least one of the plurality of indications of input to decrease the first likelihood.

12. The computing device of claim 8, wherein the instructions, when executed, further cause the at least one processor to decrease the standard deviation value associated with the key of the graphical keyboard that is located at the respective location of the display corresponding to the at least one of the plurality of indications of input to decrease the first likelihood.

13. A non-transitory, non-signal computer-readable storage medium encoded with instructions that, when executed, cause at least one processor to:

present, via a display, a graphical keyboard, each key of the graphical keyboard is defined by a respective touch-point distribution pattern for that key, the respective touch-point distribution pattern for each key of the graphical keyboard specifies a variability in expected touch points for the key, and a respective standard deviation value associated with each key of the graphical keyboard indicates an amount of variability in the expected touch points for the key;

receive a plurality of indications of input, each respective indication of input from the plurality of indications of input corresponding to a respective location of the display that presents the graphical keyboard;

determine, based on the respective locations of the plurality of indications of input, a first likelihood that the plurality of indications of input corresponds to a set of characters from the graphical keyboard;

determine a second likelihood that the set of characters from the graphical keyboard is included in a word in a lexicon;

improve accuracy of the first likelihood by modifying the first likelihood based on the second likelihood by increasing or decreasing a respective standard deviation value associated with a key of the graphical keyboard that is located at the respective location of the display corresponding to at least one of the plurality of indications of input;

determine, based on a product of the modified first likelihood and the second likelihood, at least one predicted current word; and

output, for display, the at least one predicted current word as a soft commit word in a text entry area of a graphical user interface.

14. The non-transitory, non-signal computer-readable storage medium of claim 13, wherein the instructions, when executed, further cause the at least one processor to:

increase the respective standard deviation value associated with the key of the graphical keyboard that is located at the respective location of the display corresponding to the at least one of the plurality of indications of input to decrease the first likelihood.

15. The non-transitory, non-signal computer-readable storage medium of claim 13, wherein the instructions, when executed, further cause the at least one processor to decrease the respective standard deviation value associated with the key of the graphical keyboard that is located at the respective location of the display corresponding to the at least one of the plurality of indications of input to increase the first likelihood.