Decoding imprecise gestures for gesture-keyboards

• US 8,994,681 B2
• Filed: 04/26/2013
• Issued: 03/31/2015
• Est. Priority Date: 10/19/2012
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Kataoka, Satoshi, Kuroyanagi, Keisuke
• Primary Examiner(s): Sharifi-Tafreshi, Koosha
• Application Number:
  US13/871,876
• Publication Number:
  US 20140115521A1
• Time in Patent Office: 704 Days
• Field of Search: 345/156, 345/168, 345/169, 345/173, 345/174, 715/863, 715/864, 715/773
• US Class Current: 345/173
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0237 using prediction or retrieval techniques
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: G06F 40/274 Converting codes to words; Guess-ahead of partial word inputs

Abstract

In one example, a method includes outputting, by a computing device and for display, a graphical keyboard comprising a plurality of keys. The method may also include receiving, by the computing device, an indication of a gesture detected at a presence-sensitive input device. The method may also include determining, by the computing device and based on a plurality of features associated with the gesture, a degree of precision of the gesture. The method may also include modifying, by the computing device and based at least in part on the degree of precision, a probability that the gesture indicates at least one key of the plurality of keys. The method may also include outputting, by the computing device and for display, a candidate word that is based at least in part on the probability that the gesture indicates at least one key of the plurality of keys.

Claims

1. A method comprising:

outputting, by a computing device and for display, a graphical keyboard comprising a plurality of keys;

receiving, by the computing device, an indication of a gesture detected at a presence-sensitive input device that traverses a first key and a second key that are each included in the plurality of keys;

determining, by the computing device, a first distance of a portion of the gesture between the first key and the second key, and a second distance between the first key and the second key, wherein the first distance is a distance traversed by the gesture between first key and the second key, wherein the second distance is a distance of a straight-line path between the first key and the second key;

comparing, by the computing device, the first distance and the second distance to determine a value that represents a degree of precision of the gesture;

determining, by the computing device and based at least in part on comparing a particular location of the gesture with a spatial model associated with a particular key of the plurality of keys, a spatial model probability that indicates a probability that the particular key is selected by a user of the computing device;

responsive to determining that the value that represents the degree of precision of the gesture satisfies a threshold degree of degree of precision, modifying, by the computing device and based at least in part on the value that represents the degree of precision, the spatial model probability that the gesture indicates the particular key; and

outputting, by the computing device and for display, a candidate word that is based at least in part on the probability that the gesture indicates the the particular key of the plurality of keys.

2. The method of claim 1, wherein the degree of precision of the gesture satisfies the threshold degree of degree of precision if the degree of precision is less than the threshold degree of precision, wherein modifying the spatial model probability that the gesture indicates the particular key of the plurality of keys further comprises:

responsive to determining that the degree of precision is less than the threshold degree of precision, decreasing, by the computing device, the spatial model probability that the gesture indicates the particular key of the plurality of keys.

3. The method of claim 1, wherein the degree of precision of the gesture satisfies the threshold degree of degree of precision if the degree of precision is greater than the threshold degree of precision, wherein modifying the spatial model probability that the gesture indicates the particular key of the plurality of keys further comprises:

responsive to determining that the degree of precision is greater than the threshold degree of precision, increasing, by the computing device, the probability that the gesture indicates the particular key of the plurality of keys.

4. The method of claim 1,

wherein the probability that the gesture indicates the particular key of the plurality of keys is based at least in part on the spatial model probability associated with the particular key of the plurality of keys, and

wherein modifying, by the computing device and based at least in part on the degree of precision, the spatial model probability that the gesture indicates the particular key of the plurality of keys further comprises modifying, by the computing device and based at least in part on the degree of precision, the spatial model probability of the spatial model associated with the particular key.

5. The method of claim 1, wherein a combined probability that the gesture indicates the particular key of the plurality of keys is based at least in part on the spatial model probability associated with the particular key of the plurality of keys and a language model probability, wherein the language model probability is based at least in part on a character associated with the particular key.

20. The method of claim 1, further comprising:

determining, by the computing device, the first key and the second key; and

responsive to determining the first key and the second key, determining, by the computing device, the distance of the straight-line path between the first key and the second key.

21. The method of claim 1, wherein the spatial model generates one or more probabilities that a particular key of a graphical keyboard has been selected based on location data associated with a user input.

22. The method of claim 1, wherein the spatial model includes a plurality of bivariate Gaussian models, each respective bivariate Gaussian model associated with a respective key of the graphical keyboard.

23. The method of claim 22, wherein a respective bivariate Gaussian model associated with a respective key comprises a distribution of coordinates corresponding to locations of a presence-sensitive input device that are associated with the given key.

6. A non-transitory computer-readable storage medium encoded with instructions that, when executed, cause one or more processors of a computing device to perform operations comprising:

outputting, for display, a graphical keyboard comprising a plurality of keys;

receiving an indication of a gesture detected at a presence-sensitive input device that traverses a first key and a second key that are each included in the plurality of keys;

determining a first distance of a portion of the gesture between the first key and the second key, and a second distance between the first key and the second key, wherein the first distance is a distance traversed by the gesture between first key and the second key, wherein the second distance is a distance of a straight-line path between the first key and the second key;

comparing the first distance and the second distance to determine value that represents a the degree of precision of the gesture;

determining, based at least in part on comparing a particular location of the gesture with a spatial model associated with a particular key of the plurality of keys, a spatial model probability that indicates a probability that the particular key is selected by a user of the computing device;

responsive to determining that the value that represents the degree of precision of the gesture satisfies a threshold degree of degree of precision, modifying, based at least in part on the value that represents the degree of precision, the spatial model probability that the gesture indicates the particular key; and

outputting, for display, a candidate word that is based at least in part on the probability that the gesture indicates the particular key of the plurality of keys.

7. The non-transitory computer-readable storage medium of claim 6, wherein the degree of precision of the gesture satisfies the threshold degree of degree of precision if the degree of precision is less than the threshold degree of precision, the non-transitory computer-readable storage medium encoded with instructions that, when executed, cause one or more processors of the computing device to perform operations comprising:

responsive to determining that the degree of precision is less than the threshold degree of precision, decreasing the spatial model probability that the gesture indicates the particular key of the plurality of keys.

8. The non-transitory computer-readable storage medium of claim 6, wherein the degree of precision of the gesture satisfies the threshold degree of degree of precision if the degree of precision is less than the threshold degree of precision, encoded with instructions that, when executed, cause one or more processors of the computing device to perform operations comprising:

responsive to determining that the degree of precision is greater than the threshold degree of precision, increasing the probability that the gesture indicates the particular key of the plurality of keys.

9. The non-transitory computer-readable storage medium of claim 6,

wherein the probability that the gesture indicates the particular key of the plurality of keys is based at least in part on the spatial model probability associated with the particular key of the plurality of keys, and

wherein modifying, based at least in part on the degree of precision, the spatial model probability that the gesture indicates the particular key of the plurality of keys further comprises modifying, based at least in part on the degree of precision, the spatial model probability of the spatial model associated with the particular key.

10. The non-transitory computer-readable storage medium of claim 6, wherein a combined probability that the gesture indicates the particular key of the plurality of keys is based at least in part on the spatial model probability associated with the particular key of the plurality of keys and a language model probability, wherein the language model probability is based at least in part on a character associated with the particular key.

11. A computing device comprising:

at least one processor,

wherein the at least one processor is configured to;

output, for display, a graphical keyboard comprising a plurality of keys;

receive an indication of a gesture detected at a presence-sensitive input device that traverses a first key and a second key that are each included in the plurality of keys;

determine a first distance of a portion of the gesture between the first key and the second key, and a second distance between the first key and the second key, wherein the first distance is a distance traversed by the gesture between first key and the second key, wherein the second distance is a distance of a straight-line path between the first key and the second key;

compare the first distance and the second distance to determine a value that represents a degree of precision of the gesture;

determine, based at least in part on comparing a particular location of the gesture with a spatial model associated with a particular key of the plurality of keys, a spatial model probability that indicates a probability that the particular key is selected by a user of the computing device;

responsive to determining that the value that represents the degree of precision of the gesture satisfies a threshold degree of degree of precision, modify, based at least in part on the value that represents the degree of precision, the spatial model probability that the gesture indicates the particular key of the plurality of keys; and

output, for display, a candidate word that is based at least in part on the probability that the gesture indicates the particular key of the plurality of keys.

12. The computing device of claim 11, wherein the degree of precision of the gesture satisfies the threshold degree of degree of precision if the degree of precision is less than the threshold degree of precision, wherein the at least one processor is configured to:

responsive to determining that the degree of precision is less than the threshold degree of precision, decrease the spatial model probability that the gesture indicates the particular key of the plurality of keys.

13. The computing device of claim 11, wherein the degree of precision of the gesture satisfies the threshold degree of degree of precision if the degree of precision is less than the threshold degree of precision, wherein the at least one processor is configured to:

responsive to determining that the degree of precision is greater than the threshold degree of precision, increase the probability that the gesture indicates the particular key of the plurality of keys.

14. The computing device of claim 11,

wherein the probability that the gesture indicates at least one key of the plurality of keys is based at least in part on a spatial model probability associated with the at least one key of the plurality of keys, and

wherein the at least one processor configured to modify, based at least in part on the degree of precision, the spatial model probability that the gesture indicates the particular key of the plurality of keys is configured to modify, based at least in part on the degree of precision, the spatial model probability of the spatial model associated with the particular key.

15. The computing device of claim 11, wherein a combined probability that the gesture indicates the particular key of the plurality of keys is based at least in part on the spatial model probability associated with the particular key of the plurality of keys and a language model probability, wherein the language model probability is based at least in part on a character associated with the particular key.

16. A method comprising:

outputting, by a computing device and for display, a graphical keyboard comprising a plurality of keys;

receiving, by the computing device, an indication of a gesture detected at a presence-sensitive input device that traverses a first key and a second key that are each included in the plurality of keys;

determining, by the computing device, a speed of a portion of the gesture between the first key and the second key;

comparing, by the computing device, the speed of the portion of the gesture to at least one threshold speed to determine a value that represents a degree of precision of the gesture;

determining, by the computing device and based at least in part on comparing a particular location of the gesture with a spatial model associated with a particular key of the plurality of keys, a spatial model probability that indicates a probability that the particular key is intended to be selected by a user of the computing device;

responsive to determining that the value that represents the degree of precision of the gesture is greater than a threshold degree of degree of precision, modifying, by the computing device and based at least in part on the value that represents the degree of precision, the spatial model probability that the gesture indicates the particular key of the plurality of keys; and

outputting, by the computing device and for display, a candidate word that is based at least in part on the probability that the gesture indicates the particular key of the plurality of keys.

17. A computing device comprising:

at least one processor,

wherein the at least one processor is configured to;

output, for display, a graphical keyboard comprising a plurality of keys;

receive an indication of a gesture detected at a presence-sensitive input device that traverses a first key and a second key that are each included in the plurality of keys;

determine a speed of a portion of the gesture between the first key and the second key;

compare the speed of the portion of the gesture to at least one threshold speed to determine a value that represents a degree of precision of the gesture;

determine, based at least in part on comparing a particular location of the gesture with a spatial model associated with a particular key, a spatial model probability that indicates a probability that the particular key is intended to be selected by a user of the computing device;

responsive to determining that the value that represents the degree of precision of the gesture is greater than a threshold degree of degree of precision, modify, based at least in part on the value that represents the degree of precision, the spatial model probability that the gesture indicates the particular key of the plurality of keys; and

output, for display, a candidate word that is based at least in part on the probability that the gesture indicates the particular key of the plurality of keys.

18. A method comprising:

outputting, by a computing device and for display, a graphical keyboard comprising a plurality of keys;

receiving, by the computing device, an indication of a gesture detected at a presence-sensitive input device that traverses a first key and a second key that are each included in the plurality of keys;

determining, by the computing device, a curvature of a portion of the gesture between the first key and the second key;

comparing, by the computing device, the curvature of the portion of the gesture to at least one threshold curvature to determine a value that represents a degree of precision of the gesture;

determining, by the computing device and based at least in part on comparing a particular location of the gesture with a spatial model associated with a particular key, a spatial model probability that indicates a probability that the particular key is selected by a user of the computing device;

responsive to determining that the value that represents the degree of precision of the gesture satisfies a threshold degree of degree of precision, modifying, by the computing device and based at least in part on the value that represents the degree of precision, the spatial model probability that the gesture indicates the particular key of the plurality of keys; and

outputting, by the computing device and for display, a candidate word that is based at least in part on the probability that the gesture indicates the particular key of the plurality of keys.

19. A computing device comprising:

at least one processor,

wherein the at least one processor is configured to;

output, for display, a graphical keyboard comprising a plurality of keys;

receive an indication of a gesture detected at a presence-sensitive input device that traverses a first key and a second key that are each included in the plurality of keys;

determine a curvature of a portion of the gesture between the first key and the second key;

compare the curvature of the portion of the gesture to at least one threshold curvature to determine a value that represents a degree of precision of the gesture;

determine, based at least in part on comparing a particular location of the gesture with a spatial model associated with a particular key, a spatial model probability that indicates a probability that the particular key is intended to be selected by a user of the computing device;

responsive to determining that the value that represents the degree of precision of the gesture is greater than a threshold degree of degree of precision, modify, based at least in part on the value that represents the degree of precision the spatial model probability that the gesture indicates the particular key of the plurality of keys; and

output, for display, a candidate word that is based at least in part on the probability that the gesture indicates the particular key of the plurality of keys.