Gestural input at a virtual keyboard

• US 8,667,414 B2
• Filed: 08/22/2012
• Issued: 03/04/2014
• Est. Priority Date: 03/23/2012
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Zhai, Shumin, Li, Kun
• Primary Examiner(s): ULRICH, NICHOLAS S
• Application Number:
  US13/592,131
• Publication Number:
  US 20130249818A1
• Time in Patent Office: 559 Days
• Field of Search: 715/773, 715/863, 345/156, 345/168, 345/173
• US Class Current: 715/773
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus

Abstract

A computing device is operatively coupled to a presence-sensitive surface associated with a virtual keyboard. The computing device receives an indication of a gesture input on a portion of the presence-sensitive surface associated with the virtual keyboard. In response, the computing device defines a series of input points that represent a path of the gesture input across the virtual keyboard. In addition, the computing device defines anchor points for virtual keys of the virtual keyboard. The computing device then determines distance scores for the anchor points words based on distances on the presence-sensitive surface between the input points and the anchor points. The computing device identifies a word that corresponds to the gesture input based on the distance scores for the anchor points.

Claims

1. A method comprising:

receiving, by a device, an indication of a gesture input at a portion of a presence-sensitive surface associated with a virtual keyboard, the virtual keyboard comprising a set of virtual keys, each virtual key from the set of virtual keys being associated with a region of the presence-sensitive surface;

defining, by the device, a series of input points that represent a path of the gesture input across the virtual keyboard;

defining, by the device, respective sets of anchor points corresponding to respective words in a set of candidate words, each of the anchor points corresponding to a location of the presence-sensitive surface associated with a virtual key from the set of virtual keys;

identifying, by the device, for each respective anchor point in each of the sets of anchor points, a respective relevant input point in the series of input points;

determining, by the device, a respective distance score for each respective anchor point in the sets of anchor points, wherein for each respective anchor point in the sets of anchor points, the respective distance score for the respective anchor point is based at least in part on a distance on the presence-sensitive surface between the respective anchor point and the respective relevant input point for the respective anchor point, wherein determining the respective distance score for each respective anchor point in each respective set of anchor points comprises adjusting, by the device, the respective distance score for the respective anchor point in response to determining that the respective anchor point is not a closest one of the anchor points in the respective set of anchor points to the respective relevant input point for the respective anchor point; and

identifying, by the device, based at least in part on the respective distance scores for the respective anchor points, a word in the set of candidate words that corresponds to the gesture input.

2. The method of claim 1, wherein the anchor points correspond to centers of the virtual keys in the virtual keyboard.

3. The method of claim 1, further comprising after identifying the word, outputting, by the computing device, the word for display.

4. The method of claim 1,

wherein the method further comprises determining, by the device, a score for the word based at least in part on the respective distance scores for the respective anchor points that correspond to the word;

and

wherein identifying the word that corresponds to the gesture input comprises identifying, by the device, the word based at least in part on the score for the word.

5. The method of claim 4,

wherein the score for the word is a first score for the word and the method further comprises determining, by the device, a second score for the word based at least in part on a shape of the gesture input;

and

wherein identifying the word comprises identifying, by the device, the word based at least in part on the first score for the word and the second score for the word.

6. The method of claim 5, wherein determining the second score for the word comprises determining, by the device, the second score for the word based at least in part on distances between the input points and closest points on a shape template for the word.

7. The method of claim 6, further comprising:

resampling, by the device, the series of input points, wherein after re-sampling, the input points are spatially distributed evenly along the path of the gesture input;

scaling, by the device, the shape template into a bounding box, the bounding box having a width that is equal to a width of the gesture input;

translating, by the device, a position of the shape template such that a centroid of the shape template is located at a centroid of the series of input points; and

determining, by the device, the second score for the word after scaling the shape template and translating the position of the shape template.

8. The method of claim 1, wherein the word is a first word and the method comprises:

determining, by the device, respective anchor scores for respective words in the set of candidate words, the respective anchor scores for the respective words based at least in part on the distance scores for the anchor points that correspond to the respective words;

pruning, by the device, one or more words from the set of candidate words based at least in part on the respective anchor scores for the respective words;

after pruning the one or more words from the set of candidate words based at least in part on the respective anchor scores for the respective words, determining, by the device, respective shape scores for respective words remaining in the set of candidate words, the respective shape scores for the respective words remaining in the set of candidate words being based at least in part on distances between the input points and closest points on shape templates for the respective words remaining in the set of candidate words; and

identifying, by the device, based at least in part on the respective anchor scores and the respective shape scores for the respective words remaining in the set of candidate words, that the first word corresponds to the gesture input.

9. The method of claim 1,

wherein the score for the word is a first score for the word and identifying the word comprises:

determining, by the device, an anchor score for the word, the anchor score for the word based at least in part on the respective distance scores for the respective anchor points that correspond to the word; and

determining, by the device, a second score for the word, the second score based at least in part on a conditional probability of the word given one or more previous words; and

wherein identifying the word comprises identifying, by the device, the word from among the set of words based at least in part on the anchor score for the word and the second score for the word.

10. The method of claim 1, further comprising selecting, by the device, the set of candidate words based at least in part on an initial input point of the gesture input and a terminal input point of the gesture input.

11. The method of claim 1, wherein defining the series of input points comprises:

determining, by the device, that the number of input points in the series of input points exceeds a threshold; and

down-sampling, by the device prior to identifying the respective relevant input points, the series of input points in response to determining that the number of input points in the series of input points exceeds the threshold.

12. The method of claim 1,

wherein the method further comprises storing, by the device, a cache that indicates distances between the anchor points and the input points;

and

wherein determining the respective distance score for each respective anchor point in the sets of anchor points comprises determining, by the device, the distance score for an anchor point based at least in part on one or more of the distances indicated by the cache.

13. The method of claim 1, wherein receiving the indication of the gesture input comprises:

detecting, by the device, a start of the gesture input if the presence-sensitive surface detects a presence of an input object at a position associated with the virtual keyboard; and

detecting, by the device, a completion of the gesture input if the presence-sensitive surface detects a movement of the input object to a position associated with the completion of the gesture input.

14. The method of claim 1, wherein identifying a respective relevant input point for each respective anchor point in each of the sets of anchor points comprises:

identifying, by the device, an initial input point in the series of input points as the respective relevant input point for the respective anchor point in response to determining that the respective anchor point is an anchor point for an initial key of the word;

identifying, by the device, a terminal input point in the series of input points as the respective relevant input point for the respective anchor point in response to determining that the respective anchor point is an anchor point for a terminal key of the word; and

identifying, by the device, a given input point as the respective relevant input point for the respective anchor point in response to determining that the given input point is closer to the respective anchor point than any other one of the input points and that the respective anchor point does not correspond to the initial or terminal key of the word.

15. A computing device that comprises:

one or more processors; and

a memory that stores instructions that, when executed by the one or more processors, configure the computing device to;

output a virtual keyboard for display at a presence-sensitive display;

receive an indication of a gesture input at a portion of the presence-sensitive display associated with the virtual keyboard;

define a series of input points that represent a path of the gesture input across the virtual keyboard;

define respective sets of anchor points that correspond to respective words in a set of candidate words, each of the anchor points corresponding to a location associated with a virtual key from the set of virtual keys;

identify, for each respective anchor point in each of the sets of anchor points, a respective relevant input point in the series of input points;

determine a respective distance score for each respective anchor point in the sets of anchor points, wherein for each respective anchor point in the sets of anchor points, the respective distance score for the respective anchor point is based at least in part on a distance on the presence-sensitive surface between the respective anchor point and the respective relevant input point for the anchor point;

for each respective anchor point in each respective set of anchor points, adjust the respective distance score for the respective anchor point in response to determining that the respective anchor point is not a closest one of the anchor points in the respective set of anchor points to the respective relevant input point for the respective anchor point;

determine respective scores for respective words from the set of candidate words based at least in part on the respective distance scores for the respective anchor points that correspond to the respective words; and

identify, based at least in part on the respective scores for the respective words, that a given word in the set of candidate words corresponds to the gesture input.

16. The computing device of claim 15, wherein the instructions, when executed by the one or more processors, configure the computing device to:

receive an indication of a start of the gesture input if the presence-sensitive display detects a presence of an input object at a position associated with the virtual keyboard; and

receive an indication of a completion of the gesture input if the presence-sensitive display detects a movement of the input object to a position associated with the completion of the gesture input.

17. The computing device of claim 15, wherein the instructions, when executed by the one or more processors, configure the computing device to output the given word for display in response to determining that the given word corresponds to the gesture input.

18. The computing device of claim 15, wherein the respective scores for the respective words are respective first scores for the respective words and the instructions, when executed by the one or more processors, configure the computing device to:

determine, based at least in part on a shape of the gesture input, respective second scores for the respective words; and

determine, based at least in part on the respective first scores for the respective words and the respective second scores for the respective words, that the given word corresponds to the gesture input.

19. A computer-readable storage medium comprising instructions for causing one or more processors of a computing device to perform operations comprising:

outputting a virtual keyboard for display at a presence-sensitive display, the virtual keyboard including virtual keys associated with characters;

receiving an indication of a start of a gesture input if a presence-sensitive display detects a presence of an input object at a position associated with the virtual keyboard;

receiving an indication of a completion of the gesture input if the presence-sensitive display detects a movement of the input object to a position associated with the completion of the gesture input;

defining a series of input points that represent a path of the gesture input across the virtual keyboard;

down-sampling the series of input points;

defining respective sets of anchor points that correspond to respective words in a set of candidate words, each of the anchor points corresponding to a location of the presence-sensitive display associated with one of the virtual keys;

identifying, for each respective anchor point in each of the sets of anchor points, a respective relevant input point in the set of anchor points;

determining a respective distance score for each respective anchor point in the sets of anchor points, wherein for each respective anchor point in the sets of anchor points, the respective distance score for the respective anchor point is based at least in part on a distance on the presence-sensitive surface between the respective anchor point and the respective relevant input point for the respective anchor point;

for each respective anchor point in each respective set of anchor points, adjusting the respective distance score for the respective anchor point in response to determining that the respective anchor point is not a closest one of the anchor points in the respective set of anchor points to the respective relevant input point for the respective anchor point;

determining, based at least in part on the respective distance scores for the respective anchor points, respective scores for the respective words in the set of candidate words;

identifying, based at least in part on the respective scores for the respective words, a word that corresponds to the gesture input; and

outputting the word for display at the presence-sensitive display.