Character recognition for overlapping textual user input

• US 8,094,942 B1
• Filed: 09/30/2011
• Issued: 01/10/2012
• Est. Priority Date: 06/13/2011
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Rowley, Henry A., Och, Franz, Li, Yang
• Primary Examiner(s): PERUNGAVOOR, SATHYANARAYA V
• Application Number:
  US13/250,746
• Time in Patent Office: 102 Days
• Field of Search: 382/187
• US Class Current: 382/187
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 40/274 Converting codes to words; Guess-ahead of partial word inputs
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/127 with the intervention of an operator
• CPC Class Codes: G06V 30/1423 the instrument generating sequences of position coordinates corresponding to handwriting preprocessing or recognising digital ink G06V30/32
• CPC Class Codes: G06V 30/387 using human interaction, e.g. selection of the best displayed recognition candidate

Abstract

Techniques described herein may recognize handwritten characters that are written at least partially over the top of one another that are input to a computing device. The handwritten characters may be formed of one or more strokes. A user may write characters or parts of words over approximately the same area of graphical user interface (i.e., on top of each other) without having to wait for a timeout between character input and without having to select a button or provide another input indicating the character is complete before entering input for another character. Once a character is at least partially recognized, a graphical indication corresponding to the user input displayed on a screen may be altered. Such alterations may include fading or changing size or location of the graphical indication.

Claims

1. A method for recognizing characters, comprising:

receiving touch-based input relating to a sequence of strokes at a touch-based interface of a computing device, wherein a first subset of the sequence of strokes corresponds to a first area of the touch-based interface and a second subset of the sequence of strokes corresponds to a second area of the touch-based interface that at least partially overlaps the first area;

displaying a graphical representation of the first subset of the sequence of strokes on an output device coupled to the computing device;

arranging the sequence of strokes into a chronological order based at least on when at least one stroke of the sequence of strokes was received at the touch-based interface;

selecting at least a first stroke and comparing at least the first stroke to one or more characters;

performing a first ranking of the one or more characters based at least in part on the comparison between the first stroke and the one or more characters;

comparing a combination of the first stroke and at least a second stroke consecutive with the first stroke to the one or more characters;

performing a second ranking of the one or more characters based at least on the comparison of the one or more characters with the combination of the first and second strokes;

determining a confidence level that a first character approximately matches the first subset of the sequence of strokes based on at least the first ranking and the second ranking, wherein the confidence level is of at least a first confidence threshold;

altering the display of the graphical representation of the first subset of the sequence of strokes based on the confidence level; and

providing the first character for processing by an application executing on the computing device when the confidence level is of at least a second confidence threshold, wherein the application is designed to process characters from touch-based input.

2. The method of claim 1, wherein altering the display of the graphical representation of the first subset of the sequence of strokes further comprises:

ceasing to display the graphical representation of the first subset of the sequence of strokes when the confidence level is of at least the second confidence threshold.

3. The method of claim 1, wherein determining the confidence level that the first character approximately matches the first subset of the sequence of strokes further comprises:

comparing a graphical representation of the first subset of the sequence of strokes to one or more characters of a set of characters;

ranking the one or more characters based at least on the comparison of the graphical representation to the one or more characters, wherein the rank is related to the confidence level; and

selecting the one or more characters as the first character based at least in part on the rank.

4. The method of claim 3, wherein ranking is based at least in part on how closely the graphical representation matches the one or more characters.

5. The method of claim 3, wherein comparing the graphical representation of the first subset to one or more characters further comprises:

comparing at least one property of the first subset of the sequence of strokes with a corresponding property of the one or more characters,

wherein the at least one property includes one of congruency between the graphical representation of the first subset and the one or more characters, vertical spacing of strokes, horizontal spacing of strokes, crossing of strokes, size of strokes, and number of strokes.

6. The method of claim 5, wherein ranking the one or more characters based at least on the comparison further comprises:

calculating a weighted average of the at least one property and at least a second property, wherein a weight applied to the one or more properties is determined using minimum error rate training.

7. The method of claim 1, further comprising:

determining that the second subset of the sequence of strokes corresponds to a second character; and

generating the second character for processing by the application.

8. The method of claim 7, further comprising:

displaying the first character at a first location in a textual display; and

displaying the second character at a second location in the textual display, wherein the second location is based in part on a relationship between the first character and the second character.

9. The method of claim 1, wherein the first area is approximately the same as the second area.

10. The method of claim 1, further comprising:

selecting the first character from the one or more characters based at least on the ranking of the one or more characters.

11. The method of claim 1, further comprising:

comparing at least one ranking corresponding to the first stroke with at least one ranking corresponding to the combination of the first and second strokes;

selecting one of at least the first stroke or the combination of the first and second strokes to be the subset of the sequence of strokes based at least in part on the comparison.

12. The method of claim 1, wherein determining the confidence level that the first character approximately matches the first subset of the sequence of strokes further comprises:

ranking the first character based at least in part on a determined character, wherein the ranking is related to the confidence level; and

selecting the first character when the ranking is above a selected threshold level.

13. The method of claim 12, wherein ranking the first character based at least in part on previously determined characters further comprises:

checking a combination of the first character and the determined character against a language reference; and

providing a rank to the first character based at least in part on approximate matches of the combination of the first character and the previously determined characters found in the language reference.

14. The method of claim 1, further comprising:

determining which strokes of the sequence of strokes to include in the first subset of the sequence of strokes based at least in part on vertical positions of the strokes within the touch-based interface, timing information relating to when the strokes were received, vertical spacing between the strokes, and horizontal spacing between the strokes.

15. The method of claim 14, wherein the first subset of strokes contains less than or equal to a threshold level of number of strokes.

16. The method of claim 1, wherein altering the display of the graphical representation of the first subset of the sequence of strokes further comprises:

fading the display of the graphical representation of the first subset of the sequence of strokes.

17. A tangible non-transitory computer-readable medium comprising instructions for causing a programmable processor to perform operations comprising:

receiving touch-based input relating to a sequence of strokes at a touch-based interface of a computing device, wherein a first subset of the sequence of strokes corresponds to a first graphical area of the touch-based interface and a second subset of the sequence of strokes corresponds to a second graphical area of the touch-based interface that at least partially overlaps the first graphical area;

displaying a graphical representation of the first subset of the sequence of strokes on an output device coupled to the computing device;

arranging the sequence of strokes into a chronological order based at least on when at least one stroke of the sequence of strokes was received at the touch-based interface;

selecting at least a first stroke and comparing at least the first stroke to one or more characters;

performing a first ranking of the one or more characters based at least in part on the comparison between the first stroke and the one or more characters;

comparing a combination of the first stroke and at least a second stroke consecutive with the first stroke to the one or more characters;

performing a second ranking of the one or more characters based at least on the comparison of the one or more characters with the combination of the first and second strokes;

determining a confidence level that a first character approximately matches the first subset of the sequence of strokes based on at least the first ranking and the second ranking, wherein the confidence level is of at least a first confidence threshold;

altering the graphical representation of the first subset of the sequence of strokes based on the determination that the first subset of the sequence of strokes corresponds to the first character; and

providing the first character for processing by an application executing on the computing device, wherein the application is designed to process characters from touch-based input.

18. A computing device, comprising:

one or more processors;

an input device to receive touch-based user input of a sequence of strokes related to textual information, wherein the sequence of strokes comprises a first subset of strokes corresponding to a first area of the input device and a second subset of strokes corresponding to a second area of the input device at least partially overlapping the first subset of strokes;

means for determining which strokes of the series of stroke fall into the first subset of the sequence of strokes;

means for arranging the sequence of strokes into a chronological order based at least on when at least one stroke of the sequence of strokes was received at the touch-based interface;

means for selecting at least a first stroke and comparing at least the first stroke to one or more characters;

means for performing a first ranking of the one or more characters based at least in part on the comparison between the first stroke and the one or more characters;

means for comparing a combination of the first stroke and at least a second stroke consecutive with the first stroke to the one or more characters;

means for performing a second ranking of the one or more characters based at least on the comparison of the one or more characters with the combination of the first and second strokes;

means for determining that the first subset of the sequence of strokes corresponds to a first character; and

an output device to display a graphical representation of the first subset of the sequence of strokes, wherein the output device alters the graphical representation based on the determination that the first subset of the sequence of strokes corresponds to the first character, and wherein the output device further outputs the first character.