Character recognition for overlapping textual user input

• US 8,363,949 B2
• Filed: 01/06/2012
• Issued: 01/29/2013
• Est. Priority Date: 06/13/2011
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Rowley, Henry A., Och, Franz, Li, Yang
• Primary Examiner(s): STREGE, JOHN B
• Application Number:
  US13/345,311
• Publication Number:
  US 20120327105A1
• Time in Patent Office: 389 Days
• Field of Search: 382/186, 382/187, 345/173, 345/179
• US Class Current: 382/186
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 40/274 Converting codes to words; Guess-ahead of partial word inputs
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/127 with the intervention of an operator
• CPC Class Codes: G06V 30/1423 the instrument generating sequences of position coordinates corresponding to handwriting preprocessing or recognising digital ink G06V30/32
• CPC Class Codes: G06V 30/387 using human interaction, e.g. selection of the best displayed recognition candidate

Abstract

Techniques described herein may recognize handwritten characters that are written at least partially over the top of one another that are input to a computing device. The handwritten characters may be formed of one or more strokes. A user may write characters or parts of words over approximately the same area of graphical user interface (i.e., on top of each other) without having to wait for a timeout between character input and without having to select a button or provide another input indicating the character is complete before entering input for another character. Once a character is at least partially recognized, a graphical indication corresponding to the user input displayed on a screen may be altered. Such alterations may include fading or changing size or location of the graphical indication.

Claims

1. A method for recognizing characters, comprising:

receiving touch-based input relating to a sequence of strokes at a touch-based interface of a computing device, wherein a first subset of the sequence of strokes corresponds to a first area of the touch-based interface and a second subset of the sequence of strokes corresponds to a second area of the touch-based interface that at least partially overlaps the first area;

displaying a graphical representation of the first subset of the sequence of strokes on an output device coupled to the computing device;

determining a confidence level that a first character approximately matches the first subset of the sequence of strokes, wherein the confidence level is of at least a first confidence threshold, further comprising;

checking a combination of the first character and a determined character against a language reference;

ranking the first character based at least in part on approximate matches of the combination of the first character and one or more previously determined characters found in the language reference, wherein the ranking is related to the confidence level; and

selecting the first character when the ranking is above a selected threshold level;

altering the display of the graphical representation of the first subset of the sequence of strokes based on the confidence level; and

providing the first character for processing by an application executing on the computing device when the confidence level is of at least a second confidence threshold, wherein the application is designed to process characters from touch-based input.

2. The method of claim 1, wherein altering the display of the graphical representation of the first subset of the sequence of strokes further comprises:

ceasing to display the graphical representation of the first subset of the sequence of strokes when the confidence level is of at least the second confidence threshold.

3. The method of claim 1, wherein determining the confidence level that the first character approximately matches the first subset of the sequence of strokes further comprises:

comparing a graphical representation of the first subset of the sequence of strokes to one or more characters of a set of characters;

ranking the one or more characters based at least on the comparison of the graphical representation to the one or more characters, wherein the rank is related to the confidence level; and

selecting the one or more characters as the first character based at least in part on the rank.

4. The method of claim 3, wherein ranking is based at least in part on how closely the graphical representation matches the one or more characters.

5. The method of claim 1, further comprising:

determining that the second subset of the sequence of strokes corresponds to a second character; and

generating the second character for processing by the application.

6. The method of claim 5, further comprising:

displaying the first character at a first location in a textual display; and

displaying the second character at a second location in the textual display, wherein the second location is based in part on a relationship between the first character and the second character.

7. The method of claim 1, wherein the first area is approximately the same as the second area.

8. The method of claim 1, wherein determining the confidence level that the first character approximately matches the first subset of the sequence of strokes further comprises:

arranging the sequence of strokes into a chronological order based at least on when at least one stroke of the sequence of strokes was received at the touch-based interface;

selecting at least a first stroke and comparing at least the first stroke to one or more characters; and

ranking the one or more characters based at least in part on the comparison between the first stroke and the one or more characters.

9. The method of claim 8, further comprising:

selecting the first character from the one or more characters based at least on the ranking of the one or more characters.

10. The method of claim 9, further comprising:

comparing at least one ranking corresponding to the first stroke with at least one ranking corresponding to the combination of the first and second strokes;

selecting one of at least the first stroke or the combination of the first and second strokes to be the subset of the sequence of strokes based at least in part on the comparison.

11. The method of claim 1, further comprising:

determining which strokes of the sequence of strokes to include in the first subset of the sequence of strokes based at least in part on vertical positions of the strokes within the touch-based interface, timing information relating to when the strokes were received, vertical spacing between the strokes, and horizontal spacing between the strokes.

12. The method of claim 11, wherein the first subset of strokes contains less than or equal to a threshold level of number of strokes.

13. The method of claim 1, wherein altering the display of the graphical representation of the first subset of the sequence of strokes further comprises:

fading the display of the graphical representation of the first subset of the sequence of strokes.

14. The method of claim 1, further comprising:

displaying a graphical representation of the second subset of the sequence of strokes on the output device coupled to the computing device.

15. The method of claim 1, wherein the sequence of strokes is a first sequence of strokes and wherein the determined character is a character determined from a previously received touch-based input relating to a second sequence of strokes.

16. The method of claim 1, wherein the first subset of the sequence of strokes comprises a first character or word and the second subset of the sequence of strokes comprises a second character or word.

17. A non-transitory computer-readable medium comprising instructions for causing a programmable processor to perform operations comprising:

receiving touch-based input relating to a sequence of strokes at a touch-based interface of a computing device, wherein a first subset of the sequence of strokes corresponds to a first graphical area of the touch-based interface and a second subset of the sequence of strokes corresponds to a second graphical area of the touch-based interface that at least partially overlaps the first graphical area;

displaying a graphical representation of the first subset of the sequence of strokes on an output device coupled to the computing device;

determining that the first subset of the sequence of strokes corresponds to a first character by at least a first confidence threshold, further comprising;

checking a combination of the first character and a determined character against a language reference;

ranking the first character based at least in part on approximate matches of the combination of the first character and previously determined characters found in the language reference, wherein the ranking is related to the confidence level; and

selecting the first character when the ranking is above a selected threshold level;

altering the graphical representation of the first subset of the sequence of strokes based on the determination that the first subset of the sequence of strokes corresponds to the first character; and

providing the first character for processing by an application executing on the computing device, wherein the application is designed to process characters from touch-based input.

18. A computing device, comprising:

one or more processors;

an input device to receive touch-based user input of a sequence of strokes related to textual information, wherein the sequence of strokes comprises a first subset of strokes corresponding to a first area of the input device and a second subset of strokes corresponding to a second area of the input device at least partially overlapping the first subset of strokes;

means for determining which strokes of the series of stroke fall into the first subset of the sequence of strokes and for determining that the first subset of the sequence of strokes corresponds to a first character, further comprising;

means for checking a combination of the first character and a determined character against a language reference;

means for ranking the first character based at least in part on approximate matches of the combination of the first character and previously determined characters found in the language reference, wherein the ranking is related to the confidence level; and

means for selecting the first character when the ranking is above a selected threshold level;

an output device to display a graphical representation of the first subset of the sequence of strokes, wherein the output device alters the graphical representation based on the determination that the first subset of the sequence of strokes corresponds to the first character;

wherein the output device further output the first character.