MANAGING REAL-TIME HANDWRITING RECOGNITION

• US 20140363083A1
• Filed: 05/29/2014
• Published: 12/11/2014
• Est. Priority Date: 06/09/2013
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: BELLEGARDA, Jerome R., XIA, Mei-Qun, DOLFING, Jannes G., DIXON, Ryan S., GROETHE, Karl M., MISRA, Karan, MEIER, Ueli
• Granted Patent: US 9,465,985 B2
• Time in Patent Office: Days
• US Class Current: 382/189
• CPC Class Codes: G06F 3/018 Input/output arrangements for oriental characters
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 40/166 Editing, e.g. inserting or deleting
• CPC Class Codes: G06F 40/274 Converting codes to words; Guess-ahead of partial word inputs
• CPC Class Codes: G06V 30/347 Sampling; Contour coding; Stroke extraction
• CPC Class Codes: G06V 30/36 Matching; Classification
• CPC Class Codes: G06V 30/387 using human interaction, e.g. selection of the best displayed recognition candidate

Abstract

Methods, systems, and computer-readable media related to a technique for providing handwriting input functionality on a user device. A handwriting recognition module is trained to have a repertoire comprising multiple non-overlapping scripts and capable of recognizing tens of thousands of characters using a single handwriting recognition model. The handwriting input module provides real-time, stroke-order and stroke-direction independent handwriting recognition for multi-character handwriting input. In particular, real-time, stroke-order and stroke-direction independent handwriting recognition is provided for multi-character, or sentence level Chinese handwriting recognition. User interfaces for providing the handwriting input functionality are also disclosed.

Claims

1. A non-transitory computer-readable media comprising instructions stored thereon, the instructions, when executed by one or more processors, cause the processors to perform operations comprising:

receiving a first handwriting input from a user, the first handwriting input comprising a plurality of handwritten strokes, and the plurality of handwritten strokes forming multiple recognition units distributed along a respective writing direction associated with a handwriting input area of a handwriting input interface;

rendering each of the plurality of handwritten strokes in the handwriting input area as the handwritten stroke is provided by the user;

starting a respective fading process for each of the multiple recognition units after the recognition unit is completely rendered, wherein during the respective fading process, the rendering of the recognition unit in the first handwriting input are becomes increasingly faded;

receiving a second handwriting input from the user over a region of the handwriting input area occupied by a faded recognition unit of the multiple recognition unit; and

in response to receiving the second handwriting input;

rendering the second handwriting input in the handwriting input area; and

clearing all faded recognition units from the handwriting input area.

2. The media of claim 1, comprising instructions, which when executed by the one or more processors, cause the processors to perform operations comprising:

generating one or more recognition results for the first handwriting input;

displaying the one or more recognition results in a candidate display area of the handwriting input interface; and

in response to receiving the second handwriting input, automatically, without user selection, entering a top-ranked recognition result displayed in the candidate display area into a text input area of the handwriting input interface.

3. The media of claim 1, comprising instructions, which when executed by the one or more processors, cause the processors to perform operations comprising:

storing an input stack comprising the first handwriting input and the second handwriting input;

generating one or more multi-character recognition results each comprising a respective spatial sequence of characters recognized from a concatenation of the first handwriting input and the second handwriting input; and

displaying the one or more multi-character recognition results in a candidate display area of the handwriting input interface, while the rendering of the second handwriting input has replaced the rendering of the first handwriting input in the handwriting input area.

4. The media of claim 1, wherein the respective fading process for each recognition unit is started when a predetermined time period has elapsed after the recognition unit is completed by the user.

5. The media of claim 1, wherein the respective fading process for each recognition unit is started when the user has started inputting the strokes for a next recognition unit after the recognition unit.

6. The media of claim 1, wherein an end state of the respective fading process for each recognition unit is a state with a predetermined minimum visibility for the recognition unit.

7. The media of claim 1, wherein an end state of the respective fading process for each recognition unit is a state with zero visibility for the recognition unit.

8. The media of claim 1, comprising instructions, which when executed by the one or more processors, cause the processors to perform operations comprising:

after a last recognition unit in the first handwriting input has become faded, receiving a predetermined revival input from the user; and

in response to receiving the predetermined revival input, reverting the last recognition unit from the faded state to an un-faded state.

9. The media of claim 8, wherein the predetermined revival input is an initial contact detected on a deletion button provided in the handwriting input interface.

10. The media of claim 8, wherein a sustained contact detected on the deletion button deletes the last recognition unit from the handwriting input area and revives the second to last recognition unit from the faded state to the un-faded state.

11. A method of providing real-time handwriting recognition, comprising:

at a device having one or more processors and memory;

receiving a first handwriting input from a user, the first handwriting input comprising a plurality of handwritten strokes, and the plurality of handwritten strokes forming multiple recognition units distributed along a respective writing direction associated with a handwriting input area of a handwriting input interface;

rendering each of the plurality of handwritten strokes in the handwriting input area as the handwritten stroke is provided by the user;

starting a respective fading process for each of the multiple recognition units after the recognition unit is completely rendered, wherein during the respective fading process, the rendering of the recognition unit in the first handwriting input are becomes increasingly faded;

receiving a second handwriting input from the user over a region of the handwriting input area occupied by a faded recognition unit of the multiple recognition unit; and

in response to receiving the second handwriting input;

rendering the second handwriting input in the handwriting input area; and

clearing all faded recognition units from the handwriting input area.

12. A system, comprising:

one or more processors; and

memory having instructions stored thereon, the instructions, when executed by the one or more processors, cause the processors to perform operations comprising;

receiving a first handwriting input from a user, the first handwriting input comprising a plurality of handwritten strokes, and the plurality of handwritten strokes forming multiple recognition units distributed along a respective writing direction associated with a handwriting input area of a handwriting input interface;

rendering each of the plurality of handwritten strokes in the handwriting input area as the handwritten stroke is provided by the user;

starting a respective fading process for each of the multiple recognition units after the recognition unit is completely rendered, wherein during the respective fading process, the rendering of the recognition unit in the first handwriting input are becomes increasingly faded;

receiving a second handwriting input from the user over a region of the handwriting input area occupied by a faded recognition unit of the multiple recognition unit; and

in response to receiving the second handwriting input;

rendering the second handwriting input in the handwriting input area; and

clearing all faded recognition units from the handwriting input area.