MANAGING REAL-TIME HANDWRITING RECOGNITION

• US 20140365949A1
• Filed: 05/29/2014
• Published: 12/11/2014
• Est. Priority Date: 06/09/2013
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: XIA, Mei-Qun, DOLFING, Jannes G., DIXON, Ryan S., GROETHE, Karl M., MISRA, Karan, BELLEGARDA, Jerome R., MEIER, Ueli
• Granted Patent: US 9,898,187 B2
• Time in Patent Office: Days
• US Class Current: 715/780
• CPC Class Codes: G06F 3/018 Input/output arrangements for oriental characters
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 40/166 Editing, e.g. inserting or deleting
• CPC Class Codes: G06F 40/274 Converting codes to words; Guess-ahead of partial word inputs
• CPC Class Codes: G06V 30/347 Sampling; Contour coding; Stroke extraction
• CPC Class Codes: G06V 30/36 Matching; Classification
• CPC Class Codes: G06V 30/387 using human interaction, e.g. selection of the best displayed recognition candidate

Abstract

Methods, systems, and computer-readable media related to a technique for providing handwriting input functionality on a user device. A handwriting recognition module is trained to have a repertoire comprising multiple non-overlapping scripts and capable of recognizing tens of thousands of characters using a single handwriting recognition model. The handwriting input module provides real-time, stroke-order and stroke-direction independent handwriting recognition for multi-character handwriting input. In particular, real-time, stroke-order and stroke-direction independent handwriting recognition is provided for multi-character, or sentence level Chinese handwriting recognition. User interfaces for providing the handwriting input functionality are also disclosed.

Claims

1. A non-transitory computer-readable media comprising instructions stored thereon, the instructions, when executed by one or more processors, cause the processors to perform operations comprising:

receiving a handwriting input from a user, the handwriting input comprising a plurality of handwritten strokes provided in a touch-sensitive surface coupled to the device;

rendering, in real-time, the plurality of handwritten strokes in a handwriting input area of a handwriting input interface;

receiving one of a pinch gesture input and a expand gesture input over the plurality of handwritten strokes;

upon receiving a pinch gesture input, generating a first recognition result based on the plurality of handwritten strokes by treating the plurality of handwritten strokes as a single recognition unit;

upon receiving a expand gesture input, generating a second recognition result based on the plurality of handwritten strokes by treating the plurality of handwritten strokes as two separate recognition units pulled apart by the expand gesture input; and

upon generating a respective one of the first and second recognition results, displaying the generated recognition result in a candidate display area of the handwriting input interface.

2. The media of claim 1, wherein the pinch gesture input comprises two contacts on the touch-sensitive surface that converge toward each other in an area occupied by the plurality of handwritten strokes.

3. The media of claim 1, wherein the expand gesture input comprises two contacts on the touch-sensitive surface that diverge from each other in an area occupied by the plurality of handwritten strokes.

4. The media of claim 1, comprising instructions, which when executed by the one or more processors, cause the processors to perform operations comprising:

identifying two adjacent recognition units from the plurality of handwritten strokes;

displaying, in the candidate display area, an initial recognition result comprising respective characters recognized from the two adjacent recognition units; and

receiving the pinch gesture input while the initial recognition result is displayed in the candidate display area.

5. The media of claim 4, wherein displaying the first recognition result further comprises replacing the initial recognition result with the first recognition result in the candidate display area.

6. The media of claim 4, comprising instructions, which when executed by the one or more processors, cause the processors to perform operations comprising:

in response to the pinch gesture input, re-rendering the plurality of handwritten strokes to reduce a distance between the two adjacent recognition units in the handwriting input area.

7. The media of claim 1, comprising instructions, which when executed by the one or more processors, cause the processors to perform operations comprising:

identifying a single recognition unit from the plurality of handwritten strokes;

displaying, in the candidate display area, an initial recognition result comprising a character recognized from the single recognition unit; and

receiving the expand gesture input while the initial recognition result is displayed in the candidate display area.

8. The media of claim 7, wherein displaying the second recognition result further comprises replacing the initial recognition result with the second recognition result in the candidate display area.

9. The media of claim 8, comprising instructions, which when executed by the one or more processors, cause the processors to perform operations comprising:

in response to the expand gesture input, re-rendering the plurality of handwritten strokes to increase a distance between a first subset of strokes assigned to a first recognition unit and a second subset of handwritten strokes assigned to a second recognition unit in the handwriting input area.

10. A method of providing handwriting recognition, comprising:

at a device having one or more processors and memory;

receiving a handwriting input from a user, the handwriting input comprising a plurality of handwritten strokes provided in a touch-sensitive surface coupled to the device;

rendering, in real-time, the plurality of handwritten strokes in a handwriting input area of a handwriting input interface;

receiving one of a pinch gesture input and a expand gesture input over the plurality of handwritten strokes;

upon receiving a pinch gesture input, generating a first recognition result based on the plurality of handwritten strokes by treating the plurality of handwritten strokes as a single recognition unit;

upon receiving a expand gesture input, generating a second recognition result based on the plurality of handwritten strokes by treating the plurality of handwritten strokes as two separate recognition units pulled apart by the expand gesture input; and

upon generating a respective one of the first and second recognition results, displaying the generated recognition result in a candidate display area of the handwriting input interface.

11. A system, comprising:

one or more processors; and

memory having instructions stored thereon, the instructions, when executed by the one or more processors, cause the processors to perform operations comprising;

receiving a handwriting input from a user, the handwriting input comprising a plurality of handwritten strokes provided in a touch-sensitive surface coupled to the device;

rendering, in real-time, the plurality of handwritten strokes in a handwriting input area of a handwriting input interface;

receiving one of a pinch gesture input and a expand gesture input over the plurality of handwritten strokes;

upon receiving a pinch gesture input, generating a first recognition result based on the plurality of handwritten strokes by treating the plurality of handwritten strokes as a single recognition unit;

upon receiving a expand gesture input, generating a second recognition result based on the plurality of handwritten strokes by treating the plurality of handwritten strokes as two separate recognition units pulled apart by the expand gesture input; and

upon generating a respective one of the first and second recognition results, displaying the generated recognition result in a candidate display area of the handwriting input interface.