MANAGING REAL-TIME HANDWRITING RECOGNITION

• US 20170010802A1
• Filed: 09/23/2016
• Published: 01/12/2017
• Est. Priority Date: 06/09/2013
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: MEIER, Ueli, XIA, Mei-Qun, DOLFING, Jannes G., DIXON, Ryan S., GROETHE, Karl M., MISRA, Karan, BELLEGARDA, Jerome R.
• Granted Patent: US 10,346,035 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 3/018 Input/output arrangements for oriental characters
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 40/166 Editing, e.g. inserting or deleting
• CPC Class Codes: G06F 40/274 Converting codes to words; Guess-ahead of partial word inputs
• CPC Class Codes: G06V 30/347 Sampling; Contour coding; Stroke extraction
• CPC Class Codes: G06V 30/36 Matching; Classification
• CPC Class Codes: G06V 30/387 using human interaction, e.g. selection of the best displayed recognition candidate

Abstract

Methods, systems, and computer-readable media related to a technique for providing handwriting input functionality on a user device. A handwriting recognition module is trained to have a repertoire comprising multiple non-overlapping scripts and capable of recognizing tens of thousands of characters using a single handwriting recognition model. The handwriting input module provides real-time, stroke-order and stroke-direction independent handwriting recognition for multi-character handwriting input. In particular, real-time, stroke-order and stroke-direction independent handwriting recognition is provided for multi-character, or sentence level Chinese handwriting recognition. User interfaces for providing the handwriting input functionality are also disclosed.

Claims

1. A method, comprising:

at an electronic device having a one or more processors, a touch sensitive surface, and a display;

displaying a user input interface on the display including a message area and a stroke input area;

receiving a first set of strokes on the touch sensitive surface in the stroke input area;

determining first text based on the first set of strokes;

displaying the first text in the message area;

receiving a second set of strokes on the touch sensitive surface in the stroke input area after receiving the first set of strokes and displaying the first text;

determining revised first text based on the first set of strokes and the second set of strokes; and

replacing the display of the first text with the revised first text.

2. The method of claim 1, further comprising:

rendering the first set of strokes in the stroke input area; and

starting a fading process for the first set of strokes, wherein during the fading process, the rendering of the first set of strokes in the stroke input area becomes increasingly faded.

3. The method of claim 2, wherein the fading process for the first set of strokes is started when a predetermined time period has elapsed after the first set of strokes is completed by the user.

4. The method of claim 2, wherein the fading process for the first set of strokes is started when the user has started inputting the second set of strokes.

5. The method of claim 2, wherein an end state of the fading process for the first set of strokes is a state with a predetermined minimum visibility for the first set of strokes.

6. The method of claim 2, wherein an end state of the fading process for the first set of strokes is a state with zero visibility for the first set of strokes.

7. The method of claim 1, wherein displaying the first text occurs before receiving the second set of strokes.

8. The method of claim 1, further comprising:

after receiving the second set of strokes and after displaying the revised first text, receiving a third set of strokes on the touch sensitive surface in the stroke input area;

determining second text based on the third set of strokes; and

displaying the second text with the revised first text.

9. The method of claim 1, wherein the first set of strokes is a single continuous stroke.

10. The method of claim 1, wherein the second set of strokes is a single continuous stroke.

11. The method of claim 1, wherein the first text includes a character not included in the revised first text.

12. A non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of an electronic device with a touch sensitive surface and a display, the one or more programs including instructions for:

displaying a user input interface on the display including a message area and a stroke input area;

receiving a first set of strokes on the touch sensitive surface in the stroke input area;

determining first text based on the first set of strokes;

displaying the first text in the message area;

receiving a second set of strokes on the touch sensitive surface in the stroke input area after receiving the first set of strokes and displaying the first text;

determining revised first text based on the first set of strokes and the second set of strokes; and

replacing the display of the first text with the revised first text.

13. An electronic device, comprising:

a touch sensitive surface;

a display;

one or more processors; and

a memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for;

displaying a user input interface on the display including a message area and a stroke input area;

receiving a first set of strokes on the touch sensitive surface in the stroke input area;

determining first text based on the first set of strokes;

displaying the first text in the message area;

receiving a second set of strokes on the touch sensitive surface in the stroke input area after receiving the first set of strokes and displaying the first text;

determining revised first text based on the first set of strokes and the second set of strokes; and

replacing the display of the first text with the revised first text.

14. A method, comprising:

at an electronic device having one or more processors, a touch sensitive surface, and a display;

displaying a user input interface on the display including a message area and a stroke input area;

receiving a first set of strokes on the touch sensitive surface in the stroke input area;

determining first text based on the first set of strokes;

displaying the first text on the display in the message area;

determining one or more candidates based on the first set of strokes;

displaying the one or more candidates in a candidate display area;

while displaying the one or more candidates, receiving user input selecting a first candidate from the one or more candidates;

in response to the user input;

replacing display of the first text with display of the selected first candidate; and

clearing the one or more candidates from the candidate display area.

15. The method of claim 14, wherein the first text includes multiple characters.

16. The method of claim 14, wherein the first text is a single character.

17. The method of claim 14, wherein the one or more candidates include an emoji identified based on the first text.

18. The method of claim 14, further comprising:

receiving a second set of strokes on the touch sensitive surface in the stroke input area after receiving the first set of strokes and displaying the first text, wherein determining the one or more candidates is further based on the second set of strokes.

19. The method of claim 14, wherein the first set of strokes is a single continuous stroke.

20. The method of claim 18, wherein the second set of strokes is a single continuous stroke.

21. The method of claim 14, further comprising:

rendering the first set of strokes in the stroke input area; and

starting a fading process for the first set of strokes, wherein during the fading process, the rendering of the first set of strokes in the stroke input area becomes increasingly faded.

22. The method of claim 21, wherein the fading process for the first set of strokes is started when a predetermined time period has elapsed after the first set of strokes is completed by the user.

23. The method of claim 21, wherein an end state of the fading process for the first set of strokes is a state with a predetermined minimum visibility for the first set of strokes.

24. The method of claim 21, wherein an end state of the fading process for the first set of strokes is a state with zero visibility for the first set of strokes.

25. A non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of an electronic device with a touch sensitive surface and a display, the one or more programs including instructions for:

displaying a user input interface on the display including a message area and a stroke input area;

receiving a first set of strokes on the touch sensitive surface in the stroke input area;

determining first text based on the first set of strokes;

displaying the first text on the display in the message area;

determining one or more candidates based on the first set of strokes;

displaying the one or more candidates in a candidate display area;

while displaying the one or more candidates, receiving user input selecting a first candidate from the one or more candidates;

in response to the user input;

replacing display of the first text with display of the selected first candidate; and

clearing the one or more candidates from the candidate display area.

26. An electronic device, comprising:

a touch sensitive surface;

a display;

one or more processors; and

a memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for;

displaying a user input interface on the display including a message area and a stroke input area;

receiving a first set of strokes on the touch sensitive surface in the stroke input area;

determining first text based on the first set of strokes;

displaying the first text on the display in the message area;

determining one or more candidates based on the first set of strokes;

displaying the one or more candidates in a candidate display area;

while displaying the one or more candidates, receiving user input selecting a first candidate from the one or more candidates;

in response to the user input;

replacing display of the first text with display of the selected first candidate; and

clearing the one or more candidates from the candidate display area.