MANAGING REAL-TIME HANDWRITING RECOGNITION

• US 20190332259A1
• Filed: 07/08/2019
• Published: 10/31/2019
• Est. Priority Date: 06/09/2013
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: XIA, Mei-Qun, DOLFING, Jannes G., DIXON, Ryan S., GROETHE, Karl M., MISRA, Karan, BELLEGARDA, Jerome R., MEIER, Ueli
• Granted Patent: US 11,016,658 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 3/018 Input/output arrangements for oriental characters
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 40/166 Editing, e.g. inserting or deleting
• CPC Class Codes: G06F 40/274 Converting codes to words; Guess-ahead of partial word inputs
• CPC Class Codes: G06V 30/347 Sampling; Contour coding; Stroke extraction
• CPC Class Codes: G06V 30/36 Matching; Classification
• CPC Class Codes: G06V 30/387 using human interaction, e.g. selection of the best displayed recognition candidate

Abstract

Methods, systems, and computer-readable media related to a technique for providing handwriting input functionality on a user device. A handwriting recognition module is trained to have a repertoire comprising multiple non-overlapping scripts and capable of recognizing tens of thousands of characters using a single handwriting recognition model. The handwriting input module provides real-time, stroke-order and stroke-direction independent handwriting recognition for multi-character handwriting input. In particular, real-time, stroke-order and stroke-direction independent handwriting recognition is provided for multi-character, or sentence level Chinese handwriting recognition. User interfaces for providing the handwriting input functionality are also disclosed.

Claims

1. An electronic device, comprising:

a touch sensitive surface;

a display;

one or more processors; and

a memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for;

displaying a user input interface on the display including a message area and a stroke input area, wherein the stroke area is distinct from the message area;

receiving a first set of strokes on the touch sensitive surface in the stroke input area;

determining first text based on the first set of strokes;

displaying the first text on the display in the message area;

determining one or more candidates based on the first set of strokes;

displaying the one or more candidates in a candidate display area;

while displaying the one or more candidates, receiving user input selecting a first candidate from the one or more candidates;

in response to the user input;

replacing display of the first text with display of the selected first candidate; and

clearing the one or more candidates from the candidate display area.

2. The electronic device of claim 1, wherein the first text includes multiple characters.

3. The electronic device of claim 1, wherein the first text is a single character.

4. The electronic device of claim 1, wherein the one or more candidates include an emoji identified based on the first text.

5. The electronic device of claim 1, the programs further including instructions for:

receiving a second set of strokes on the touch sensitive surface in the stroke input area after receiving the first set of strokes and displaying the first text, wherein determining the one or more candidates is further based on the second set of strokes.

6. The electronic device of claim 1, wherein the first set of strokes is a single continuous stroke.

7. The electronic device of claim 5, wherein the second set of strokes is a single continuous stroke.

8. The electronic device of claim 1, the programs further including instructions for:

rendering the first set of strokes in the stroke input area; and

starting a fading process for the first set of strokes, wherein during the fading process, the rendering of the first set of strokes in the stroke input area becomes increasingly faded.

9. The electronic device of claim 8, wherein the fading process for the first set of strokes is started when a predetermined time period has elapsed after the first set of strokes is completed by the user.

10. The electronic device of claim 8, wherein an end state of the fading process for the first set of strokes is a state with a predetermined minimum visibility for the first set of strokes.

11. The electronic device of claim 8, wherein an end state of the fading process for the first set of strokes is a state with zero visibility for the first set of strokes.

12. A non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of an electronic device with a touch sensitive surface and a display, the one or more programs including instructions for:

displaying a user input interface on the display including a message area and a stroke input area, wherein the stroke area is distinct from the message area;

receiving a first set of strokes on the touch sensitive surface in the stroke input area;

determining first text based on the first set of strokes;

displaying the first text on the display in the message area;

determining one or more candidates based on the first set of strokes;

displaying the one or more candidates in a candidate display area;

while displaying the one or more candidates, receiving user input selecting a first candidate from the one or more candidates;

in response to the user input;

replacing display of the first text with display of the selected first candidate; and

clearing the one or more candidates from the candidate display area.

13. The non-transitory computer-readable storage medium of claim 12, wherein the first text includes multiple characters.

14. The non-transitory computer-readable storage medium of claim 12, wherein the first text is a single character.

15. The non-transitory computer-readable storage medium of claim 12, wherein the one or more candidates include an emoji identified based on the first text.

16. The non-transitory computer-readable storage medium of claim 12, the programs further including instructions for:

receiving a second set of strokes on the touch sensitive surface in the stroke input area after receiving the first set of strokes and displaying the first text, wherein determining the one or more candidates is further based on the second set of strokes.

17. The non-transitory computer-readable storage medium of claim 12, wherein the first set of strokes is a single continuous stroke.

18. The non-transitory computer-readable storage medium of claim 16, wherein the second set of strokes is a single continuous stroke.

19. The non-transitory computer-readable storage medium of claim 12, the programs further including instructions for:

rendering the first set of strokes in the stroke input area; and

starting a fading process for the first set of strokes, wherein during the fading process, the rendering of the first set of strokes in the stroke input area becomes increasingly faded.

20. The non-transitory computer-readable storage medium of claim 19, wherein the fading process for the first set of strokes is started when a predetermined time period has elapsed after the first set of strokes is completed by the user.

21. The non-transitory computer-readable storage medium of claim 19, wherein an end state of the fading process for the first set of strokes is a state with a predetermined minimum visibility for the first set of strokes.

22. The non-transitory computer-readable storage medium of claim 19, wherein an end state of the fading process for the first set of strokes is a state with zero visibility for the first set of strokes.

23. A method, comprising:

at an electronic device having one or more processors, a touch sensitive surface, and a display;

displaying a user input interface on the display including a message area and a stroke input area, wherein the stroke area is distinct from the message area;

receiving a first set of strokes on the touch sensitive surface in the stroke input area;

determining first text based on the first set of strokes;

displaying the first text on the display in the message area;

determining one or more candidates based on the first set of strokes;

displaying the one or more candidates in a candidate display area;

while displaying the one or more candidates, receiving user input selecting a first candidate from the one or more candidates;

in response to the user input;

replacing display of the first text with display of the selected first candidate; and

clearing the one or more candidates from the candidate display area.

24. The method of claim 23, wherein the first text includes multiple characters.

25. The method of claim 23, wherein the first text is a single character.

26. The method of claim 23, wherein the one or more candidates include an emoji identified based on the first text.

27. The method of claim 23, further comprising:

receiving a second set of strokes on the touch sensitive surface in the stroke input area after receiving the first set of strokes and displaying the first text, wherein determining the one or more candidates is further based on the second set of strokes.

28. The method of claim 23, wherein the first set of strokes is a single continuous stroke.

29. The method of claim 27, wherein the second set of strokes is a single continuous stroke.

30. The method of claim 23, further comprising:

rendering the first set of strokes in the stroke input area; and

starting a fading process for the first set of strokes, wherein during the fading process, the rendering of the first set of strokes in the stroke input area becomes increasingly faded.

31. The method of claim 30, wherein the fading process for the first set of strokes is started when a predetermined time period has elapsed after the first set of strokes is completed by the user.

32. The method of claim 30, wherein an end state of the fading process for the first set of strokes is a state with a predetermined minimum visibility for the first set of strokes.

33. The method of claim 30, wherein an end state of the fading process for the first set of strokes is a state with zero visibility for the first set of strokes.