Character recognition on a computing device

• US 9,864,508 B2
• Filed: 06/22/2015
• Issued: 01/09/2018
• Est. Priority Date: 06/24/2014
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Dixon, Ryan S., Meier, Ueli
• Primary Examiner(s): Bloomquist, Keith
• Application Number:
  US14/746,620
• Publication Number:
  US 20150370779A1
• Time in Patent Office: 932 Days
• CPC Class Codes: G06F 2203/04101 2.5D-digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface and also measures the distance of the input means within a short range in the Z direction, possibly with a separate measurement setup
• CPC Class Codes: G06F 2203/04104 Multi-touch detection in digitiser, i.e. details about the simultaneous detection of a plurality of touching locations, e.g. multiple fingers or pen and finger
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/0236 using selection techniques to select from displayed items
• CPC Class Codes: G06F 3/0412 Digitisers structurally integrated in a display
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: G06F 40/274 Converting codes to words; Guess-ahead of partial word inputs
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/32 Digital ink

Abstract

In some embodiments, a device performs character recognition based on spatial and temporal components of touch input detected on a touch-sensitive surface. In some embodiments, a device provides feedback about handwritten input and its recognition by the device. In some embodiments, a device presents a user interface for changing previously-inputted characters.

Claims

1. An electronic device, comprising:

one or more processors;

memory; and

one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for;

receiving an indication of a first input that includes movement of a contact detected on a touch-sensitive surface of a device, wherein;

the movement of the contact comprises a first stroke, and

the first stroke has a spatial component and a temporal component;

determining a first probability that the first stroke corresponds to a first character based on the spatial component of the first stroke;

determining a second probability that the first stroke corresponds to the first character based on the temporal component of the first stroke;

determining an aggregate probability that the first stroke corresponds to the first character based on the first probability and the second probability;

detecting an end of the first stroke; and

after detecting the end of the first stroke;

detecting at least a portion of a second stroke; and

in response to detecting the portion of the second stroke and while the second stroke is still being detected, determining whether the first stroke and the second stroke correspond to a single character based on temporal information about the first stroke and the second stroke.

2. The electronic device of claim 1, wherein the determinations of the first, second and aggregate probabilities are performed while the first stroke is still being detected.

3. The electronic device of claim 1, wherein the determinations of the first, second and aggregate probabilities are repeated periodically while the first stroke is still being detected.

4. The electronic device of claim 3, wherein the one or more programs further include instructions for:

while the first stroke is still being detected;

determining, at a first time, a first aggregate probability that the first stroke corresponds to the first character;

providing a first character suggestion for display on a display device based on the first aggregate probability;

determining, at a second time after the first time, a second aggregate probability that the first stroke corresponds to the first character; and

providing a second character suggestion, different from the first character suggestion, for display on the display device based on the second aggregate probability.

5. The electronic device of claim 1, wherein the first input comprises the first stroke detected during a first time period and the second stroke detected during a second time period, the second stroke having a spatial component and a temporal component, and the one or more programs further include instructions for:

determining a third probability that the first stroke corresponds to the first character and the second stroke corresponds to a second character based on the temporal and spatial components of the first and second strokes;

determining a fourth probability that a combination of the first stroke and the second stroke corresponds to a third character based on the temporal and spatial components of the combination of the first and second strokes;

in accordance with a determination that the third probability is higher than the fourth probability, selecting the first character and the second character as inputs; and

in accordance with a determination that the fourth probability is higher than the third probability, selecting the third character as an input.

6. The electronic device of claim 5, wherein determining the third probability comprises determining the third probability based on a language model indicating a likelihood that the first character will be followed by the second character in a particular language.

7. The electronic device of claim 1, wherein the one or more programs further include instructions for:

determining a plurality of aggregate probabilities for a plurality of characters, a first aggregate probability comprising an aggregate probability that the first stroke corresponds to a first character of the plurality of characters and a second aggregate probability comprising an aggregate probability that the first stroke corresponds to a second character of the plurality of characters; and

selecting, from the plurality of characters, a character having a highest aggregate probability as an input.

8. The electronic device of claim 1, wherein:

the second stroke spatially overlaps at least a portion of the first stroke, and

the first stroke and the second stroke are determined as belonging to different characters based at least in part on the temporal information.

9. The electronic device of claim 1, wherein:

the first stroke is spatially segregated from the second stroke, and

the first stroke and the second stroke are determined as belonging to the same character based at least in part on the temporal information.

10. The electronic device of claim 1, wherein detecting the end of the first stroke comprises detecting a pause.

11. The electronic device of claim 1, wherein detecting the end of the first stroke comprises detecting a liftoff of the contact detected on the touch-sensitive surface of the device.

12. The electronic device of claim 1, wherein the temporal information about the first stroke and the second stroke comprises a proximity in time between the first stroke and the second stroke.

13. A method comprising:

at an electronic device with one or more processors and memory;

receiving an indication of a first input that includes movement of a contact detected on a touch-sensitive surface of a device, wherein;

the movement of the contact comprises a first stroke, and

the first stroke has a spatial component and a temporal component;

determining a first probability that the first stroke corresponds to a first character based on the spatial component of the first stroke;

determining a second probability that the first stroke corresponds to the first character based on the temporal component of the first stroke;

determining an aggregate probability that the first stroke corresponds to the first character based on the first probability and the second probability

detecting an end of the first stroke; and

after detecting the end of the first stroke;

detecting at least a portion of a second stroke; and

in response to detecting the portion of the second stroke and while the second stroke is still being detected, determining whether the first stroke and the second stroke correspond to a single character based on temporal information about the first stroke and the second stroke.

14. The method of claim 13, wherein the determinations of the first, second and aggregate probabilities are performed while the first stroke is still being detected.

15. The method of claim 13, wherein the determinations of the first, second and aggregate probabilities are repeated periodically while the first stroke is still being detected.

16. The method of claim 13, wherein the first input comprises the first stroke detected during a first time period and the second stroke detected during a second time period, the second stroke having a spatial component and a temporal component, the method further comprising:

determining a third probability that the first stroke corresponds to the first character and the second stroke corresponds to a second character based on the temporal and spatial components of the first and second strokes;

determining a fourth probability that a combination of the first stroke and the second stroke corresponds to a third character based on the temporal and spatial components of the combination of the first and second strokes;

in accordance with a determination that the third probability is higher than the fourth probability, selecting the first character and the second character as inputs; and

in accordance with a determination that the fourth probability is higher than the third probability, selecting the third character as an input.

17. The method of claim 13, further comprising:

determining a plurality of aggregate probabilities for a plurality of characters, a first aggregate probability comprising an aggregate probability that the first stroke corresponds to a first character of the plurality of characters and a second aggregate probability comprising an aggregate probability that the first stroke corresponds to a second character of the plurality of characters; and

selecting, from the plurality of characters, a character having a highest aggregate probability as an input.

18. The method of claim 13, wherein:

the first stroke is spatially segregated from the second stroke, and

the first stroke and the second stroke are determined as belonging to the same character based at least in part on the temporal information.

19. The method of claim 13, wherein:

the second stroke spatially overlaps at least a portion of the first stroke, and

the first stroke and the second stroke are determined as belonging to different characters based at least in part on the temporal information.

20. The method of claim 13, wherein detecting the end of the first stroke comprises detecting a pause.

21. The method of claim 13, wherein detecting the end of the first stroke comprises detecting a liftoff of the contact detected on the touch-sensitive surface of the device.

22. The method of claim 13, wherein the temporal information about the first stroke and the second stroke comprises a proximity in time between the first stroke and the second stroke.

23. A non-transitory computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by an electronic device with one or more processors and memory, cause the device to:

receive an indication of a first input that includes movement of a contact detected on a touch-sensitive surface of a device, wherein;

the movement of the contact comprises a first stroke, and

the first stroke has a spatial component and a temporal component;

determine a first probability that the first stroke corresponds to a first character based on the spatial component of the first stroke;

determine a second probability that the first stroke corresponds to the first character based on the temporal component of the first stroke;

determine an aggregate probability that the first stroke corresponds to the first character based on the first probability and the second probability;

detect an end of the first stroke; and

after detecting the end of the first stroke;

detect at least a portion of a second stroke; and

in response to detecting the portion of the second stroke and while the second stroke is still being detected, determine whether the first stroke and the second stroke correspond to a single character based on temporal information about the first stroke and the second stroke.

24. The non-transitory computer readable storage medium of claim 23, wherein the determinations of the first, second and aggregate probabilities are performed while the first stroke is still being detected.

25. The non-transitory computer readable storage medium of claim 23, wherein the determinations of the first, second and aggregate probabilities are repeated periodically while the first stroke is still being detected.

26. The non-transitory computer readable storage medium of claim 23, wherein the first input comprises the first stroke detected during a first time period and the second stroke detected during a second time period, the second stroke having a spatial component and a temporal component, and the one or more programs further comprise instructions that cause the device to:

determine a third probability that the first stroke corresponds to the first character and the second stroke corresponds to a second character based on the temporal and spatial components of the first and second strokes;

determine a fourth probability that a combination of the first stroke and the second stroke corresponds to a third character based on the temporal and spatial components of the combination of the first and second strokes;

in accordance with a determination that the third probability is higher than the fourth probability, select the first character and the second character as inputs; and

in accordance with a determination that the fourth probability is higher than the third probability, select the third character as an input.

27. The non-transitory computer readable storage medium of claim 23, wherein the one or more programs further comprise instructions that cause the device to:

determine a plurality of aggregate probabilities for a plurality of characters, a first aggregate probability comprising an aggregate probability that the first stroke corresponds to a first character of the plurality of characters and a second aggregate probability comprising an aggregate probability that the first stroke corresponds to a second character of the plurality of characters; and

select, from the plurality of characters, a character having a highest aggregate probability as an input.

28. The non-transitory computer readable storage medium of claim 23, wherein:

the second stroke spatially overlaps at least a portion of the first stroke, and

the first stroke and the second stroke are determined as belonging to different characters based at least in part on the temporal information.

29. The non-transitory computer readable storage medium of claim 23, wherein:

the first stroke is spatially segregated from the second stroke, and

the first stroke and the second stroke are determined as belonging to the same character based at least in part on the temporal information.

30. The non-transitory computer readable storage medium of claim 23, wherein detecting the end of the first stroke comprises detecting a pause.

31. The non-transitory computer readable storage medium of claim 23, wherein detecting the end of the first stroke comprises detecting a liftoff of the contact detected on the touch-sensitive surface of the device.

32. The non-transitory computer readable storage medium of claim 23, wherein the temporal information about the first stroke and the second stroke comprises a proximity in time between the first stroke and the second stroke.