INLINE HANDWRITING RECOGNITION AND CORRECTION

• US 20090161958A1
• Filed: 12/21/2007
• Published: 06/25/2009
• Est. Priority Date: 12/21/2007
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: MURAYAMA, TAKANOBU, GARSIDE, ADRIAN JAMES, DZIADOSZ, SUSAN E., MARKIEWICZ, JAN-KRISTIAN, KOTIPALI, KRISHNA
• Granted Patent: US 8,116,569 B2
• Time in Patent Office: Days
• US Class Current: 382/186
• CPC Class Codes: G06F 2203/04806 Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
• CPC Class Codes: G06F 3/0237 using prediction or retrieval techniques
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06V 30/387 using human interaction, e.g. selection of the best displayed recognition candidate

Abstract

As a user writes using a handheld writing device, such as an electronic pen or stylus, handwriting input is received and initially displayed as digital ink. The display of the digital ink is converted to recognized text inline with additional digital ink as the user continues to write. A user may edit a word of recognized text inline with other text by selecting the word. An enlarged version of the word is displayed in a character correction user interface that allows a user to make corrections on an individual character basis and also provides other correction options for the word.

Claims

1. One or more computer-storage media embodying computer-useable instructions for performing a method comprising:

receiving input corresponding with user handwriting from a handheld writing device;

displaying digital ink representing the user handwriting based on the input;

analyzing the input using a recognizer to identify one or more words as recognition text for the digital ink;

employing at least three triggers to determine when to convert display of the digital ink to display of the recognition text, wherein the at least three triggers include a distance-based trigger, a recognition-based trigger, and an overall timer-based trigger;

determining that at least one of the at least three triggers has been satisfied indicating to convert display of the digital ink to the recognition text; and

displaying the recognition text in place of the digital ink.

2. The one or more computer-storage media of claim 1, wherein the distance-based trigger includes a short-distance trigger and a longer-distance trigger, wherein the short-distance trigger is satisfied when the handheld writing device is moved past a first distance for more than a predefined period of time, and wherein the long-distance trigger is satisfied when the handheld writing device is moved past a second distance, the second distance being greater than the first distance.

3. The one or more computer-storage media of claim 2, wherein the first distance is based on an average spacing between words identified from previous user handwriting.

4. The one or more computer-storage media of claim 1, wherein displaying the recognition text in place of the digital ink comprises gradually fading out the digital ink as the recognition text is displayed.

5. The one or more computer-storage media of claim 1, wherein the method further comprises saving the digital ink after the recognition text has been displayed in place of the digital ink and using the digital ink with new digital ink for recognition purposes.

6. The one or more computer-storage media of claim 5, wherein the digital ink is no longer used for recognition purposes when recognition results for the digital ink have not changed for a predetermined period of time.

7. The one or more computer-readable media of claim 5, wherein the digital ink is no longer used for recognition purposes when the one or more words of text corresponding with the digital ink are more than a predetermined number of words prior to the new digital ink.

8. The one or more computer-storage media of claim 1, wherein the method further comprises:

receiving additional input corresponding with additional user handwriting from the handheld writing device, wherein the additional input is received between two words of recognition text;

expanding a space between the two words to provide space for receiving the additional user handwriting;

displaying additional digital ink based on the additional input in the space between the two words;

analyzing the additional input using the recognizer to identify one or more additional words as additional recognition text for the additional digital ink;

employing the at least three triggers to determine when to convert display of the additional digital ink to display of the additional recognition text;

determining that at least one of the at least three triggers has been satisfied indicating to convert display of the additional digital ink to the additional recognition text; and

displaying the additional recognition text in place of the additional digital ink.

9. One or more computer-storage media embodying computer-useable instructions for performing a method comprising:

displaying text including two or more words, the text having been previously converted from digital ink corresponding with handwriting input;

receiving a user selection of a word within the text;

responsive to the user selection, displaying the word in a correction user interface allowing correction of the word on an individual character basis, wherein the correction user interface provides an enlarged display of the word in place of an original display of the word in a manner in which the enlarged display of the word is inline with other words within the text; and

adjusting the display of the other words within the text to provide space for displaying the correction user interface including the enlarged display of the word.

10. The one or more computer-storage media of claim 9, wherein adjusting the display of the other words within the text comprises at least one of resizing at least one of the other words and moving at least one of the other words.

11. The one or more computer-storage media of claim 10, wherein adjusting the display of the other words further comprises providing at least one of a horizontal scroll bar and a vertical scroll bar.

12. The one or more computer-storage media of claim 9, wherein adjusting the display of the other words comprises reducing spacing between the other words.

13. The one or more computer-storage media of claim 9, wherein the method further comprises:

receiving handwriting input at a location corresponding with a character of the word within the correction user interface;

displaying digital ink corresponding with the handwriting input;

analyzing the handwriting input using a recognizer to identify a new character corresponding with the digital ink; and

replacing display of the character with the new character.

14. The one or more computer-storage media of claim 9, wherein displaying the correction user interface includes displaying one or more alternate words for the word, and wherein the method further comprises:

receiving a user selection of one of the one or more alternate words; and

replacing the word with the selected alternate word.

15. The one or more computer-storage media of claim 9, wherein the method further comprises:

receiving a user selection of a character within the word; and

presenting one or more single-character features for editing the selected character.

16. The one or more computer-storage media of claim 9, wherein the method further comprises:

receiving handwriting input at a location corresponding with a character of the word within the correction user interface;

displaying digital ink corresponding with the handwriting input;

analyzing at least the handwriting input using a recognizer to identify an alternate word for the word; and

automatically replacing display of the word with display of the alternate word.

17. One or more computer-storage media embodying computer-useable instructions for performing a method comprising:

receiving handwriting input corresponding with a handheld writing device being moved relative to a display surface;

displaying digital ink at a location on a display corresponding with a location of the display surface at which the handwriting input is received;

analyzing the handwriting input using a recognizer to identify one or more words of text corresponding with the handwriting input;

replacing display of the digital ink with the text as the user continues to provide additional handwriting input using the handheld writing device, wherein the text is displayed inline with additional digital ink corresponding with the additional handwriting input;

receiving input to present a word of the text in a character correction interface, the character correction interface providing an enlarged display of the word in place of an original display of the word to facilitate receiving further handwriting input to correct the word, wherein the character correction interface allows correction of the word on an individual character basis;

displaying the word in the character correction interface by replacing the original display of the word with the enlarged display of the word; and

adjusting display of other words within the text to provide space for the enlarged display of the word.

18. The one or more computer-storage media of claim 17, wherein replacing the display of the digital ink with the text comprises:

monitoring for an occurrence of at least one predetermined event; and

upon detecting the occurrence of a predetermined event, displaying the text in place of the digital ink.

19. The one or more computer-storage media of claim 18, wherein monitoring for the occurrence of at least one predetermined event comprises:

monitoring for movement of the handheld writing device a predetermined distance from the location of the digital ink;

monitoring for stabilization of recognition results for the handwriting input for a predetermined period of time; and

monitoring for further handwriting input within a second predetermined period of time.

20. The one or more computer-storage media of claim 17, wherein the method further comprises:

receiving further handwriting input at a location corresponding with a character of the word within the character correction user interface;

displaying further digital ink corresponding with the further handwriting input;

analyzing at least the further handwriting input using a recognizer to identify an alternate word for the word; and

automatically replacing display of the word with display of the alternate word.