Inline handwriting recognition and correction

• US 8,116,569 B2
• Filed: 12/21/2007
• Issued: 02/14/2012
• Est. Priority Date: 12/21/2007
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Markiewicz, Jan-Kristian, Kotipali, Krishna, Garside, Adrian James, Murayama, Takanobu, Dziadosz, Susan E.
• Primary Examiner(s): Le, Vu
• Assistant Examiner(s): VANCHY JR, MICHAEL J
• Application Number:
  US11/962,438
• Publication Number:
  US 20090161958A1
• Time in Patent Office: 1,516 Days
• Field of Search: 382/186, 382/187, 382/199, 345/179, 345/467
• US Class Current: 382/186
• CPC Class Codes: G06F 2203/04806 Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
• CPC Class Codes: G06F 3/0237 using prediction or retrieval techniques
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06V 30/387 using human interaction, e.g. selection of the best displayed recognition candidate

Abstract

As a user writes using a handheld writing device, such as an electronic pen or stylus, handwriting input is received and initially displayed as digital ink. The display of the digital ink is converted to recognized text inline with additional digital ink as the user continues to write. A user may edit a word of recognized text inline with other text by selecting the word. An enlarged version of the word is displayed in a character correction user interface that allows a user to make corrections on an individual character basis and also provides other correction options for the word.

Claims

1. One or more computer-storage media embodying computer-useable instructions for performing a method comprising:

receiving input corresponding with user handwriting from a handheld writing device;

displaying digital ink representing the user handwriting based on the input;

analyzing the input using a recognizer to identify one or more words as recognition text for the digital ink;

employing at least three triggers to determine when to convert display of the digital ink to display of the recognition text, wherein the at least three triggers include a distance-based trigger, a recognition-based trigger, and an overall timer-based trigger;

determining that at least one of the at least three triggers has been satisfied indicating to convert display of the digital ink to the recognition text;

displaying the recognition text in place of the digital ink;

saving the digital ink after the recognition text has been displayed in place of the digital ink; and

using the digital ink with new digital ink for recognition purposes, wherein the digital ink is no longer used for recognition purposes when the one or more words of recognition text corresponding with the digital ink are more than a predetermined number of words prior to the new digital ink.

2. The one or more computer-storage media of claim 1, wherein the distance-based trigger includes a short-distance trigger and a longer-distance trigger, wherein the short-distance trigger is satisfied when the handheld writing device is moved past a first distance for more than a predefined period of time, and wherein the long-distance trigger is satisfied when the handheld writing device is moved past a second distance, the second distance being greater than the first distance.

3. The one or more computer-storage media of claim 2, wherein the first distance is based on an average spacing between words identified from previous user handwriting.

4. The one or more computer-storage media of claim 1, wherein displaying the recognition text in place of the digital ink comprises gradually fading out the digital ink as the recognition text is displayed.

5. The one or more computer-storage media of claim 1, wherein the digital ink is no longer used for recognition purposes when recognition results for the digital ink have not changed for a predetermined period of time.

6. The one or more computer-storage media of claim 1, wherein the method further comprises:

receiving additional input corresponding with additional user handwriting from the handheld writing device, wherein the additional input is received between two words of recognition text;

expanding a space between the two words to provide space for receiving the additional user handwriting;

displaying additional digital ink based on the additional input in the space between the two words;

analyzing the additional input using the recognizer to identify one or more additional words as additional recognition text for the additional digital ink;

employing the at least three triggers to determine when to convert display of the additional digital ink to display of the additional recognition text;

determining that at least one of the at least three triggers has been satisfied indicating to convert display of the additional digital ink to the additional recognition text; and

displaying the additional recognition text in place of the additional digital ink.

7. One or more computer-storage media embodying computer-useable instructions for performing a method comprising:

receiving handwriting input corresponding with a handheld writing device being moved relative to a display surface;

displaying digital ink at a location on a display corresponding with a location of the display surface at which the handwriting input is received;

analyzing the handwriting input using a recognizer to identify one or more words of text corresponding with the handwriting input;

replacing display of the digital ink with the text as the user continues to provide additional handwriting input using the handheld writing device, wherein the text is displayed inline with additional digital ink corresponding with the additional handwriting input;

saving the digital ink after the text has been displayed in place of the digital ink; and

using the digital ink with new digital ink for recognition purposes, wherein the digital ink is no longer used for recognition purposes when the one or more words of text corresponding with the digital ink are more than a predetermined number of words prior to the new digital ink;

receiving input to present a word of the text in a character correction interface, the character correction interface providing an enlarged display of the word in place of an original display of the word to facilitate receiving further handwriting input to correct the word, wherein the character correction interface allows correction of the word on an individual character basis;

displaying the word in the character correction interface by replacing the original display of the word with the enlarged display of the word; and

adjusting display of other words within the text to provide space for the enlarged display of the word.

8. The one or more computer-storage media of claim 7, wherein replacing the display of the digital ink with the text comprises:

monitoring for an occurrence of at least one predetermined event; and

upon detecting the occurrence of a predetermined event, displaying the text in place of the digital ink.

9. The one or more computer-storage media of claim 8, wherein monitoring for the occurrence of at least one predetermined event comprises:

monitoring for movement of the handheld writing device a predetermined distance from the location of the digital ink;

monitoring for stabilization of recognition results for the handwriting input for a predetermined period of time; and

monitoring for further handwriting input within a second predetermined period of time.

10. The one or more computer-storage media of claim 7, wherein the method further comprises:

receiving further handwriting input at a location corresponding with a character of the word within the character correction user interface;

displaying further digital ink corresponding with the further handwriting input;

analyzing at least the further handwriting input using a recognizer to identify an alternate word for the word; and

automatically replacing display of the word with display of the alternate word.