Integrated platform for user input of digital ink

• US 8,315,482 B2
• Filed: 06/26/2007
• Issued: 11/20/2012
• Est. Priority Date: 06/26/2007
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Hou, Xiaohui, Qiu, Yingjun, Zhang, Dongmei, Wang, Jian
• Primary Examiner(s): Patel, Jayesh A
• Application Number:
  US11/821,870
• Publication Number:
  US 20090002392A1
• Time in Patent Office: 1,974 Days
• Field of Search: 382/305, 382185-187, 382/290, 382/101
• US Class Current: 382/305
• CPC Class Codes: G06F 40/171 by use of digital ink
• CPC Class Codes: G06V 30/36 Matching; Classification

Abstract

Described is a technology that provides an integrated platform for users to use different kinds of digital ink (e.g., handwritten characters, sketched shapes, handwritten formulas) when interacting with computer programs. The platform interprets the user'"'"'s digital ink input and outputs one or more associated items into an application program. The output items can be customized for different application programs. In one aspect, the platform includes an ink panel having different operating modes for receiving digital ink, and a recognition service that recognizes different types of digital ink. The recognition service may include a unified recognizer that recognizes different types of digital ink, e.g., characters and shapes. Another recognizer may be included such as an equation recognizer. If the recognition result is text while in a non-text mode, the text may be used in a keyword search to locate items; otherwise, the recognition result may be used without keyword searching.

Claims

1. A computer-readable storage device having computer-executable instructions which, when executed by a computing device, cause the computing device to perform:

receiving digital ink from a user as input to an input panel having user-selectable modes that implement different input functions, the user-selectable modes comprising at least a text input mode and a shape input mode that is different than the text input mode;

providing the digital ink to a recognition service that is capable of recognizing the digital ink;

receiving a recognition result from the recognition service, the recognition result comprising recognized text or a recognized non-textual shape;

in a first instance when the user has placed the input panel into the text input mode before the digital ink is received and the recognition result comprises the recognized text, providing the recognized text to a text processing application;

in a second instance when the user has placed the input panel into the shape input mode before the digital ink is received and the recognition result comprises the recognized non-textual shape, providing the recognized non-textual shape to a non-textual shape processing application that is different from the text processing application; and

in a third instance when the user has placed the input panel into the shape input mode before the digital ink is received and the recognition result comprises the recognized text, providing the recognized text as a keyword to the non-textual shape processing application to use in a keyword search to locate another non-textual shape that is related to the keyword.

2. The computer-readable storage device of claim 1, further comprising instructions which, when executed by the computing device, cause the computing device to perform:

providing a visible representation of the recognized non-textual shape to an output area of a results panel.

3. The computer-readable storage device according to claim 1, further comprising instructions which, when executed by the computing device, cause the computing device to perform:

selecting an individual recognizer from a plurality of recognizers associated with the recognition service, a first one of the plurality of recognizers being selected when the input panel is in the text input mode and a second, different one of the plurality of recognizers being selected when the input panel is in an equation input mode.

4. A method comprising:

displaying an input panel having user-selectable modes, the user-selectable modes comprising at least a text input mode and a shape input mode, wherein;

at least the text input mode and the shape input mode are displayed concurrently on the input panel, and

the user selects from the user-selectable modes displayed on the input panel before entering digital ink to the input panel;

receiving the digital ink as input to the input panel after the user selects from the user-selectable modes displayed on the input panel;

providing the digital ink to a recognition service that recognizes the digital ink;

receiving a recognition result from the recognition service, the recognition result comprising recognized text or a recognized non-textual shape;

determining whether the user has selected the text input mode displayed on the input panel or the shape input mode displayed on the input panel;

in a first instance, responsive to determining that the user has selected the text input mode displayed on the input panel, providing the recognized text to a text processing application, wherein the recognized text corresponds to one or more handwritten characters entered by the user via the digital ink while the input panel is in the text input mode; and

in a second instance, responsive to determining that the user has selected the shape input mode displayed on the input panel, providing the recognized non-textual shape to a non-textual shape processing application that is different from the text processing application, wherein the recognized non-textual shape corresponds to a sketched shape that is entered by the user via the digital ink while the input panel is in the shape input mode, and

wherein at least the providing the recognized text to the text processing application and the providing the recognized non-textual shape to the non-textual shape processing application is performed by a computing device, and

wherein, in the second instance, the recognition result comprises an individual value that is assigned to the recognized non-textual shape.

5. The method according to claim 4, the user-selectable modes displayed on the input panel further comprising an equation input mode.

6. The method according to claim 5, further comprising:

in a third instance when the user selects the equation input mode displayed on the input panel, providing a math equation recognized by the recognition service to the text processing application.

7. The method according to claim 4, further comprising assigning Unicode values to a plurality of non-textual shapes, the Unicode values including the individual value that is assigned to the recognized non-textual shape.

8. The method according to claim 4, wherein, in the second instance, the individual value is a Unicode value that is assigned to the recognized non-textual shape.

9. A system comprising:

a platform configured to;

provide an input panel having user-selectable modes comprising at least a text input mode and a shape input mode, wherein a user selects from the user-selectable modes before entering digital ink to the input panel;

receive the digital ink as input to the input panel after the user selects from the user-selectable modes;

provide the digital ink to a recognition service that recognizes the digital ink;

receive a recognition result from the recognition service, the recognition result comprising recognized text or a recognized non-textual shape;

in a first instance when the user has placed the input panel into the text input mode before entering the digital ink and the recognition result comprises the recognized text, provide the recognized text to a text processing application;

in a second instance when the user has placed the input panel into the shape input mode before entering the digital ink and the recognition result comprises the recognized non-textual shape, provide the recognized non-textual shape to a non-textual shape processing application that is different from the text processing application; and

in a third instance when the user has placed the input panel into the shape input mode before the digital ink is received and the recognition result comprises the recognized text, provide the recognized text as a keyword to the non-textual shape processing application to use in a keyword search to locate another non-textual shape that is related to the keyword; and

at least one processing device configured to execute the platform.

10. The system according to claim 9, wherein the input panel comprises a topmost application that does not receive focus.

11. The system according to claim 9, wherein, in the first instance, the text processing application receives focus and the input panel does not receive focus.

12. The system according to claim 9, wherein, in the second instance, the non-textual shape processing application receives focus and the input panel does not receive focus.

13. The system according to claim 9, wherein the platform is further configured to, in the first instance, delay providing the recognized text to the text processing application and, during the delay, allow the user to cancel providing the recognized text to the text processing application in event of an error.

14. The system according to claim 9, wherein, in the first instance, the recognized text comprises Chinese characters.

15. The system according to claim 9, wherein, in the third instance, the recognized text comprises one or more Chinese characters that are provided as the recognized text for the keyword search to locate the another non-textual shape that is related to the one or more Chinese characters.

16. The system according to claim 9, wherein the platform is further configured to display the another non-textual shape and allow the user to drag and drop the another non-textual shape into the non-textual shape processing application.

17. The system according to claim 9, wherein the text processing application and the non-textual shape processing application are different applications.

18. The system according to claim 9, wherein the platform is further configured to, in a fourth instance when the user has placed the input panel into an equation mode, provide the recognized non-textual shape to an equation recognizer that recognizes at least the following symbols from the digital ink entered by the user:

“

a²”

;

“

b²”

;

“

+”

; and

“

{square root over ( )}.”