In-place adaptive handwriting input method and system

• US 20030071850A1
• Filed: 10/12/2001
• Published: 04/17/2003
• Est. Priority Date: 10/12/2001
• Status: Abandoned Application

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Geidl, Erik M.
• Application Number:
  US09/976,188
• Publication Number:
  US 20030071850A1
• Time in Patent Office: Days
• US Class Current: 345/781
• CPC Class Codes: G06F 2203/04804 Transparency, e.g. transparent or translucent windows
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06V 30/1423 the instrument generating sequences of position coordinates corresponding to handwriting preprocessing or recognising digital ink G06V30/32

Abstract

A system and method that displays a semi-transparent user input interface relative to an application'"'"'s currently focused input field at times when handwritten input is appropriate. The semi-transparent user interface starts when a program'"'"'s text input field receives focus, can grow as needed to receive input, or will disappear when not used for a time. Handwritten data is recognized and passed to the application as if it was typed in the focused field, and the application need not be aware of handwriting, as the system and method are external to the application. Pen events that are not handwriting, but comprise gestures directed to the program through the semi-transparent input user interface, are detected by a gesture detection engine and sent to the application. A user is thus guided to enter handwriting, while handwriting recognition appears to be built into applications, whether or not those applications are aware of handwriting.

Claims

1. In a computing device having an executing program, a method comprising:

evaluating a program field that has focus against information indicative of whether the field is configured to receive text input; and

if the field is configured to receive text input;

1) providing a visible user input interface at a displayed location relative to the field;

2) receiving handwritten data at the input interface;

3) providing the handwritten data to a recognition engine; and

4) returning a recognition result to the program.

2. The method of claim 1 wherein the visible user input interface is semi-transparent.

3. The method of claim 1 wherein the handwritten data received at the input interface is evaluated to determine whether the handwritten data corresponds to a gesture.

4. The method of claim 3 wherein the handwritten data corresponds to a gesture, and further comprising, providing at least one pen event corresponding to the gesture to the program.

5. The method of claim 4 wherein the visible user input interface is semi-transparent, and wherein the gesture comprises user input directed to an area of the program that is visible through the semi-transparent user interface.

6. The method of claim 1 wherein providing the handwritten data to a recognition engine is performed in response to detection of a submit button associated with the visible user interface.

7. The method of claim 1 wherein providing the handwritten data to a recognition engine is performed in response to a time being achieved.

8. The method of claim 1 wherein providing the handwritten data to a recognition engine is performed in response to a gesture being detected.

9. The method of claim 1 wherein evaluating the program field that has focus comprises evaluating at least one window attribute corresponding to the field.

10. The method of claim 9 wherein evaluating at least one window attribute corresponding to the field comprises accessing window class information.

11. The method of claim 1 further comprising, accessing a database to obtain the information indicative of whether the field is configured to receive text input.

12. The method of claim 1 further comprising, adjusting the appearance of the visible input window.

13. The method of claim 12 wherein adjusting the appearance of the visible input window comprises increasing its size to enable entry of additional handwritten data.

14. The method of claim 1 further comprising, erasing the visible input window.

15. The method of claim 14 wherein the visible input window is erased in response to receiving a close request.

16. The method of claim 14 wherein the visible input window is erased in response to a time being achieved.

17. The method of claim 14 wherein the visible input window is erased in response to a gesture being detected.

18. In a computing device having a program, a system comprising:

user input interface code;

a field typing engine configured to evaluate a field of the program, determine if that field is supported by the user input interface code, and if so, to communicate information to the user input interface code;

the user input interface code drawing a visible input area to indicate that data may be entered therein, the drawing of the visible input area based on the information received from the field typing engine; and

a recognition engine that receives entered data from the user input interface code and converts the entered data to a recognition result that is made available to the program by the user input interface.

19. The system of claim 18, wherein the visible input area is semi-transparent.

20. The system of claim 18, wherein the field typing engine evaluates at least one window attribute corresponding to the field against hard-coded or retrieved information to determine whether the field is supported.

21. The system of claim 18 wherein the entered data comprises handwritten data, and further comprising a gesture detection engine that evaluates the handwritten data to determine whether the handwritten data corresponds to a gesture, and if so, to provide least one event to the program.

22. The system of claim 21 wherein the visible user input interface is semi-transparent, and wherein the gesture comprises user input directed to an area of the program that is visible through the semi-transparent user interface.

23. The system of claim 18 wherein the entered data comprises handwritten data, and further comprising a rulebase that determines an appearance of the visible input area including a displayed size thereof.

24. The system of claim 23 wherein the rulebase increases the displayed size of the visible input area based on handwritten data approaching an end thereof.

25. The system of claim 18 wherein the visible input area has at least one button associated therewith for receiving a command.

26. The system of claim 25 wherein at least one button comprises a submit button associated with the visible user interface, activation of the submit button commanding the user input interface code to communicate the entered data to the recognition engine.

27. The system of claim 18 wherein the user input interface code provides the recognition result to the program in a message queue associated with the program.

28. The system of claim 18 wherein the drawing of the visible input area positions the visible input area relative to the field based on the information received from the field typing engine.

29. The system of claim 18 wherein the drawing of the visible input area sizes the visible input area based on the information received from the field typing engine.

30. In a computer system having a graphical user interface, a system comprising,

an application program having at least one application input area into which user input data can be entered;

user interface code external to the application program;

a typing engine that determines whether to call the user interface code for a selected application input area of the application program based on attribute information associated with that application input area, the user interface code providing a semi-transparent input area based on the attribute information when called;

a timing mechanism configured to cause removal of the semi-transparent input area when no user interaction with the visible input area is detected for a period of time;

a gesture engine, the gesture engine invoked to determine whether user input data directed to the semi-transparent input area is a gesture directed to the application program or information that should be recognized as text; and

a handwriting recognition engine, the handwriting recognition engine configured to receive the information that the gesture engine has decided should be recognized as text, the handwriting recognition engine responding by returning recognized text when provided with the information.

31. The system of claim 30 wherein the recognized text is received by the user interface code and made available to the application program.

32. The system of claim 30 wherein the application program displays the recognized text in the application input area.

33. The system of claim 30 further comprising a growth rulebase, the growth rulebase determining whether to alter an appearance of the semi-transparent input area in response to the information received therein.