Handwriting user interface for personal digital assistants and the like

• US 20030007018A1
• Filed: 07/09/2001
• Published: 01/09/2003
• Est. Priority Date: 07/09/2001
• Status: Abandoned Application

• Current Assignee: Motorola, Inc. (Motorola Solutions, Inc.)
• Original Assignee: Motorola, Inc. (Motorola Solutions, Inc.)
• Inventors: Ho, Fahfu, Seni, Giovanni
• Application Number:
  US09/901,878
• Publication Number:
  US 20030007018A1
• Time in Patent Office: Days
• US Class Current: 345/864
• CPC Class Codes: G06F 3/0237 using prediction or retrieval techniques
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: G06V 30/1423 the instrument generating sequences of position coordinates corresponding to handwriting preprocessing or recognising digital ink G06V30/32

Abstract

A handheld device (100), a graphical handwriting user interface (“HUI”), a method of interfacing handwritten text and a program product therefor. A lower portion of a touch-enabled display is designated as a handwriting input area (104). Recognized text is displayed at the top of the screen. As each handwritten word is entered (142) into the designated screen input area, a check is made (144) to determine when the handwritten entry is complete by pressing a space key or by a special pen gesture. When the handwritten entry is complete, the handwriting recognition engine matches (146) the handwritten input against words in the system dictionary as supplemented by the user dictionary and a confidence score is attached (148) to the top scoring word.

Claims

1. A hand-held electronic apparatus having a small housing for ease of transport thereof and to contain control circuitry for running different applications therewith, the apparatus comprising:

a screen on the housing having a predetermined size for displaying information to a user;

handwriting recognition circuitry configured for recognizing single and multiple character words handwritten on the predetermined screen area for high writing throughput;

a predetermined area of the screen less than the predetermined screen size on which handwriting is recognized; and

an input device which cooperates with the screen and underlying circuitry for use in inputting handwriting only in the predetermined screen area and selecting application operations displayed on the remainder of the screen to provide the input device with distinct functions based on where the device is used on the screen.

2. The apparatus of claim 1 wherein the handwriting recognition circuitry is configured to display a predetermined number of output words that are ordered by the circuitry based on likelihood of matching the input handwritten word, the output words being displayed in a menu of word choices each time a word is handwritten in the predetermined screen area.

3. The apparatus of claim 1 wherein the handwriting recognition circuitry is configured to display a predetermined number of output words each having an underlying value associated therewith indicative of the probability of recognition accuracy thereof based on the input handwritten word, the output words being ordered from words having highest to least recognition accuracy probabilities.

4. The apparatus of claim 3 wherein the output words include one word having the highest value amongst the displayed output words, and a predetermined threshold recognition level that is compared to a confidence level for said one word such that if the confidence level exceeds the threshold recognition level the one word is used in the application that is active without requiring user intervention, and if the confidence level does not exceed the threshold recognition level user selection is required from amongst the output words for use in the active application.

5. The apparatus of claim 3 wherein the handwriting recognition circuitry includes at least one dictionary database and having a user interface therewith for inputting changes to the database based on low recognition values for handwritten words indicative of the absence of the words from the database.

6. A handwriting recognition user interface (HUI) for a portable device having a touch-enabled input screen, said HUI comprising:

a handwriting input area residing in a portion of a touch-enabled input screen, handwritten words being entered, one at a time using a stylus, recognition results being displayed in said handwritten input area;

a recognition engine capable of recognizing handwritten words; and

a main dictionary, said recognition engine comparing each handwritten input word against words in said main dictionary and providing a probability score indicative of the likelihood that each dictionary word is a correct interpretation of the handwritten input word.

7. A HUI as in claim 6, wherein said handwritten input area is located at a lower portion of said touch enabled screen.

8. A HUI as in claim 7, wherein said handwritten input area occupies less than one third of said touch-enabled screen and spans said touch-enabled screen'"'"'s width.

9. A HUI as in claim 7 wherein the recognition engine is adapted to recognize handwritten entries made in cursive writing.

10. A HUI as in claim 7 wherein the recognition engine is adapted to recognize printed handwritten entries.

11. A HUI as in claim 7 further comprising:

a user dictionary supplementing said main dictionary, words in said user dictionary being matched against said each handwritten input word and assigned a probability score.

12. A HUI as in claim 7 wherein the recognition engine is adapted to recognize stylus entries made in said handwritten input area as handwritten entries and stylus entries made outside of said handwritten input area as pointer function entries.

13. A HUI as in claim 7 further comprising:

a pop-up list of word choices, during word recognition a plurality of highest scoring words are identified as most likely word recognition results, one highest scoring result is designated a primary word choice and any remaining most likely word recognition results are designated secondary word choices.

14. A HUI as in claim 13, wherein the recognition engine is adapted to define a predetermined threshold confidence level so that when said primary word choice has a confidence level above said predetermined threshold, said primary word is automatically loaded into an input buffer for delivery to an active application.

15. A HUI as in claim 7 further comprising one or more action icons on said touch-enabled screen.

16. A HUI as in claim 15 wherein said one or more action icons are displayed together on a side of said touch-enabled screen.

17. A HUI as in claim 15 including software configured so that selecting one of said action icons selects an editing operation selected from the group consisting of:

inserting a space, backspacing, deleting, capitalizing recognition result, and undoing automatic insertion of a last recognition result.

18. A HUI as in claim 17 wherein said recognition engine is configured so that a stylus entry outside of said handwritten input area selects one or more characters of a previously entered word, whereby one or more characters of said previously entered word may be edited.

19. A HUI as in claim 18 further comprising a correction keyboard automatically being displayed upon selection of one or more of said action icons.

20. A HUI as in claim 19 wherein said correction keyboard is displayed in said handwritten input area.

21. A HUI as in claim 20 wherein said correction keyboard includes an add corrected word key, selecting said add corrected key adding an edited word to a user dictionary, said user dictionary supplementing said main dictionary.

22. A HUI as in claim 17, further comprising at least one configuration button icon, selecting said configuration button icon allowing the user to change configuration settings, said configuration settings comprising at least one of:

selecting handwriting style;

propose upper-case at the beginning of a word;

propose punctuation at the end of a word;

number of pop-up list recognition results;

editing button icons location; and

user dictionary maintenance.

24. A PDA as in claim 23, wherein said input area occupies less than one third of said touch-enabled screen and spans said touch-enabled screen'"'"'s width.

25. A PDA as in claim 24 further comprising a user dictionary stored in said storage and supplementing said main dictionary, words in said user dictionary being matched against each said handwritten input word and assigned a probability score.

26. A PDA as in claim 25, wherein said HUI identifies any highest scoring word having confidence level above a predetermined threshold as a primary word and automatically loads said primary word into an input buffer for delivery to an active application.

27. A PDA as in claim 26 wherein said communications port is a wireless communications port, e-mail messages being communicated over said wireless communications port.

28. A PDA as in claim 27, wherein selecting one of said button icons selects an editing operation selected from the group consisting of:

inserting a space, backspacing, deleting, capitalizing recognition result, and undoing automatic insertion of a last recognition result.

29. A PDA as in claim 27 wherein a stylus entry at a previously entered displayed word is recognized as selecting one or more characters of said previously entered displayed word, whereby one or more characters of said selected characters may be edited.

30. A PDA as in claim 29 further comprising an expansion port capable of receiving an expansion keyboard, whereby characters may be entered to correct entered words through a keyboard attached to said expansion port.

31. A PDA as in claim 30 wherein the HUI further comprises:

a correction keyboard automatically being displayed in said designated handwriting input area upon selection of one or more of said button icons.

32. A PDA as in claim 31 wherein said correction keyboard includes an add corrected word key, selecting said add corrected key adding an edited word to a user dictionary, said user dictionary supplementing said main dictionary.

23. A personal digital assistant (PDA) capable of recognizing handwritten words, said PDA comprising:

a touch-enabled input screen;

a recognition engine capable of recognizing handwritten words;

a main dictionary containing a plurality of words;

a communications port for communicating with a remotely connected computer, data being transferred between said remotely connected computer and said PDA;

a local storage storing said main dictionary, application data and applications to be run on said PDA;

a plurality of switches providing manual input to said PDA; and

a handwriting recognition user interface (HUI) comprising;

a designated handwriting input area residing in a lower portion of said touch-enabled input screen, handwritten words being entered a single word at a time using a stylus, recognition results being displayed on said touch enabled screen in said designated handwriting input area, stylus entries made in said designated handwriting area being handwritten entries and stylus entries made outside of said designated handwriting input area being pointer function entries,

a pop-up list listing word candidates, said recognition engine matching each handwritten input word against words in said main dictionary and providing a probability score indicative of the likelihood that each given word is a correct interpretation of the handwritten input word, all words scoring less than a highest scoring word being secondary words, and

one or more action icons displayed together on a side of said touch-enabled screen and providing access to editing functions for editing previously recognized displayed words.

33. A method of providing textual information to a computer, said method comprising the steps of:

a) receiving an entry from a designated handwritten-entry screen area;

b) passing said received entry to a handwriting recognition engine;

c) receiving a probability score from said recognition engine, said probability score indicating a likelihood for a corresponding dictionary word that said corresponding dictionary word matches said received entry; and

d) displaying a list of one or more words in descending order according to said probability score for each displayed word.

34. A method as in claim 33 further comprising the step of:

e) selecting one displayed word as a corresponding to said handwritten input.

35. A method as in claim 34 wherein said handwriting recognition engine matches said entry against words in one or more dictionaries, each word in said one or more dictionaries being assigned a probability score indicative of a likelihood that said scored word is said entry.

36. A method as in claim 35 wherein the step d) of displaying listed words further comprises the steps of:

i) determining a confidence level for a highest scoring of said matched words, any said highest scoring word having a confidence level above a selected threshold level being identified as a primary word;

ii) inserting any identified primary word into an input buffer as a primary word choice; and

iii) inserting a plurality of remaining words in a pop-up list.

37. A method as in claim 36 wherein one of said words displayed in said pop-up list is selected and displayed in place of a previously recognized displayed word.

38. A method as in claim 36 further comprising the steps of:

f) selecting an action icon for editing previously displayed words;

g) displaying a correction keyboard in said handwritten input area; and

h) editing words displayed in said other screen area, one or more characters of each edited word being replaced by characters entered from said correction keyboard.

39. A method as in claim 38 further comprising the step of:

j) storing an edited word in a user dictionary responsive to selection of a key on said correction keyboard.

40. A method of handwriting recognition for an electronic device having circuitry for running different applications, incorporating graphical interface and stylus to allow a user to interact with the application through said graphical interface, the method comprising:

providing a predetermined data entry area on the graphical user interface to receive handwritten data input, one word or character at a time;

allocating a memory buffer for the handwritten data input;

allocating a system input buffer for copying recognition data to be forwarded to an application that is active via the underlying operating system of the device;

recognizing handwritten data as words or characters;

comparing the recognition data after input in the memory buffer with data in one or more electronically stored dictionaries;

calculating recognition probability indices between associated dictionary data entries and the recognition data;

displaying candidates determined from the dictionaries as having a probability of matching the handwritten data input based on the recognition probability calculations;

prompting user intervention when said recognition probability calculations indicate the recognition data does not match a present dictionary entry;

accepting user input correcting inaccurate recognition;

modifying user-defined dictionaries in response to input of new words or characters; and

copying the correct recognition candidate to the system input buffer and forwarding the same to the active application software via the operating system.

41. The method of claim 40 wherein the said handwritten data input can be in the style of cursive, print or a mixture of both.

42. The method of claim 40 wherein said word or character input can be formed from a character string comprised of one or more members from the group consisting of alphanumeric, punctuation, symbols and control characters.

43. The method of claim 40 including editing and expanding the electronically stored user-defined dictionary.

44. The method of claim 40 including copying the recognition candidate with the highest probability to the system input buffer to be forwarded to the underlying active application without user input when said recognition candidate has a confidence level above a predetermined high threshold value.

45. The method of claim 40 including the step of selecting of the number of displayed probable recognition candidates by the user with the graphical interface.

46. The method of claim 45 wherein the probable recognition candidates are displayed in a pop-up selection list, in rank order according to the values of their respective recognition probability indices.

47. The method according to claim 46 wherein the user-selected entry or recognition candidate is copied to the system buffer, deleting the previous entry where one exists, the content of the system buffer to be forwarded to the active application.

49. A computer program product for interfacing handwritten text with a computer as in claim 48 wherein the computer readable program code means for selecting a list of words selects one word as a primary word corresponding to said handwritten input.

48. A computer program product for interfacing handwritten text with a computer, said computer program product comprising a computer usable medium having computer readable program code thereon, said computer readable program code comprising:

computer readable program code means for receiving a handwritten entry;

computer readable program code means for converting said handwritten entry into a character string;

computer readable program code means for storing a plurality of correctly spelled words;

computer readable program code means for generating a probability score for each of said plurality of words, said probability score indicating a likelihood for a corresponding one word of said plurality of words that said corresponding one word matches said handwritten entry; and

computer readable program code means for selecting a list of one or more words for display in descending order according to probability score.

50. A computer program product for interfacing handwritten text with a computer as in clam 49 wherein the computer readable program code means for selecting a list of words further comprises:

computer readable program code means for determining whether a highest scoring word of said selected words has a confidence level exceeding a selected threshold level, any said highest scoring word having a confidence level above said selected confidence level being identified as a primary word;

computer readable program code means for inserting any identified primary word into an input buffer as a primary word choice; and

computer readable program code means for inserting any said primary word and a plurality of remaining words in a pop-up list.

51. A computer program product for interfacing handwritten text with a computer as in claim 50 further comprising computer readable program code means for replacing a previously identified primary word with another one of said words in said pop-up list.

52. A computer program product for interfacing handwritten text with a computer as in claim 51 further comprising:

computer readable program code means for selecting previously displayed words for editing;

computer readable program code means for selecting a correction keyboard; and

computer readable program code means for replacing one or more characters of each edited word with characters entered from said correction keyboard.

53. A computer program product for interfacing handwritten text with a computer as in claim 52 further comprising:

computer readable program code means for storing an edited word in a user dictionary responsive to selection of a key on said correction keyboard.