On-line handwriting recognition using a prototype confusability dialog

• US 5,315,667 A
• Filed: 10/31/1991
• Issued: 05/24/1994
• Est. Priority Date: 10/31/1991
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Leibman, George J., Fujisaki, Tetsunosuke, Tappert, Charles C., Kim, Joonki
• Primary Examiner(s): COUSO, YON JUNG
• Application Number:
  US07/785,735
• Time in Patent Office: 936 Days
• Field of Search: 382/13, 382/7, 382/57, 382/30, 382/34, 340/718, 340/794
• US Class Current: 382/187
• CPC Class Codes: G06F 18/28 Determining representative reference patterns, e.g. by averaging or distorting; Generating dictionaries
• CPC Class Codes: G06V 10/772 Determining representative reference patterns, e.g. averaging or distorting patterns; Generating dictionaries
• CPC Class Codes: G06V 10/987 with the intervention of an operator

Abstract

A handwriting recognition system in which a user, on-line can reduce similarity between different prototypes. The user interactively can delete a prototype or add a new prototype, while at the same time is not allowed to delete a prototype when it is the only prototype for a given character.

Claims

1. In a handwriting recognition system, the combination comprising:

an integrated electronic tablet on which a user may write characters with a stylus and display on which recognized characters may be displayed, said tablet including action button surfaces which when touched by the user with the stylus invoke predetermined actions relative to written characters and displayed recognized characters;

a prototype processor for processing prototype characters written on said tablet by the user in a training session;

a character prototype memory in which said prototype characters processed by said prototype processor are stored;

a character matcher which compares unknown characters written by the user with prototype characters stored in said character prototype memory to determine the best match between the unknown characters and the prototype characters for producing recognized characters;

means for displaying on said display what the user is writing on said tablet;

means for displaying recognized characters from said character matches on said display in response to the user writing on said tablet;

means for determining if any characters in corresponding character positions in what the user has written and is displayed on said display, and what is recognized and displayed on said display as a recognized character are different;

means responsive to said means for determining for displaying the written character and the recognized character at the corresponding character position as an intended character indicative of the displayed written character and an unintended character indicative of the displayed recognized character, respectively;

means for displaying at least one prototype character stored in said character prototype memory for each of the intended character and the unintended character; and

means for removing or replacing intended and unintended character prototypes in said character prototype memory in response to the user touching predetermined ones of the action button surfaces with the stylus in response to viewing the intended character and the unintended character and their respective prototypes on the display.

2. The combination claimed in claim 1, wherein said action button surfaces include for an intended character:

(a) a create button surface which in response to being touched by the user with the stylus includes means for creating a new prototype character for the intended character in response to a new writing of the character by the user;

(b) a keep button surface which in response to being touched by said user with the stylus includes means for for storing the created new prototype of the intended character in said character prototype memory; and

(c) a discard button surface which in response to being touched by said user with the stylus includes means for discarding the unknown character writing on said tablet.

3. The combination claimed in claim 2, wherein the action button surfaces include for the unintended character:

(a) a replace button surface which in response to being touched by the user with the stylus includes means for replacing the prototype for the unintended character with a new prototype produced by the user making a new writing of the corresponding character on said tablet with said stylus, including means for the storing said new writing as a prototype in said character prototype memory; and

(b) a discard button surface which in response to being touch by the user with the stylus includes means for discarding the prototype for the unintended character by removing same from said character prototype memory.

4. The combination claimed in claim 3, including:

means for inhibiting the discarding of the prototype for the unintended character if the prototype for the unintended character is the only prototype for that character.

13. The combination claim in claim 1, including means for displaying indicia on said display which is indicative of an error which caused what is written to be different than what is recognized.

14. The combination claimed in claim 13, wherein said indicia is a sentence explaining the error.

5. In a handwriting recognition system, the combination comprising:

an electronic tablet on which a user may write characters with a stylus;

a display surface on which a plurality of screens can be displayed, and on which the characters written by said user is displayed, and on which what is recognized by a character recognizer as what was written by the user is displayed, said display screens including;

(a) a first screen having a first area where at least one character of what is written is displayed, a second area where said at least one character of what is recognized as being written is displayed, and a third area in which there is at least one action button surface which in response to being touched by said stylus invokes a predetermined action to display a second screen, wherein;

(b) said second screen includes a first area in which said at least one character of what is written is displayed as an intended character, a second area in which prototypes of the intended character is displayed, and a third area which includes a plurality of action button surfaces which in response to being touched by said user with the stylus invokes predetermined actions relative to the intended or the intended character prototypes, and a fourth area in which said at least one character of the recognized writing is displayed as an unintended character, a fifth area in which prototypes of the unintended character is displayed, and sixth area which includes a plurality of action button surfaces which in response to being touched by said stylus invokes predetermined actions relative to said unintended character or the unintended character prototypes; and

means responsive to the touching of any of the action button surfaces to invoke the corresponding predetermined actions.

6. The combination claimed in claim 5, wherein said action button surfaces in said third area of said second screen include for intended characters:

(a) a create button surface which in response to being touched by the user with the stylus includes means for creating a new prototype character for the intended character in response to a new writing of the character by the user;

(b) a keep button surface which in response to being touched by said user with the stylus includes means for storing the created new prototype of the intended character in a character prototype memory; and

(c) a discard button surface which in response to being touched by said user with the stylus includes means for discarding the unknown character writing on said tablet.

7. The combination claimed in claim 6, wherein the action button surfaces in said sixth area of said second screen include for unintended characters:

(a) a replace button surface which in response to being touched by the user with the stylus includes means for replacing the prototype for the unintended character with a new prototype produced by the user making a new writing of the corresponding character on said tablet with said stylus, including means for storing said new writing as a prototype in said character prototype memory; and

(b) a discard button surface which in response to being touched by the user with the stylus includes means for discarding the prototype for the unintended character by removing same from said character prototype memory.

8. The combination claimed in claim 7, including:

means for inhibiting the discarding of the prototype for the unintended character if the prototype for the unintended character is the only prototype for that character.

15. The combination claimed in claim 5, including means for displaying indicia on said display surface which is indicative of an error which caused what is written to be different than what is recognized.

16. The combination claimed in claim 15, wherein said indicia is a sentence explaining the error.

9. In a handwriting recognition system which includes an integrated electronic tablet on which a user may write characters with a stylus and display on which recognized characters may be displayed, said tablet including action button surfaces which when touched by the user with the stylus invoke predetermined actions relative to written characters and displayed recognized characters;

a prototype processor for processing prototype characters written on said tablet by the user in a training session;

a character prototype memory in which said prototype characters processed by said prototype processor are stored;

a character matcher which compares unknown characters written by the user with prototype characters stored in said character prototype memory to determine the best match between the unknown characters and the prototype characters for producing recognized characters, a method comprising the steps of;

displaying on said display what the user is writing on said tablet;

displaying said recognized characters on said display;

determining if any characters in corresponding character positions in what the user has written and what is recognized and displayed on said display as a recognized character are different;

displaying the written character and the recognized character at the corresponding character position as an intended character indicative of the written character and an unintended character indicative of the displayed recognized character;

displaying at least one prototype character stored in said character prototype memory for each of the intended character and the unintended character; and

removing or replacing intended and unintended character prototypes in said character prototype memory in response to the user touching predetermined ones of the action button surfaces with the stylus in response to viewing the intended character and the unintended character and their respective prototypes on the display.

10. The method claimed in claim 9, wherein said action button surfaces include for intended characters:

(a) a create button surface which in response to being touched by the user with the stylus includes means for creating a new prototype character for the intended character in response to a new writing of the character by the user;

(b) a keep button surface which in response to being touched by said user with the stylus includes means for for storing the created new prototype of the intended character in said character prototype memory; and

(c) a discard button surface which in response to being touched by said user with the stylus includes means for discarding the unknown character writing on said tablet.

11. The method claimed in claim 10, wherein the action button surfaces include for the unintended character:

(a) a replace button surface which in response to being touched by the user with the stylus includes means for replacing the prototype for the unintended character with a new prototype produced by the user making a new writing of the corresponding character on said tablet with said stylus, including means for storing said new writing as a prototype in said character prototype memory; and

(b) a discard button surface which in response to being touched by the user with the stylus includes means for discarding the prototype for the unintended character by removing same from said character prototype memory.

12. The method claimed in claim 11, including the steps of:

inhibiting the discarding of the prototype for the unintended character if the prototype for the unintended character is the only prototype for that character.

17. The method of claim 9, including the step of:

displaying indicia on said display which is indicative of an error which caused what is written to be different than what is recognized.

18. The method of claim 17, wherein said indicia is a sentence explaining the error.

19. In a handwriting recognition system, the combination comprising:

an integrated electronic tablet on which a user may write characters with a stylus and display on which recognized characters may be displayed, said tablet including action button surfaces which when touched by the user with the stylus invoke predetermined actions relative to written characters and displayed recognized characters;

a prototype processor for processing prototype characters written on said tablet by the user in a training session;

a character prototype memory in which said prototype characters processed by said prototype processor are stored;

a character matcher which compares unknown characters written by the user with prototype characters stored in said character prototype memory to determine the best match between the unknown characters and the prototype characters for producing recognized characters;

means for displaying on said display what the user is writing on said tablet;

means for displaying recognized characters from said character matches on said display in response to the user writing on said tablet;

means for determining if any characters in corresponding character positions in what the user has written and is displayed on said display, and what is recognized and displayed on said display as a recognized character are different; and

means for displaying indicia on said display which is indicative of an error which caused what is written to be different than what is recognized.

20. The combination claimed in claim 19, wherein said indicia is a sentence explaining the error.