Handwritten keyboardless entry computer system

• US 4,972,496 A
• Filed: 03/24/1987
• Issued: 11/20/1990
• Est. Priority Date: 07/25/1986
• Status: Expired due to Term

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Grid Systems Corporation
• Inventors: Sklarew, Ralph
• Primary Examiner(s): Boudreau, Leo H.
• Application Number:
  US07/029,772
• Time in Patent Office: 1,337 Days
• Field of Search: 382/13, 382/24, 382/57, 382/14, 340/706, 340/707, 340/709, 178/18, 178/19, 178/20
• US Class Current: 382/187
• CPC Class Codes: F02B 2075/025 two
• CPC Class Codes: G06F 3/033 Pointing devices displaced or positioned by the user , e.g. mice, trackballs, pens or joysticks; Accessories therefor digitisers characterised by the transducing means G06F3/041
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06V 30/1423 the instrument generating sequences of position coordinates corresponding to handwriting preprocessing or recognising digital ink G06V30/32
• CPC Class Codes: G06V 30/333 Preprocessing; Feature extraction
• CPC Class Codes: G06V 30/373 using a special pattern or subpattern alphabet
• CPC Class Codes: G09B 5/062 Combinations of audio and printed presentations, e.g. magnetically striped cards, talking books, magnetic tapes with printed texts thereon
• CPC Class Codes: G09B 5/065 Combinations of audio and video presentations, e.g. videotapes, videodiscs, television systems
• CPC Class Codes: Y10S 345/901 Electronic book with display

Abstract

A keyboardless entry computer system includes a transparent input screen that generates positional information when contacted by a stylus, and a display screen mounted physically below the input screen such that a character that is displayed can be seen below the input screen. The system includes a computer that has been programmed to compile the positional information into Strokes, to calculate Stroke characteristics, and then compare the Stroke characteristics with those stored in a database in order to recognize the symbol drawn by the stylus. Key features of the system are: (1) transparent position sensing subsystem; (2) underlying display on which to mimic drawing of sensed positions and to show characters or symbols; (3) means to convert sensed positions first into plotted Points and then into recognized characters or symbols; and (4) means to "learn" to associate sensed input positions with a character or symbol.

Claims

1. A microprocessor-based process for recognition, translation and display of Handwritten Symbols and execution of Editing Functions and other commands comprising the steps of:

(a) Creating a personalized database for each individual user of the process by having the user initially input a Handwritten Symbol for each character corresponding with a Font Symbol to be displayed or Editing Function or other command to be executed;

(b) Determining a unique set of Stroke characteristics to characterize each such symbol and storing it in the database;

(c) Writing or drawing Handwritten Symbols on a computer screen with a stylus;

(d) Digitizing each Handwritten Symbol to identify the x, y coordinates of a multiplicity of Points into Strokes defining the Handwritten Symbol;

(e) Processing the digitized characteristics of each Stroke of a Handwritten Symbol to determine the predetermined characteristics of the Handwritten Symbol;

(f) Each of steps (b) and (e) comprising a determination of the (i) length of the Stroke, (ii) average slope of the Stroke, (iii) the Stroke centroid'"'"'s height above base line, (iv) rate of change in slope of Stroke and (v) comparison of the location of the centroid of each Stroke with the centroid of the Handwritten Symbol;

(g) Searching the data base to find a "hit," a perfect or the closest correspondence to the characteristics of a Font Symbol;

(h) Displaying the Font Symbol or performing the Editing Function or other command most closely associated with the "hit" characteristics.

2. A process as set forth in claim 1 including the additional steps of:

(a) Determining if the Font Symbol displayed or command performed is an error; and

(b) If there is an error, re-inputting a Handwritten Symbol for the desired Font Symbol or command to modify the database.

3. A process as set forth in claim 1, including the additional steps of:

(a) Substantially contemporaneously displaying the Handwritten Symbol on the screen as it is being written or drawn.

4. A process as set forth in claim 1, wherein step (g) comprises:

(a) Determining the value of each of the five characteristics;

(b) Subtracting the database value of the characteristic from the newly-measured value;

(c) Scaling the absolute values of each measurement to make each of the five measurements reasonably equivalent to the others;

(d) Adding all five values;

(e) If a predetermined threshold is exceeded, discarding the comparison and creating an error message; and

(f) Searching the database to find a numerical "hit," or, if not feasible, the closest numerical match between the database value and the newly inputted value.

5. A process as set forth in claim 4, wherein the program for step (f) requires less than 6K of machine memory.

6. A process as set forth in claim 1, wherein said Handwritten Symbols include Editing Symbols and said process includes recognition of the Editing Symbols drawn on the screen and executing the Editing Functions represented.

7. A process as set forth in claim 1, in which text in Font Symbol format is displayed on the screen, and including the steps of:

(a) Creating a window on the screen proximate to, but not overlying, an area of said text to be edited;

(b) Inputting and displaying said Handwritten Symbols in said window;

(c) Displaying the Font Symbols corresponding to said Handwritten Symbols in close proximity to said Handwritten Symbols.

8. A process as set forth in claim 1, including the steps of:

(a) Creating one or more Softkeys on said screen to cause the microprocessor to perform operational functions; and

(b) Touching one or more of said Softkeys to execute the corresponding functions.

9. A process as set forth in claim 1, including the steps of determining the x,y coordinates of from about 100 to 200 Points per inch and from about 100 to 200 points per second in characterizing said Handwritten Symbols.

10. A process as set forth in claim 1 wherein the end of a Stroke is determined by the detection of a change in angle or direction exceeding a maximum set threshold.

11. Apparatus for recognizing Handwritten Symbols and respectively displaying Handwritten and Font Symbols and executing Editing Functions or other commands on a screen comprising:

(a) A visual display screen having the capability to display Font Symbols and means to execute Editing Functions and other commands;

(b) Hand-held means to write or draw Handwritten Symbols on or over said screen;

(c) Means to display a true representation of the Handwritten Symbols on said screen as they are created;

(d) Digitizing means to sense the position of said hand-held means and to convert the same into a series of electrical signals defining position, size and shape of each Handwritten Symbol;

(e) Means to compare predetermined characteristics of each digitized Handwritten Symbol with a database of predetermined characteristics of Font Symbols and Editing Functions to assess the identity of each Handwritten Symbol;

(f) Means to (i) convert each Handwritten Symbol to an Editing Function or other command or a predetermined Font Symbol and (ii) execute the Editing Function or other command or display the Font Symbol on the screen in close proximity to the screen area in which the corresponding Handwritten Symbol was originally entered, as the case may be;

(g) Means to display predetermined combinations of Font Symbols in the form of text or forms on said screen, whereby Handwritten Symbols can be used to input information to complete a form, to edit a predetermined text, or both, as the case may be; and

(h) Means operative upon execution of a predetermined command to create a window on said screen, free of text or other information, for entering Handwritten Symbols.

12. Apparatus as set forth in claim 11, wherein said apparatus comprises a portable unit having dimensions up to about 16"×

16"×

4", a weight of up to about 15 pounds and is adapted to include a self-contained power supply.

13. Apparatus as set forth in claim 11, including means to display predetermined Font Symbols including Editing Symbols on said screen, so as to permit Handwritten Symbols representing each predetermined Font Symbol and Editing Symbol to be manually defined on the screen and means to identify the manually defined Handwritten Symbol with the corresponding predetermined Font Symbol in said database.

14. Apparatus as set forth in claim 13, including means to modify said database when a Handwritten Symbol causes an erroneous Font Symbol to be displayed on the screen or command to be performed, as the case may be.

15. Apparatus as set forth in claim 11, including areas on said screen sensitive to touch to cause predetermined functions to be performed.

16. Apparatus as set forth in claim 11, wherein said screen is substantially flat and adapted to be used in a substantially horizontal position.

17. Apparatus as set forth in claim 11, including Softkeys on said screen and means to execute operational commands in response to touching any of said Softkeys.

18. An apparatus as claimed in claim 11 wherein said command means (h) is a caret.

19. An apparatus as claimed in claim 11, wherein said command means (h) is the writing of one or more horizontal or vertical lines through displayed Font Symbols on said screen.

20. An apparatus as claimed in claim 11, wherein said window can be of various sizes and shapes.

21. An apparatus as claimed in claim 11, wherein said window can be used to insert graphic images into a document as well as Handwritten Symbols.