Handwritten keyboardless entry computer system

• US 5,157,737 A
• Filed: 05/14/1990
• Issued: 10/20/1992
• Est. Priority Date: 07/25/1986
• Status: Expired due to Term

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Grid Systems Corporation
• Inventors: Sklarew, Ralph
• Primary Examiner(s): Boudreau, Leo H.
• Application Number:
  US07/523,447
• Time in Patent Office: 890 Days
• Field of Search: 382/13, 382/59, 178/18
• US Class Current: 382/315
• CPC Class Codes: F02B 2075/025 two
• CPC Class Codes: G06F 3/033 Pointing devices displaced or positioned by the user , e.g. mice, trackballs, pens or joysticks; Accessories therefor digitisers characterised by the transducing means G06F3/041
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06V 30/333 Preprocessing; Feature extraction
• CPC Class Codes: G06V 30/373 using a special pattern or subpattern alphabet
• CPC Class Codes: G09B 5/065 Combinations of audio and video presentations, e.g. videotapes, videodiscs, television systems

Abstract

A keyboardless entry computer system includes a transparent input screen that generates positional information when contacted by a stylus, and a display screen mounted physically below the input screen such that a character that is displayed can be seen below the input screen. The system includes a computer that has been programmed to compile the positional information into Strokes, to calculate Stroke characteristics, and then compare the Stroke characteristics with those stored in a database in order to recognize the symbol drawn by the stylus. Key features of the system are: (1) transparent position sensing sybsystem; (2) underlying display on which to mimic drawing of sensed positions and to show characters or symbols; (3) means to convert sensed positions first into plotted Points and then into recognized characters or symbols; and (4) means to "learn" to associate sensed input positions with a character or symbol.

Claims

1. A computer that includes a system for assigning the identity of one of a plurality of predetermined symbols to a digitized symbol input signal, with the plurality of predetermined symbols including a plurality of computer generated font symbols and a first editing symbol for indicating a change to be made on a group of computer generated font symbols in displayed text, said computer comprising:

a locator device;

a screen for displaying images, including preexisting graphic images, text, and combinations of graphic images and text, generated by said computer and for providing a position signal indicating the position of said locator device over said screen;

means for coincidentally displaying on said screen an image of a handwritten symbol formed by moving said locator device over said screen;

means, coupled to said screen to receive said position signal generated when said handwritten symbol is formed, for converting the received position signal into a first input signal;

means for assigning to said first input signal, the identity of one of

a first of the plurality of computer generated font symbols and

the first editing symbol

means for displaying an image of said first computer generated font symbol on said screen when the first input signal is assigned the identity of said first computer generated font symbol; and

means for changing a first group of computer generated font symbols in text displayed on said screen to a second group of computer generated font symbols assigned to a group of handwritten symbols formed by said locator device over said screen when said first input signal is assigned the identity of the first editing symbol.

16. A computer, as claimed in claim 1, further including storage means for storing, in named files, said pre-existing graphic images, text, and combinations of graphic images and text and means to retrieve said pre-existing graphic images, text, and combinations of graphic images and text by writing the name of the file on the screen using said locator device.

17. A computer, as claimed in claim 1, further including storage means for storing, in named files, said pre-existing graphic images, text, and combinations of graphic images and text and means to retrieve said pre-existing graphic images, text, and combinations of graphic images and text by touching the name of an existing file, displayed on said screen, using said locator device.

2. Apparatus for recognizing handwritten symbols, comprising:

a display screen;

hand-held means, coupled to said display screen, for writing a handwritten symbol over at least a portion of a first area of said display screen;

means, coupled to said display screen, for digitizing said handwritten symbol to provide a digitized symbol;

means, coupled to said display screen and said means for digitizing, for displaying an image of the handwritten symbol on said display screen in at least a portion of said first area as it is written;

means, coupled to said display screen, for executing a plurality of different editing functions;

means, coupled to said means for digitizing, for correlating said digitized symbol with at least one of a plurality of predetermined symbols including a plurality of font symbols and a plurality of editing symbols, by comparing characteristics of said digitized symbol with stored characteristics of at least some of said plurality of predetermined symbols, to provide one of said plurality of predetermined symbols as a designated symbol which is correlated with said handwritten symbol;

means, coupled to said display screen for displaying a computer-generated symbol on said display screen when said designated symbol is a font symbol and for executing at least a first of said plurality of editing functions when said designated symbol is a first of said plurality of editing symbols, and for executing at least a second of said editing functions, different from said first editing function, when said designated symbol is a second of said plurality of editing symbols.

3. Apparatus, as claimed in claim 2, wherein at least said means for displaying a computer-generated symbol and for executing at least a first of said plurality of editing functions and said means for correlating said digitized symbol include an electronic processor.

18. Apparatus, as claimed in claim 2, wherein said plurality of font symbols includes at least one chemical equation symbol.

19. Apparatus, as claimed in claim 2, wherein said plurality of font symbols includes at least one music symbol.

20. Apparatus, as claimed in claim 2, wherein said plurality of font symbols includes at least one symbol for graphic manipulation of data.

21. Apparatus, as claimed in claim 20, wherein said symbol for graphic manipulation of data is a symbol for transfer of graphic data to a spreadsheet.

22. Apparatus, as claimed in claim 2, wherein said plurality of font symbols includes at least one equation.

23. Apparatus, as claimed in claim 2, wherein said plurality of font symbols includes at least one of an ideograph and a scientific symbol.

24. Apparatus, as claimed in claim 2, wherein one of said plurality of editing functions is a margin set function and wherein one of said plurality of editing symbols, corresponding to said margin set function, is a vertical line drawn down the side of displayed text where the new margin should appear.

25. Apparatus, as claimed in claim 2, wherein at least one of said editing functions is a function to move text and wherein one of said plurality of editing symbols, corresponding to said text move function includes brackets placed around a displayed group of symbols, and a caret.

4. A method for recognizing handwritten symbols, comprising:

providing a display screen;

writing a handwritten symbol over at least a portion of a first area of said display screen;

digitizing said handwritten character to provide a digitized symbol;

displaying an image of the handwritten symbol on said display screen in at least a portion of said first area as it is written;

correlating said digitized symbol with at least one of a plurality of a plurality of predetermined symbols including a plurality of font symbols and a plurality of editing symbols, by comparing characteristics of said digitized symbol with stored characteristics of at least some of said plurality of predetermined symbols, to provide one of said plurality of predetermined symbols as a designated symbol which is correlated with said handwritten symbol;

displaying a computer-generated symbol on said display screen when said designated symbol is a font symbol;

executing at least a first of said plurality of editing functions when said designated symbol is a first of said plurality of editing symbols; and

executing at least a second of said plurality of editing functions when said designated symbol is a second of said plurality of editing symbols.

26. Apparatus, as claimed in claim 4, wherein said step of correlating is performed by a computer and further comprising using said computer to learn the editing symbols used by a particular writer.

5. A computer that includes a system for assigning the identity of one of a plurality of predetermined symbols to a digitized symbol input signal, with the plurality of predetermined symbols including computer generated kanji symbols and a first editing symbol for indicating a change to be made on a group of computer generated kanji symbols in displayed text, said computer comprising:

a locator device;

a screen for displaying images, including preexisting graphic images, text, and combinations of graphic images and text, generated by said computer and for providing a position signal indicating the position of said locator device over said screen;

means for coincidentally displaying on said screen an image of a handwritten symbol formed by moving said locator device over said screen;

means, coupled to said screen to receive said position signal generated when said handwritten symbol is formed, for converting the received position signal into a first input signal;

means for assigning to said first input signal, the identity of one of

a first of the plurality of computer generated kanji symbols, and

the first editing symbol

means for displaying an image of said first computer generated kanji symbol on said screen when the first input signal is assigned the identity of said first computer generated kanji symbol; and

means for changing a first group of computer generated kanji symbols in text displayed on said screen to a second group of computer generated kanji symbols assigned to a group of handwritten symbols formed by said locator device over said screen when said first input signal is assigned the identity of the first editing symbol.

6. Apparatus for recognizing symbols and editing text comprising:

a display which includes apparatus for providing output signals, each output signal indicative of a plurality of successive positions on said display selected by a hand-held stylus;

a computer, coupled to said display, configured to receive said signals,

to recognize at least some of said output signals as corresponding to at least one of a plurality of font symbols,

to recognize at least some of said output signals, indicative of positions in a first area of said display as corresponding to at least a first of a plurality of editing symbols,

to recognize at least some of said output signals indicative of positions in said first area of said display as corresponding to at least a second of said plurality of editing symbols,

to control said display causing it to display a computer-generated symbol when said recognized symbol is a font symbol

to execute a first editing function when said recognized symbol is said first editing symbol and

to execute a second editing function different from said first editing functions, when said recognized symbol is said second editing symbol.

27. Apparatus, as claimed in claim 6, wherein said computer is further configured to request further input from said hand-held stylus if at least some of said output signals are not recognized as corresponding to at least one of a plurality of editing symbols.

28. Apparatus, as claimed in claim 6, wherein said computer is further configured to include a database of stroke sequences and corresponding symbols and to add a new symbol to the database when at least some output signals are not recognized as corresponding to at least one of said plurality of editing symbols or font symbols.

7. A method for recognizing symbols and editing text comprising:

providing a computing device which includes a display, said display having apparatus for providing output signals, each output signal indicative of a plurality of successive positions on said display selected by a hand-held stylus;

receiving said signals in said computing device;

using said computing device to recognize at least some of said output said output signals as corresponding to at least some of a plurality of font symbols;

using said computing device to recognize at least some of said output signals, indicative of positions in a first area of said display, as corresponding to at least a first of a plurality of editing symbols;

using said computing device to recognize at least some of said output signals, indicative of positions in said first area of said display as corresponding to at least a second of said plurality of editing symbols;

displaying a predetermined symbol on said display when said recognized symbol is a font symbol;

executing a first editing function when said recognized symbol is said first editing symbol; and

executing a second editing function, different from said first editing function, when said recognized symbol is said second editing symbol.

8. A method for recognizing symbols and editing text comprising:

providing a computing device which includes a display said display having apparatus for providing handwritten symbol output signals, each handwritten symbol output signal indicative of a plurality of successive positions on said display selected by a hand-held stylus;

receiving said signals in said computing device;

using said computing device to recognize at least some of said output signals as corresponding to at least some of a plurality of font symbols;

storing in said computing device, data corresponding to editing function symbols;

using said computing device to compare said handwritten symbol output signals with said data corresponding to editing function symbols and to recognize at least some of the handwritten symbol output signals as corresponding to at least some of a plurality of editing symbols;

displaying a predetermined symbol on said display when said recognized symbol is a font symbol; and

executing an editing function when said recognized symbol is an editing symbol.

29. Apparatus, as claimed in claim 8, wherein said computer is further configured to provide a database of stroke sequences corresponding to characters and to effectively reorder the database to minimize time required to recognize the most frequently identified characters.

9. A method for recognizing handwritten symbols comprising:

providing a display screen;

displaying a plurality of characters or symbols on at least a first area of said display screen;

writing a handwritten symbol over at least a portion of said first area of said display screen;

digitizing said handwritten symbol to provide a digitized symbol;

correlating said digitized symbol with at least one of a plurality of predetermined symbols, including a plurality of editing symbols by comparing characteristics of said digitized symbol with stored characteristics of at least some of said plurality of predetermined symbols, to provide one of said plurality of predetermined symbols as a designated symbol which is correlated with said handwritten symbol;

executing at least a first of said plurality of editing functions when said designated symbol was written over at least one of said displayed characters or symbols; and

executing at least a second of said plurality of editing functions when said designated symbol is one of said plurality of editing symbols.

10. A method for recognizing handwritten computer commands comprising:

providing a computer having at least a display screen and a memory, and capable of executing a plurality of commands to perform operations on data;

storing data in a plurality of memory locations in said memory;

writing a handwritten command symbol over a location on said display screen digitizing said handwritten symbol to provide at least one digitized symbol, and an indication of said location on said display screen;

correlating said at least one digitized symbol with at least one of a plurality of predetermined symbols, by comparing characteristics of said digitized symbol with stored characteristics of at least some of said plurality of predetermined symbols, to provide at least one designated symbol, said at least one designated symbol being correlated with at least one of said plurality of commands to perform operations on data;

selecting at least a first portion of said data, said selecting being based on said location of said handwritten symbol on the display screen;

executing said at least one computer command to perform an operation on said selected data.

11. A method, as claimed in claim 10, wherein said data corresponds to font symbols and wherein font symbols corresponding to at least some of said data are displayed on said display screen and wherein:

said step of selecting at least a first portion of said data includes selecting data corresponding to at least one displayed font symbol which is adjacent to said location of said handwritten symbol on the display screen.

12. A method, as claimed in claim 11, wherein said step of executing at least one computer command includes inserting data in said memory, at least some of said inserted data corresponding to at least a first font symbol, and further comprising:

displaying at least said first font symbol adjacent to said location of said handwritten symbol on the display screen.

13. A method, as claimed in claim 11, wherein said step of executing at least one computer command includes adjusting a margin of said displayed font symbols.

14. A method, as claimed in claim 11, wherein said step of executing at least one computer command includes deleting at least one displayed font symbol.

15. A method, as claimed in claim 11, wherein said step of executing at least one computer command includes moving at least one displayed font symbol.