Method and apparatus for transforming sensor signals into graphical images

• US 6,212,296 B1
• Filed: 12/23/1997
• Issued: 04/03/2001
• Est. Priority Date: 12/23/1997
• Status: Expired due to Term

• Current Assignee: Ricoh Company Limited
• Original Assignee: Ricoh Company Limited
• Inventors: Angelo, Michael, Wolff, Gregory J., Stork, David G.
• Primary Examiner(s): MEHTA, BHAVESH M
• Application Number:
  US08/996,537
• Time in Patent Office: 1,197 Days
• Field of Search: 382/119, 382/120, 382/121, 382/122, 382/123, 382/188, 382/187, 345/179, 345/169, 178/19.01, 178/19.03, 178/19.04, 178/19.05
• US Class Current: 382/188
• CPC Class Codes: G06F 3/03545 Pens or stylus
• CPC Class Codes: G06V 30/228 of three-dimensional handwriting, e.g. writing in the air
• CPC Class Codes: G06V 30/333 Preprocessing; Feature extraction
• CPC Class Codes: G06V 30/373 using a special pattern or subpattern alphabet

Abstract

A method and apparatus to precisely track the position of the writing instrument, specifically the writing end, such as the tip, of a pen. One embodiment of the invention described herein includes a writing instrument containing acceleration sensors and angular velocity sensors. Data from these sensors are processed via an Euler transform to determine the strokes made with the writing instrument. Corrections may be made to improve upon the output of the Euler transform. The data describing the strokes may include graphical representations of the strokes actually made with the writing instrument. Data describing the strokes may be stored or processed to accomplish a variety of tasks such as recording information, faxing a message, recording an appointment, or other such tasks.

Claims

1. A method for generating a graphical image of strokes made by a writing instrument, the method comprising:

obtaining data indications of strokes, via a plurality of sensors, made with the writing instrument;

performing a transform on data from the sensors to generate transformed data;

adjusting the transformed data by;

determining a probability that a character made with the writing instrument corresponds each of a first plurality of character templates in a probability table;

selecting a second plurality of character templates, wherein the second plurality of character templates is a subset of the first plurality of character templates;

selecting one character template of the second plurality of templates with a highest probability of corresponding to the character made with the writing instrument;

adjusting the transformed data based on a difference between the character made with the writing instrument and the selected template; and

mapping the adjusted transformed data to a graphical image of the strokes.

2. A method for generating a graphical image of strokes made by a writing instrument, the method comprising:

obtaining data indications of strokes, via a plurality of sensors, made with the writing instrument;

performing a transform on data from the sensors to generate transformed data;

mapping the transformed data to a graphical image of the strokes; and

adjusting the transformed data by;

receiving sensor data corresponding to a tip of the writing instrument being traced over a predetermined pattern;

comparing the sensor data to a known configuration of the predetermined pattern; and

adjusting the transformed data based on a difference between the sensor data and the known configuration.

3. A method for generating a graphical image of strokes made by a writing instrument, the method comprising:

obtaining data indications of strokes, via a plurality of sensors, made with the writing instrument;

performing a transform on data from the sensors to generate transformed data;

mapping the transformed data to a graphical image of the strokes; and

adjusting the transformed data by performing pattern recognition on the transformed data; and

adjusting the transformed data based on results of the pattern recognition such that a graphical representation of the strokes more closely represents a character written with the writing instrument.

4. The method of claim 3, wherein performing pattern recognition further comprises use of Hidden Markov Modeling.

5. The method of claim 3, wherein performing pattern recognition further comprises use of a neural network.

6. A method for generating a graphical image of strokes made by a writing instrument, the method comprising:

obtaining data indications of strokes, via a plurality of sensors, made with the writing instrument;

performing a transform on data from the sensors to generate transformed data;

mapping the transformed data to a graphical image of the strokes; and

adjusting the transformed data by performing a statistical analysis on the transformed data; and

adjusting the transformed data based on results of the statistical analysis such that a graphical representation of the strokes more closely represents a character written with the writing instrument.

7. The method of claim 6, wherein performing statistical analysis further comprises:

analyzing a previous stroke made with the writing instrument;

determining a set of possible subsequent strokes;

comparing an actual subsequent stroke to the set of possible subsequent strokes;

determining one of the set of possible subsequent strokes that corresponds to the actual subsequent stroke; and

adjusting the transformed data based on a difference between the actual subsequent stroke and the corresponding possible subsequent stroke.

8. The method of claim 6, wherein performing statistical analysis further comprises:

analyzing a previous character made with the writing instrument;

determining a set of possible subsequent characters;

comparing an actual subsequent character to the set of possible subsequent characters;

determining one of the set of possible subsequent characters that corresponds to the actual subsequent character; and

adjusting the transformed data based on a difference between the actual subsequent character and the corresponding possible subsequent character.

9. The method of claim 6 further comprising:

obtaining sensor readings for each of a plurality of predefined positions of the writing instrument; and

generating a mapping of sensor outputs to writing instrument positions based on the sensor readings at the plurality of predefined positions.

10. A system for mapping strokes made by a writing instrument to a graphical image, the system comprising:

a writing instrument;

a plurality of sensors comprising accelerometers and gyroscopes that monitor positions of the writing instrument;

means for performing a transform on outputs from the plurality sensors to generate transformed data; and

means for adjusting the transformed data comprising;

means for providing a probability table having a first plurality of character templates;

means for determining a probability that a character made with the writing instrument corresponds each of the character templates in a second plurality of character templates, wherein the second plurality of character templates is a subset of the first plurality of character templates;

means for selecting the character templates with a highest probability;

means for adjusting the transformed data based on a difference between the character made with the writing instrument and the selected character.

11. A system for mapping strokes made by a writing instrument to a graphical image, the system comprising:

a writing instrument;

a plurality of sensors comprising accelerometers and gyroscopes that monitor positions of the writing instrument;

means for performing a transform on outputs from the plurality sensors to generate transformed data; and

means for adjusting the transformed data comprises;

means for providing a probability table having a first plurality of character templates;

means for determining a probability that a character made with the writing instrument corresponds each of the character templates in a second plurality of character templates, wherein the second plurality of character templates is a subset of the first plurality of character templates;

means for selecting the character templates with a highest probability;

means for adjusting the transformed data based on a difference between the character made with the writing instrument and the selected character.

12. A system for mapping strokes made by a writing instrument to a graphical image, the system comprising:

a writing instrument;

a plurality of sensors comprising accelerometers and gyroscopes that monitor positions of the writing instrument;

means for performing a transform on outputs from the plurality sensors to generate transformed data; and

means for adjusting the transformed data comprises;

means for receiving sensor data corresponding to a tip of the writing instrument being traced over a predetermined pattern;

means for comparing the sensor data to a known configuration of the predetermined pattern; and

means for adjusting the transformed data based on a difference between the sensor data and the known configuration.

13. A system for mapping strokes made by a writing instrument to a graphical image, the system comprising:

a writing instrument;

a plurality of sensors comprising accelerometers and gyroscopes that monitor positions of the writing instrument;

means for performing a transform on outputs from the plurality sensors to generate transformed data; and

means for adjusting the transformed data comprises;

means for preparing pattern recognition on the transformed data; and

means for adjusting the transformed data based on results of the pattern recognition.

14. The system of claim 13, wherein the means for performing pattern recognition comprises means for performing Hidden Markov Modeling.

15. The system of claim 13, wherein the means for performing pattern recognition comprises a neural network.

16. The system of claim 13, wherein the means for adjusting the transformed data further comprises:

means for performing a statistical analysis on the transformed data; and

means for adjusting the transformed data based on results of the statistical analysis.

17. The system of claim 16, wherein the means for performing statistical analysis further comprises:

means for analyzing a previous stroke made with the writing instrument;

means for determining a set of possible subsequent strokes;

means for comparing an actual subsequent stroke to the set of possible subsequent strokes;

means for determining one of the set of possible subsequent strokes that corresponds to the actual subsequent stroke; and

means for adjusting the transformed data based on a difference between the actual subsequent stroke and the corresponding possible subsequent stroke.

18. The system of claim 16, wherein the means for performing statistical analysis further comprises:

means for analyzing a previous character made with the writing instrument;

means for determining a set of possible subsequent characters;

means for comparing an actual subsequent character to the set of possible subsequent characters;

means for determining one of the set of possible subsequent characters that corresponds to the actual subsequent character; and

means for adjusting the transformed data based on a difference between the actual subsequent character and the corresponding possible subsequent character.