Handwritting input apparatus for handwritting recognition using more than one sensing technique

• US 5,517,579 A
• Filed: 04/13/1994
• Issued: 05/14/1996
• Est. Priority Date: 02/04/1994
• Status: Expired due to Fees

• Current Assignee: Invibro Technologies Limited
• Original Assignee: Baron Company Limited
• Inventors: Korsensky, Victor, Bar-Itzhak, Zeev, Prishvin, Alexander, Baron, Ehud
• Primary Examiner(s): Boudreau, Leo
• Assistant Examiner(s): Johns, Andrew W.
• Application Number:
  US08/227,275
• Time in Patent Office: 762 Days
• Field of Search: 382/3, 382/13, 382/24, 382/25, 382/29, 382/30, 382/59, 382/120, 382/122, 382/123, 382/187, 382/188, 382/202, 382/203, 382/207, 382/209, 382/313, 382/314, 364/705.03, 364/709.11, 178/18
• US Class Current: 382/187
• CPC Class Codes: G06V 30/1423 the instrument generating sequences of position coordinates corresponding to handwriting preprocessing or recognising digital ink G06V30/32
• CPC Class Codes: G06V 30/36 Matching; Classification
• CPC Class Codes: G06V 30/373 using a special pattern or subpattern alphabet

Abstract

Handwriting recognition apparatus including handwriting input apparatus employing at least two different sensing techniques to sense handwriting and symbol identification apparatus receiving an output of the handwriting input apparatus for providing an output indication of symbols represented by the handwriting.

Claims

1. Handwriting recognition apparatus comprising:

handwriting input apparatus employing at least two different sensing techniques to sense handwriting and including at least one accelerometer located in a hand held writing implement other than at a tip thereof; and

symbol identification apparatus receiving an output of the handwriting input apparatus for providing an output indication of symbols represented by the handwriting, and wherein said symbol identification apparatus includes a first identification channel for employing an input from said handwriting input apparatus to provide a first output representing identification of symbols, a second identification channel for employing an input from said at least one accelerometer to provide a second output representing identification of symbols, and symbol selection apparatus for selecting between the first and the second outputs based on predetermined criteria.

2. Handwriting recognition apparatus according to claim 1 and wherein said handwriting input apparatus comprises a position digitizer.

3. Handwriting recognition apparatus according to claim 2 and wherein said at least one accelerometer comprises a plurality of mutually orthogonally disposed accelerometers.

4. Handwriting recognition apparatus according claim 2 and wherein said symbol identification apparatus includes combined position and acceleration processing apparatus which combines the inputs from said digitizer and said at least one accelerometer in identifying symbols.

5. Handwriting recognition apparatus according to claim 1 and wherein said handwriting input apparatus comprises ultrasonic position determination apparatus operative in air.

6. Handwriting recognition apparatus according to claim 5 and wherein said handwriting input apparatus is operative to write on a computer screen.

7. Apparatus for detecting pen-surface contact comprising:

at least one accelerometer located in a pen other than at a tip thereof measuring at least one component of acceleration of a user'"'"'s hand manipulating the pen; and

a pen-surface contact detector receiving an input from the accelerometer which is filtered to remove components other than noise.

8. Apparatus for detecting pen-surface contact according to claim 7 and wherein said detector comprises an acceleration noise analyzer operative to extract noise out of the acceleration data and employs the noise for detecting pen-surface contact.

9. Apparatus for detecting pen-surface contact according to claim 7 wherein said at least one accelerometer is retrofittable onto the pen.

10. A handwriting recognition method comprising:

receiving information regarding the acceleration of a writing implement from three mutually orthogonal accelerometers mounted on the writing implement other than at a tip thereof; and

identifying symbols written by the writing implement at least partly on the basis of a comparison between the acceleration of the writing implement and stored acceleration sequences characterizing each of a plurality of symbols, and wherein said step of identifying symbols employs input from a digitizer to provide a first output representing identification of symbols along a first identification channel and employs input from said accelerometers to provide a second output representing identification of symbols along a second identification channel and selects between the first and the second outputs based on predetermined criteria.

11. Handwriting recognition apparatus comprising:

a characteristic acceleration sequence memory operative to store acceleration sequences characterizing each of a plurality of symbols;

a writing implement monitor operative to receive information regarding acceleration of a writing implement from three mutually orthogonal accelerometers mounted on said writing implement other than at a tip thereof; and

a symbol identifier operative to provide an output indication of symbols written by the writing implement at least partly on the basis of a comparison between the acceleration of the writing implement and the stored acceleration sequences,

wherein said symbol identifier includes a first identification channel for employing input from a digitizer to provide a first output representing identification of symbols, a second identification channel for employing input from said accelerometers to provide a second output representing identification of symbols, and symbol selection apparatus for selecting between the first and the second outputs based on predetermined criteria.

12. Handwriting recognition apparatus according to claim 11 wherein said writing implement monitor is also operative to receive information regarding the position of said writing implement.

13. Handwriting recognition method comprising:

employing at least one accelerometer located in a hand held writing implement other than at a tip thereof and at least one additional different sensing technique to sense handwriting; and

receiving an output indication of the handwriting and providing an output indication of symbols represented by the handwriting,

and wherein said step of providing an output indication of symbols employs input from a digitizer to provide a first output representing identification of symbols along a first identification channel and employs input from said at least one accelerometer to provide a second output representing identification of symbols along a second identification channel and selects between the first and the second outputs based on predetermined criteria.

14. A handwriting recognition method according to claim 13 and wherein said at least one additional sensing technique comprises:

position digitizing.

15. A handwriting recognition method according to claim 14 and also comprising identifying symbols by combining digitized position inputs and sensed acceleration information.

16. A handwriting recognition method according to claim 13 and wherein said receiving includes:

employing digitized position inputs to provide a first output representing identification of symbols;

employing acceleration information to provide a second output representing identification of symbols; and

selecting between the first and second outputs.

17. A method for detecting pen-surface contact comprising:

employing at least one accelerometer located in a pen other than at a tip thereof for measuring at least one component of acceleration of a user'"'"'s hand manipulating a pen; and

receiving an input from the accelerometer which is filtered to remove components other than noise.

18. Handwriting recognition apparatus comprising:

handwriting input apparatus including;

at least first and second different handwriting sensors producing first and second handwriting outputs, at least one of said first and second different handwriting sensors comprising at least one accelerometer located in a hand held writing implement other than at a tip thereof; and

an associator for combining the first and second handwriting outputs to provide an enhanced performance handwriting output; and

symbol identification apparatus receiving said enhanced performance handwriting output for providing an output indication of symbols represented by the handwriting,

and wherein said symbol identification apparatus includes a first identification channel employing said first handwriting output to provide a first identification output, a second identification channel employing said second handwriting output to provide a second symbol identification output, and symbol selection apparatus for selecting between the first and the second symbol identification outputs based on predetermined criteria.

19. Handwriting recognition apparatus according to claim 18 and wherein said second handwriting sensor comprises a pad.

20. Handwriting recognition apparatus according to claim 18 and wherein said second handwriting sensor comprises ultrasonic position determination apparatus operative in air.

21. Handwriting recognition apparatus according to claim 20 and wherein said second handwriting sensor is operative to sense handwriting on a computer screen.

22. Handwriting recognition apparatus according to claim 18 and wherein at least one of said first and second handwriting sensors comprises apparatus operative to sense the location of the writing implement.

23. Handwriting recognition apparatus according to claim 18 and wherein said associator is operative to correlate first and second handwriting outputs in a two dimensional domain.

24. Handwriting recognition apparatus according to claim 18 and wherein said associator is operative to correlate first and second handwriting outputs in a time domain.

25. A handwriting recognition method comprising:

employing at least one accelerometer located in a hand held writing implement other than at a tip thereof and at least one additional different sensor to produce first and second handwriting outputs;

combining the first and second handwriting outputs to provide an enhanced performance handwriting output; and

employing said enhanced performance handwriting output for providing an output indication of symbols represented by the handwriting,

and wherein said steps of combining and employing employ said first handwriting output to provide a first output representing identification of symbols along a first identification channel and employ said second handwriting output to provide a second output representing identification of symbols along a second identification channel and select between the first and the second outputs based on predetermined criteria.

26. A method according to claim 25 and wherein said at least one additional different sensor is operative to sense the location of a writing implement.

27. A method according to claims 25 and wherein the step of combining includes correlating said first and second handwriting outputs in a two dimensional domain.

28. A method according to claim 25 and wherein the step of combining includes correlating said first and second handwriting outputs in a time domain.

29. Hand imaging recognition apparatus comprising:

hand imaging input apparatus employing at least two different sensing techniques to sense hand imaging and including at least one accelerometer located in a hand held writing implement other than at a tip thereof; and

shape identification apparatus receiving an output of the hand imaging input apparatus for providing an output indication of shapes represented by the hand imaging,

wherein said shape identification apparatus includes a first identification channel for employing an input produced using a first one of said at least two different sensing techniques to provide a first output representing identification of shapes, a second identification channel for employing an input produced using a second one of said at least two different sensing techniques to provide a second output representing identification of shapes, and shape selection apparatus for selecting between the first and the second outputs based on predetermined criteria.

30. Hand imaging recognition apparatus according to claim 29 and wherein said hand imaging input apparatus comprises a position digitizer.

31. Hand imaging recognition apparatus according to claim 29 and wherein said hand imaging includes at least one of drawing and handwriting.

32. Hand imaging recognition apparatus according to claim 29 and wherein said input apparatus comprises ultrasonic position determination apparatus operative in air.

33. Hand imaging recognition apparatus according to claim 29 and wherein said shape identification apparatus includes combined position and acceleration processing apparatus which combines the inputs from said digitizer and said at least one accelerometer in identifying shapes.

34. Hand imaging recognition apparatus according to claim 29 and wherein said at least one accelerometer is retrofittable onto the writing implement.

35. A hand imaging recognition method comprising:

receiving information regarding the acceleration of a writing implement from three mutually orthogonal accelerometers mounted on said writing implement other than at a tip thereof; and

identifying shapes written by the writing implement at least partly on the basis of a comparison between the acceleration of the writing implement and stored acceleration sequences characterizing each of a plurality of shapes,

and wherein said step of identifying shapes employs input from a digitizer to provide a first output representing identification of shapes along a first identification channel and employs input from said accelerometers to provide a second output representing identification of shapes along a second identification channel and selects between the first and the second outputs based on predetermined criteria.

36. A hand imaging recognition method according to claim 35 and also comprising receiving information regarding the position of a writing implement.

37. A hand imaging recognition method according to claim 35 and also comprising storing acceleration sequences characterizing each of a first plurality of shapes as generated by each of a second plurality of users.

38. Hand imaging recognition apparatus comprising:

a characteristic acceleration sequence memory operative to store acceleration sequences characterizing each of a plurality of shapes;

a writing implement monitor operative to receive information regarding acceleration of a writing implement from three mutually orthogonal accelerometers mounted on said writing implement other than at a tip thereof; and

a shape identifier operative to provide an output indication of shapes written by the writing implement at least partly on the basis of a comparison between the acceleration of the writing implement and the stored acceleration sequences,

wherein said shape identifier includes a first identification channel for employing an input produced using a first sensing technique to provide a first output representing identification of shapes, a second identification channel for employing an input produced using a second sensing technique to provide a second output representing identification of shapes, and shape selection apparatus for selecting between the first and the second outputs based on predetermined criteria.

39. Hand imaging recognition apparatus according to claim 38 wherein said writing implement monitor is also operative to receive information regarding the position of said writing implement.

40. A hand imaging recognition method comprising:

employing at least one accelerometer located in a hand held writing implement other than at a tip thereof and at least one additional different sensing technique to sense hand imaging; and

receiving an output of the hand imaging and providing an output indication of shapes represented by the hand imaging,

and wherein said step of providing an output indication of shapes employs input from a digitizer to provide a first output representing identification of shapes along a first identification channel and employs input from said at least one accelerometer to provide a second output representing identification of shapes along a second identification channel and selects between the first and the second outputs based on predetermined criteria.

41. A hand imaging recognition method according to claim 40 and wherein said at last one additional sensing technique comprises:

position digitizing.

42. A hand imaging recognition method according to claim 40 and also comprising identifying shapes by combining digitized position inputs and sensed acceleration information.

43. A hand imaging recognition method according to claim 40 and wherein said receiving step includes:

employing digitized position inputs to provide a first output representing identification of shapes;

employing acceleration information to provide a second output representing identification of shapes; and

selecting between the first and second outputs.

44. A hand imaging recognition method comprising:

employing at least one accelerometer located in a hand held writing implement other than at a tip thereof and at least one additional different sensing technique to sense hand imaging; and

receiving an output of the hand imaging and providing an output indication of shapes represented by the hand imaging and wherein said receiving and providing comprises extracting noise out of the acceleration data and employing the noise for detecting pen-surface contact.

45. Hand imaging recognition apparatus comprising:

hand imaging input apparatus including;

at least first and second different hand imaging sensors producing first and second hand imaging outputs, at least one of said first and second different hand imaging sensors comprising at least one accelerometer located in a hand held writing implement other than at a tip thereof; and

an associator for combining the first and second hand imaging outputs to provide an enhanced performance hand imaging output; and

shape identification apparatus receiving said enhanced performance hand imaging output for providing an output indication of shapes represented by the hand imaging,

wherein said shade identification apparatus includes a first identification channel employing said first hand imaging output to provide a first shape identification output, a second identification channel employing said second hand imaging output to provide a second shape identification output, and shape selection apparatus for selecting between the first and the second shape identification outputs based on predetermined criteria.

46. Hand imaging recognition apparatus according to claim 45 and wherein said second hand imaging sensor comprises a pad.

47. Handwriting recognition apparatus according to claim 45 and wherein said second handwriting sensor comprises an ultrasonic position determination apparatus operative in air.

48. Hand imaging recognition apparatus according to claim 45 and wherein at least one of said first and second handwriting sensors comprises apparatus operative to sense the location of the writing implement.

49. Hand imaging recognition apparatus according to claim 45 and wherein said associator is operative to correlate first and second hand imaging outputs in a two dimensional domain.

50. Hand imaging recognition apparatus according to claim 45 and wherein said associator is operative to correlate first and second hand imaging outputs in a time domain.

51. A hand imaging recognition method comprising:

employing at least one accelerometer located in a hand held imaging implement other than at a tip thereof and at least one additional different sensor to produce first and second hand imaging outputs;

combining the first and second hand imaging outputs to provide an enhanced performance hand imaging output; and

employing said enhanced performance hand imaging output and providing an output indication of shapes represented by the hand imaging,

and wherein said steps of combining and employing employ said first hand imaging output to provide a first output representing identification of shades along a first identification channel and employ said second hand imaging output to provide a second output representing identification of shapes along a second identification channel and select between the first and the second outputs based on predetermined criteria.

52. A method according to claim 51 and wherein at least one of said first and second handwriting sensors is operative to sense the location of a writing implement.

53. A method according to claim 51 and wherein the combining includes correlating said first and second hand imaging outputs in a two dimensional domain.

54. A method according to claim 51 and wherein the combining includes correlating said first and second hand imaging outputs in a time domain.