Apparatus and methods for hand motion tracking and handwriting recognition

• US 20020145596A1
• Filed: 04/09/2001
• Published: 10/10/2002
• Est. Priority Date: 04/09/2001
• Status: Active Grant

• Current Assignee: I.C. + Technologies Ltd.
• Original Assignee: Iclimited
• Inventors: Vardi, Micha
• Granted Patent: US 6,831,632 B2
• Time in Patent Office: Days
• US Class Current: 345/179
• CPC Class Codes: G06F 3/03545 Pens or stylus
• CPC Class Codes: G06F 3/0383 Signal control means within the pointing device
• CPC Class Codes: G06F 3/046 by electromagnetic means
• CPC Class Codes: G06V 30/1423 the instrument generating sequences of position coordinates corresponding to handwriting preprocessing or recognising digital ink G06V30/32
• CPC Class Codes: G06V 30/228 of three-dimensional handwriting, e.g. writing in the air

Abstract

Apparatus and method for handwriting recognition including a hand-held device including at least one accelerometer providing an acceleration indicating output, computation circuitry receiving the acceleration indicating output and providing a velocity indicating output and a handwritten multiple character recognizer receiving the velocity indicating output and providing a multiple character recognition output indication.

Claims

1. Apparatus for handwriting recognition comprising:

a hand-held device comprising at least one accelerometer providing an acceleration indicating output;

computation circuitry receiving said acceleration indicating output and providing a velocity indicating output; and

a handwritten multiple character recognizer receiving said velocity indicating output and providing a multiple character recognition output indication.

2. Apparatus for handwriting recognition according to claim 1 and wherein said at least one accelerometer comprises a plurality of accelerometers providing outputs to acceleration processing circuitry, which provides said acceleration indicating output.

3. Apparatus for handwriting recognition according to claim 1 and wherein said computation circuitry is also operable to perform at least one of Calculator, Database, Personal Information Management (PIM) and Personal Digital Assistant (PDA) functionalities, including at least one of address and phone book, data-bank, calendar, time management, calculator, expenses tracking and management, “

to do list”

management, note and memo taking, dictionary, translator and speller, smart-card, magnetic card, credit-card, electronic-wallet, remote-control and smart key functionalities and to provide a computation output resulting therefrom.

4. Apparatus for handwriting recognition according to claim 1 and also comprising communication circuitry.

5. Apparatus for handwriting recognition according to claim 1 and also comprising at least one of a microphone, a speaker, an earphone, an audio interface, an audio processor, a digital signal processor, a display, an image processor, a control circuitry, a CPU, a MPU, a memory, a transceiver, an antenna, a baseband controller, a baseband interface, an Input/Output port, a wireless port, a Bluetooth output, an infra-red port, a serial port, a USB port and data acquisition circuitry.

6. Apparatus according to claim 4 and wherein said communication circuitry provides functionality for at least one of a telephone, paging, SMS, WAP, Email, Internet browsing and POTS functions.

7. Apparatus for handwriting recognition according to claim 1 and wherein said apparatus is employed as a writing implement.

8. Apparatus for handwriting recognition according to claim 1 and wherein said computation circuitry provides at least one of proportional normalization of velocity,

identification of minimum motion points, normalization at least a portion of the acceleration indicating output to a level of an average of a first few samples of the acceleration indicating output, and determination a last intentional motion sample of the acceleration indicating output and eliminating samples following said last intentional motion sample of the acceleration indicating output.

9. Apparatus for handwriting recognition according to claim 1 and wherein said providing a multiple character recognition output indication comprises at least one of:

compressing said velocity indicating output;

separately analyzing information for the x and y velocity components;

analyzing the directions of the velocity;

calculating velocity thresholds;

digitizing velocity components;

comparing and matching between a digital dictionary and a digitized velocity record and using only part of the digitized velocity record;

comparing and matching between a digital dictionary and a few different representation forms of digitized velocity using a prioritization procedure which takes place in case of disagreement between different comparing and matching results;

filtering out short duration segments of the velocity components;

indicating velocity value changes and ignoring time durations between the changes;

performing a merger operation on velocity segments; and

binarizing velocity segments.

10. Apparatus for handwriting recognition comprising:

a hand-held device comprising at least one accelerometer providing an acceleration indicating output;

computation circuitry receiving said acceleration indicating output, discarding user specific components therein and providing a non-user specific output; and

a handwritten character recognizer receiving said non-user specific output and providing a character recognition output indication.

11. Apparatus for handwriting recognition according to claim 10 and wherein said at least one accelerometer comprises a plurality of accelerometers providing outputs to acceleration processing circuitry, which provides said acceleration indicating output.

12. Apparatus for handwriting recognition according to claim 10 and wherein

said computation circuitry is also operable to perform at least one of Calculator, Database, Personal Information Management (PIM) and Personal Digital Assistant (PDA) functionalities, including at least one of address and phone book, data-bank, calendar, time management, calculator, expenses tracking and management, “

to do list”

management, note and memo taking, dictionary, translator and speller, smart-card, magnetic card, credit-card, electronic-wallet, remote-control and smart key functionalities and to provide a computation output resulting therefrom.

13. Apparatus for handwriting recognition according to claim 10 and also comprising communication circuitry.

14. Apparatus for handwriting recognition according to claim 10 and also comprising at least one of a microphone, a speaker, an earphone, an audio interface, an audio processor, a digital signal processor, a display, an image processor, a control circuitry, a CPU, a MPU, a memory, a transceiver, an antenna, a baseband controller, a baseband interface, an Input/Output port, a wireless port, a Bluetooth output, an infra-red port, a serial port, a USB port and data acquisition circuitry.

15. Apparatus according to claim 13 and also comprising a communication functionality, for providing at least one of a telephone, paging, SMS, WAP, Email, Internet browsing, POTS and other types of communications functionality.

16. Apparatus for handwriting recognition according to claim 10 and wherein said apparatus is employed as a writing implement.

18. Apparatus for hand motion tracking according to claim 17 and wherein said at least one accelerometer comprises a plurality of mutually orthogonal arranged accelerometers providing outputs to acceleration processing circuitry which provides said acceleration indicating output.

19. Apparatus for hand motion tracking according to claim 18 and wherein said hand motion tracker provides at least one of a three-dimension tracking output indication, a two-dimension tracking output indication and a character tracking output indication.

20. Apparatus for motion tracking according to claim 17 and also comprising communication circuitry.

21. Apparatus for motion tracking according to claim 17 and also comprising at least one of a microphone, a speaker, an earphone, an audio interface, an audio processor, a digital signal processor, a display, an image processor, a control circuitry, a CPU, a MPU, a memory, a transceiver, an antenna, a baseband controller, a baseband interface, an Input/Output port, a wireless port, a Bluetooth output, an infra-red port, a serial port, a USB port and data acquisition circuitry.

22. Apparatus for motion tracking according to claim 17 and wherein said apparatus is employed as a writing implement.

23. Apparatus for handwriting recognition according to claim 17 and wherein said computation circuitry provides at least one of proportional normalization of velocity, identification of minimum motion points, normalization at least a portion of the acceleration indicating output to a level of an average of a first few samples of the acceleration indicating output, and determination a last intentional motion sample of the acceleration indicating output and eliminating samples following said last intentional motion sample of the acceleration indicating output.

25. Apparatus for hand motion tracking according to claim 24 and wherein said at least one accelerometer comprises a plurality of mutually orthogonal arranged accelerometers providing outputs to acceleration processing circuitry which provides said acceleration indicating output.

26. Apparatus for hand motion tracking according to claim 25 and wherein said hand motion tracker provides at least one of a three-dimension tracking output indication, a two-dimension tracking output indication and a character tracking output indication.

27. Apparatus for motion tracking according to claim 24 and also comprising communication circuitry.

28. Apparatus for motion tracking according to claim 24 and also comprising at least one of a microphone, a speaker, an earphone, an audio interface, an audio processor, a digital signal processor, a display, an image processor, a control circuitry, a CPU, a MPU, a memory, a transceiver, an antenna, a baseband controller, a baseband interface, an Input/Output port, a wireless port, a Bluetooth output, an infra-red port, a serial port, a USB port and data acquisition circuitry.

29. Apparatus for motion tracking according to claim 24 and wherein said apparatus is employed as a writing implement.

17. Apparatus for hand motion tracking comprising:

a hand-held device comprising at least one accelerometer providing an acceleration indicating output;

computation circuitry receiving said acceleration indicating output and providing a proportionally normalized velocity indicating output; and

a hand motion tracker receiving said velocity indicating output and providing a tracking output indication.

24. Apparatus for hand motion tracking comprising:

a hand-held device comprising at least one accelerometer providing an acceleration indicating output;

computation circuitry receiving said acceleration indicating output, discarding user specific components therein and providing a non-user specific output; and

a hand motion tracker receiving said non-user specific output and providing a tracking output indication.

30. Apparatus for handwriting recognition and hand motion tracking comprising:

a hand-held device comprising at least one accelerometer providing an acceleration indicating output;

computation circuitry receiving said acceleration indicating output and providing a velocity indicating output;

a handwritten character recognizer receiving said velocity indicating output and providing a multiple character recognition output indication; and

a hand motion tracker receiving said velocity indicating output and providing a tracking output indication.

31. Apparatus for handwriting recognition and hand motion tracking according to claim 30 and wherein said at least one accelerometer comprises a plurality of accelerometers providing outputs to acceleration processing circuitry, which provides said acceleration indicating output.

32. Apparatus for handwriting recognition and hand motion tracking according to claim 30 and wherein said hand motion tracker provides at least one of a three-dimension tracking output indication, a two-dimension tracking output indication and a character tracking output indication.

33. Apparatus for handwriting recognition and hand motion tracking according to claim 30 and wherein said computation circuitry is also operable to perform at least one of Calculator, Database, Personal Information Management (PIM) and Personal Digital Assistant (PDA) functionalities, including at least one of address and phone book, data-bank, calendar, time management, calculator, expenses tracking and management, “

to do list”

management, note and memo taking, dictionary, translator and speller, smart-card, magnetic card, credit-card, electronic-wallet, remote-control and smart key functionalities and to provide a computation output resulting therefrom.

34. Apparatus for handwriting recognition and hand motion tracking according to claim 30 and also comprising communication circuitry.

35. Apparatus for handwriting recognition and hand motion tracking according to claim 30 and also comprising at least one of a microphone, a speaker, an earphone, an audio interface, an audio processor, a digital signal processor, a display, an image processor, a control circuitry, a CPU, a MPU, a memory, a transceiver, an antenna, a baseband controller, a baseband interface, an Input/Output port, a wireless port, a Bluetooth output, an infra-red port, a serial port, a USB port and data acquisition circuitry.

36. Apparatus according to claim 34 and wherein said communication circuitry provides functionality for at least one of a telephone, paging, SMS, WAP, Email, Internet browsing and POTS functions.

37. Apparatus for handwriting recognition and hand motion tracking according to claim 30 and wherein said apparatus is employed as a writing implement.

38. Apparatus for handwriting recognition and hand motion tracking according to claim 30 and wherein said computation circuitry provides at least one of proportional normalization of velocity,

identification of minimum motion points,

normalization at least a portion of the acceleration indicating output to a level of an average of a first few samples of the acceleration indicating output,

and determination a last intentional motion sample of the acceleration indicating output and eliminating samples following said last intentional motion sample of the acceleration indicating output.

39. Apparatus for handwriting recognition and hand motion tracking according to claim 30 and wherein said providing a multiple character recognition output indication comprises at least one of:

compressing said velocity indicating output;

separately analyzing information for the x and y velocity components;

analyzing the directions of the velocity;

calculating velocity thresholds;

digitizing velocity components;

comparing and matching between a digital dictionary and a digitized velocity record and using only part of the digitized velocity record;

comparing and matching between a digital dictionary and a few different representation forms of digitized velocity using a prioritization procedure which takes place in case of disagreement between different comparing and matching results;

filtering out short duration segments of the velocity components;

indicating velocity value changes and ignoring time durations between the changes;

performing a merger operation on velocity segments; and

binarizing velocity segments.

40. Apparatus for handwriting recognition and hand motion tracking comprising:

a hand-held device comprising at least one accelerometer providing an acceleration indicating output;

computation circuitry receiving said acceleration indicating output, discarding user specific components therein and providing a non-user specific output;

a handwritten character recognizer receiving said non-user specific output and providing a character recognition output indication; and

a hand motion tracker receiving said non-user specific output and providing a tracking output indication.

41. Apparatus for handwriting recognition and hand motion tracking according to claim 40 and wherein said at least one accelerometer comprises a plurality of accelerometers providing outputs to acceleration processing circuitry, which provides said acceleration indicating output.

42. Apparatus for handwriting recognition and hand motion tracking according to claim 40 and wherein said hand motion tracker provides at least one of a three-dimension tracking output indication, a two-dimension tracking output indication and a character tracking output indication.

43. Apparatus for handwriting recognition and hand motion tracking according to claim 40 and wherein said computation circuitry is also operable to perform at least one of Calculator, Database, Personal Information Management (PIM) and Personal Digital Assistant (PDA) functionality, including at least one of address and phone book, data-bank, calendar, time management, calculator, expenses tracking and management, “

to do list”

management, note and memo taking, dictionary, translator and speller, smart-card, magnetic card, credit-card, electronic-wallet, remote-control and smart key functionalities and to provide a computation output resulting therefrom.

44. Apparatus for handwriting recognition and hand motion tracking according to claim 40 and also comprising communication circuitry.

45. Apparatus for handwriting recognition and hand motion tracking according to claim 40 and also comprising at least one of a microphone, a speaker, an earphone, an audio interface, an audio processor, a digital signal processor, a display, an image processor, a control circuitry, a CPU, a MPU, a memory, a transceiver, an antenna, a baseband controller, a baseband interface, an Input/Output port, a wireless port, a Bluetooth output, an infra-red port, a serial port, a USB port and data acquisition circuitry.

46. Apparatus according to claim 44 and wherein said communication circuitry provides functionality for at least one of a telephone, paging, SMS, WAP, Email, Internet browsing and POTS functions.

47. Apparatus for handwriting recognition and hand motion tracking according to claim 40 and wherein said apparatus is employed as a writing implement.

49. A pen computer according to claim 48 and wherein said at least one accelerometer comprises a plurality of mutually orthogonal arranged accelerometers providing outputs to acceleration processing circuitry which provides said acceleration indicating output.

50. A pen computer recognition according to claim 48 and wherein said computer is also operable to perform at least one of Calculator, Database, Personal Information Management (PIM) and Personal Digital Assistant (PDA) functionalities, including at least one of address and phone book, data-bank, calendar, time management, calculator, expenses tracking and management, “

to do list”

management, note and memo taking, dictionary, translator and speller, smart-card, magnetic card, credit-card, electronic-wallet, remote-control and smart key functionalities and to provide a computation output resulting therefrom.

51. A pen computer according to claim 48 and also comprising communication circuitry.

52. A pen computer according to claim 48 and also comprising at least one of a microphone, a speaker, an earphone, an audio interface, an audio processor, a digital signal processor, a display, an image processor, a control circuitry, a memory, a transceiver, an antenna, a baseband controller, a baseband interface, an Input/Output port, a wireless port, a Bluetooth output, an infra-red port, a serial port, a USB port and data acquisition circuitry.

53. Apparatus according to claim 51 and wherein said communication circuitry provides functionality for at least one of a telephone, paging, SMS, WAP, Email, Internet browsing and POTS functions.

54. A pen computer according to claim 48 and wherein said computer is employed as a writing implement.

48. A pen computer comprising:

a hand-held device comprising at least one accelerometer providing an acceleration indicating output;

computation circuitry located within said hand-held device and receiving said acceleration indicating output and providing a non-user specific output;

a handwritten character recognizer located within said hand-held device and receiving said non-user specific output and providing a character recognition output indication; and

a computer located within said hand-held device for receiving said character recognition output indication as performing computer operations in response thereto.

55. A pen communicator comprising:

a hand-held device comprising at least one accelerometer providing an acceleration indicating output;

computation circuitry located within said hand-held device and receiving said acceleration indicating output and providing a proportionally normalized velocity indicating output;

a handwritten character recognizer located within said hand-held device and receiving said proportionally normalized velocity indicating output and providing a character recognition output indication; and

a communication device located within said hand-held device for receiving said character recognition output indication and performing communication operations in response thereto.

56. A pen communicator according to claim 55 and wherein said at least one accelerometer comprises a plurality of mutually orthogonal arranged accelerometers providing outputs to acceleration processing circuitry which provides said acceleration indicating output.

57. A pen communicator claim 55 and comprising also hand motion tracker receiving said velocity indicating output and providing at least one of a three-dimension tracking output indication, a two-dimension tracking output indication and a character tracking output indication.

58. A pen communicator according to claim 55 and wherein said computation circuitry is also operable to perform at least one of Calculator, Database, Personal Information Management (PIM) and Personal Digital Assistant (PDA) functionalities, including at least one of address and phone book, data-bank, calendar, time management, calculator, expenses tracking and management, “

to do list”

management, note and memo taking, dictionary, translator and speller, smart-card, magnetic card, credit-card, electronic-wallet, remote-control and smart key functionalities and to provide a computation output resulting therefrom.

59. A pen communicator according to claim 55 and also comprising at least one of a microphone, a speaker, an earphone, an audio interface, an audio processor, a digital signal processor, a display, an image processor, a control circuitry, a CPU, a MPU, a memory, a transceiver, an antenna, a baseband controller, a baseband interface, an Input/Output port, a wireless port, a Bluetooth output, an infra-red port, a serial port, a USB port and data acquisition circuitry.

60. A pen communicator according to claim 55 and wherein said communication device provides functionality for at least one of a telephone, paging, SMS, WAP, Email, Internet browsing and POTS functions.

61. A pen communicator according to claim 55 and wherein said communicator is employed as a writing implement.

62. A method for handwriting recognition comprising:

providing an acceleration indicating output from a hand-held device;

receiving said acceleration indicating output and providing a velocity indicating output; and

receiving said velocity indicating output and providing a multiple character recognition output indication.

63. A method for hand handwriting recognition according to claim 62 and also comprising at least one of a computation related to the character recognition output indication, a database operation related to the character recognition output indication, communication of character recognition output indication and a control operation based on the character recognition output indication.

64. A method according to claim 62 and wherein said velocity indicating output is further proportionally normalized.

65. A method according to claim 62 and further comprising:

normalizing at least a portion of the acceleration indicating output to a level of the average of the first few samples of the acceleration indicating output.

66. A method according to claim 62 and further comprising:

determining a last intentional motion sample of the acceleration indicating output and eliminating samples following said last intentional motion sample of the acceleration indicating output.

67. A method according to claim 66 and wherein determining of the last intentional motion sample comprises:

examining a record of the output of at least one of said acceleration sensors in order to determine the average of the last few samples;

thereafter, examining said record sequentially from the last few samples backwards in time direction until a variation from the average of at least a predetermined magnitude is encountered; and

determining a time location of this variation to be the time location of the last intentional motion.

68. A method according to claim 62 and wherein said providing a velocity indicating output further comprises identification of minimum motion points.

69. A method according to claim 62 and wherein said providing a multiple character recognition output indication comprises compressing said velocity indicating output;

70. A method according to claim 62 and wherein said providing a multiple character recognition output indication comprises separately analyzing information for the X and Y velocity components.

71. A method according to claim 62 and wherein said providing a multiple character recognition output indication comprises analyzing directions of the velocity.

72. A method according to claim 62 and wherein said providing a multiple character recognition output indication comprises comparing and matching between a digital dictionary and digitized velocity record using only part of the digitized velocity.

73. A method according to claim 62 and wherein said providing a multiple character recognition output indication comprises comparing and matching between a digital dictionary and a few different representation forms of digitized velocity using a prioritization procedure which takes place in case of disagreement between different comparing and matching results.

74. A method according to claim 62 and further comprising at least one of filtering out short duration segments of the velocity components, performing a merger operation on velocity segments and binarizing operation on velocity segments.

75. A method according to claim 62 and wherein said providing a multiple character recognition output indication comprises calculating velocity thresholds.

76. A method according to claim 75 and wherein said calculating of velocity thresholds further comprises dependency of the thresholds on the minimum and maximum values of the velocity components.

77. A method according to claim 62 and further comprising:

digitizing velocity components to provide digitized information.

78. A method according to claim 77 and also comprising:

analyzing said digitized information for the velocity components to detect value changes therein;

indicating each such change;

ignoring time durations between changes.

80. A method for hand handwriting recognition according to claim 79 and also comprising at least one of a computation related to the character recognition output indication, a database operation related to the character recognition output indication, communication of character recognition output indication and a control operation based on the character recognition output indication.

79. A method for handwriting recognition comprising:

providing an acceleration indicating output from a hand-held device;

receiving said acceleration indicating output, discarding user specific components therein and providing a non-user specific output; and

receiving said non-user specific output and providing a character recognition output indication.

81. A method for hand motion tracking comprising:

providing an acceleration indicating output from a hand-held device;

receiving said acceleration indicating output and providing a proportionally normalized velocity indicating output; and

receiving said velocity indicating output and providing a tracking output indication.

82. A method for hand motion tracking according to claim 81 and wherein said providing a tracking indicating output includes at least one of representing a multi-dimensional object, representing drawing in the air, representing drawing in on a surface, representing a character tracking in the air, and representing a character tracking on a surface.

83. A method for hand motion tracking according to claim 81 and also comprising at least one of a computation based on the tracking output indication, a database operation on the tracking output indication, communication of tracking output indication and a control operations based on the tracking output indications.

84. A method according to claim 81 and further comprising:

normalizing at least a portion of the acceleration indicating output to a level of the average of the first few samples of the acceleration indicating output;

85. A method according to claim 81 and further comprising:

determining the last intentional motion sample of the acceleration indicating output and eliminating samples following said last intentional motion sample of the acceleration indicating output;

86. A method according to claim 85 and wherein determining of the last intentional motion sample comprises:

examining a record of the output of at least one of said acceleration sensors in order to determine the average of the last few samples;

thereafter, examining said record sequentially from the last few samples backwards in time direction until a variation from the average of at least a predetermined magnitude is encountered; and

determining a time location of this variation to be the time location of the last intentional motion.

87. A method according to claim 81 and wherein said providing a velocity indicating output further comprises identification of minimum motion points.

88. A method for hand motion tracking comprising:

providing an acceleration indicating output from a hand-held device;

receiving said acceleration indicating output, discarding user specific components therein and providing a non-user specific output; and

receiving said non-user specific output and providing a tracking output indication.

89. A method for hand motion tracking according to claim 88 and wherein said providing a tracking indicating output includes at least one of representing a multi-dimensional object, representing drawing in the air, representing drawing in on a surface, representing a character tracking in the air, and representing a character tracking on a surface.

90. A method for hand motion tracking according to claim 88 and also comprising at least one of a computation based on the tracking output indication, a database operation on the tracking output indication, communication of tracking output indication and a control operations based on the tracking output indication.