Pen-based handwritten character recognition and storage system

• US 6,968,083 B2
• Filed: 01/05/2001
• Issued: 11/22/2005
• Est. Priority Date: 01/06/2000
• Status: Expired due to Fees

• Current Assignee: Zen Optical Technology, LLC
• Original Assignee: Zen Optical Technology, LLC
• Inventors: Williams, David R., Richter, Kathie S.
• Primary Examiner(s): Bali, Vikkram
• Application Number:
  US09/755,763
• Publication Number:
  US 20010038711A1
• Time in Patent Office: 1,782 Days
• Field of Search: 382/100, 382/186, 382/187, 382/188, 382/189, 382/181, 382/183, 341/13, 345/173
• US Class Current: 382/187
• CPC Class Codes: G06V 10/12 Details of acquisition arrangements; Constructional details thereof
• CPC Class Codes: G06V 30/142 using hand-held instruments; Constructional details of the instruments
• CPC Class Codes: G06V 30/1423 the instrument generating sequences of position coordinates corresponding to handwriting preprocessing or recognising digital ink G06V30/32

Abstract

The present invention is directed to an improved system and method in which a pen-sized and shaped device detects, recognizes and stores handwriting as it is written by the device. The invention employs both an active feedback network and a character recognition methodology of partitioning detected input into character components. The active feedback network continually monitors device output to determine the sufficiency of the data input. If the data input is insufficient, the device modifies its detection methodology to obtain data the device readily recognizes. Data recognition is performed in multiple asynchronous processes. Elements of individual characters are sampled by the detector. Character elements are processed and recognized on this elemental level. Recognized character elements are stored for subsequent assembly and recognition on a character level. Thus, preferably two recognition sub-processes take place, one on a character element level and another on a character level.

Claims

1. A device for writing and processing handwriting comprising:

a body;

a marking element for making strokes comprising a character on a surface, the marking element being coupled to the body;

a detector for detecting each stroke on the surface, and

a processor coupled to the detector, wherein the detector and processor are disposed within the body; and

wherein the detector comprises;

an array that includes at least one photo emitter and at least one photo detector;

wherein at least one photo emitter emits light towards the marking element to illuminate the strokes; and

wherein at least one photo detector (a) has a field of view that includes the marking element, (b) detects the strokes using the light reflected off the surface, (c) converts the detected strokes into electronic signals, and (d) sends the electronic signals to the processor; and

wherein the array is in the shape of a ring, with the center of the ring perpendicular to a z-axis that passes through a center of the marking element and is parallel to a major axis of the device.

2. The device according to claim 1, further comprising an active feed-back mechanism, wherein the detector detects the strokes at a periodic rate which is adjusted based on input from the active feedback-mechanism.

3. The device of claim 1, wherein the processor identifies the character by combining the recognized strokes of the character and comparing the combined recognized strokes with a reference set of combined recognized strokes.

4. The device of claim 3, wherein the processor is comprised of a first sub-processor for characterizing each detected stroke as one in a set of reference strokes, and a second sub-processor for identifying the character, the first and second sub-processors functioning asynchronously.

5. The device of claim 3, wherein the processor comprises a first processor for characterizing each detected stroke as a quadrature element using a set of reference quadrature elements, and a second processor for identifying the character, the first and second processor functioning asynchronously.

6. The device of claim 1, wherein the processor characterizes each detected stroke as one in a set of reference strokes by representing each detected stroke as a polynomial representation, comparing the polynomial representation of each detected stroke with polynomial representations of the reference strokes, and selecting for each detected stroke a reference stroke whose polynomial representation is sufficiently similar to the polynomial representation of the detected stroke.

7. The device of claim 1, wherein the processor characterizes each detected stroke as one in a set of reference strokes by representing each detected stroke as a vector representation, comparing the vector representation of each detected stroke with vector representations of the reference strokes, and selecting for each detected stroke a reference stroke whose vector representation is sufficiently similar to the vector representation of the detected stroke.

8. The device of claim 1 further comprising a character output mechanism for outputting a signal representing the character.

9. The device of claim 1, wherein the detector detects the strokes in the temporal order that the strokes are made.

10. The device according to claim 1, wherein the array contains a plurality of elements, each element being either a photo emitter or a photo detector.

11. The device according to claim 10, wherein each element is equally spaced within the array.

12. The device according to claim 1, wherein the array is mounted adjacent to the marking element.

13. The device according to claim 1, wherein at least one photo detector detects the strokes at a periodic rate which is adjusted based on input from an active feedback mechanism.

14. The device according to claim 1, wherein the light and the field of view are approximately centered upon the marking element.

15. The device according to claim 1, wherein each field of view overlaps at least one other field of view.

16. The device according to claim 1, wherein at least one photo emitter emits constant light and at least one photo detector detects diffusely reflected light.

17. The device according to claim 1, wherein at least one photo emitter emits pulsed light and at least one photo detector detects spectrally reflected light.

18. The device according to claim 1, wherein the detector detects quadrature-elements using a feed-forward and feed-backward mechanism.

19. The device of claim 1, wherein the processor identifies the character by combining recognized quadrature elements of the character and comparing the combined recognized quadrature elements with a reference set of quadrature data.

20. The device of claim 1, wherein the processor identifies the character by comparing a recognized quadrature element with a reference set of quadrature data.

21. The device of claim 1, wherein the processor characterizes each detected stroke as a quadrature element as one in a set of reference quadrature elements by representing each detected stroke as a quadrature element selected from the group consisting of:

a basis vector, eigenvector, polynomial, Fast Fourier Transform function, and a combination of vector data and function translations.

22. The device of claim 1 further comprising a character storage mechanism for storing a signal representing the character.

23. The device according to claim 1, wherein at least one photo emitter emits constant light and at least one photo detector detects spectrally reflected light.

24. The device according to claim 1, wherein at least one photo emitter emits pulsed light and at least one photo detector detects diffusely reflected light.

25. A device for writing and processing handwriting comprising:

a body;

a marking element for making strokes comprising a character on a surface, the marking element being coupled to the body;

a detector for detecting each stroke on the surface, and

a processor coupled to the detector, wherein the detector and processor are disposed within the body, wherein the detector comprises;

a photo emitter mounted on a first side of the device, wherein said photo emitter emits light towards the marking element to illuminate the strokes;

a first photo detector mounted on a second side of the device, wherein said first photo detector (a) has a first field of view that includes the marking element, (b) detects the strokes using the light reflected off the surface, (c) converts the detected strokes into electronic signals, and (d) sends the electronic signals to the processor; and

a second photo detector mounted on a third side of the device, wherein said second photo detector (a) has a second field of view that includes the marking element, (b) detects the strokes using the light reflected off the surface, (c) converts the detected strokes into electronic signals, and (d) sends the electronic signals to the processor.

26. The device according to claim 25, wherein the second side is opposite the third side.

27. The device according to claim 25, wherein the photo emitter, the first photo detector, and the second photo detector are mounted adjacent to the marking element.

28. The device according to claim 25, wherein the first photo detector and the second photo detector detect the strokes at a periodic rate which is adjusted based on input from an active feedback mechanism.

29. The device according to claim 25, wherein the light, the first field of view, and the second field of view are approximately centered upon the marking element.

30. The device according to claim 25, wherein the first field of view overlaps the second field of view.

31. The device according to claim 25, wherein the photo emitter emits constant light and the first photo detector and the second photo detector detect diffusely reflected light.

32. The device according to claim 25, wherein the photo emitter emits pulsed light and the first photo detector and the second photo detector detect spectrally reflected light.

33. The device according to claim 25, wherein the photo emitter emits constant light and the first photo detector and the second photo detector detect spectrally reflected light.

34. The device according to claim 25, wherein the photo emitter emits pulsed light and the first photo detector and the second photo detector detect diffusely reflected light.

35. The device according to claim 25, further comprising an active feed-back mechanism, wherein the detector detects the strokes at a periodic rate which is adjusted based on input from the active feedback-mechanism.

36. The device of claim 25, wherein the processor identifies the character by combining the recognized strokes of the character and comparing the combined recognized strokes with a reference set of combined recognized strokes.

37. The device of claim 36, wherein the processor is comprised of a first sub-processor for characterizing each detected stroke as one in a set of reference strokes, and

a second sub-processor for identifying the character, the first and second sub-processors functioning asynchronously.

38. The device of claim 36, wherein the processor comprises a first processor for characterizing each detected stroke as a quadrature element using a set of reference quadrature elements, and a second processor for identifying the character, the first and second processor functioning asynchronously.

39. The device of claim 25, wherein the processor characterizes each detected stroke as one in a set of reference strokes by representing each detected stroke as a polynomial representation, comparing the polynomial representation of each detected stroke with polynomial representations of the reference strokes, and selecting for each detected stroke a reference stroke whose polynomial representation is sufficiently similar to the polynomial representation of the detected stroke.

40. The device of claim 25, wherein the processor characterizes each detected stroke as one in a set of reference strokes by representing each detected stroke as a vector representation, comparing the vector representation of each detected stroke with vector representations of the reference strokes, and selecting for each detected stroke a reference stroke whose vector representation is sufficiently similar to the vector representation of the detected stroke.

41. The device of claim 25 further comprising a character output mechanism for outputting a signal representing the character.

42. The device of claim 25, wherein the detector detects the strokes in the temporal order that the strokes are made.

43. The device according to claim 25, wherein the detector detects quadrature elements using a feed-forward and feed-backward mechanism.

44. The device of claim 25, wherein the processor identifies the character by combining recognized quadrature elements of the character and comparing the combined recognized quadrature elements with a reference set of quadrature data.

45. The device of claim 25, wherein the processor identifies the character by comparing a recognized quadrature element with a reference set of quadrature data.

46. The device of claim 25, wherein the processor characterizes each detected stroke as a quadrature element as one in a set of reference quadrature elements by representing each detected stroke as a quadrature element selected from the group consisting of:

a basis vector, eigenvector, polynomial, Fast Fourier Transform function, and a combination of vector data and function translations.

47. The device of claim 25 further comprising a character storage mechanism for storing a signal representing the character.

48. A device for writing and processing handwriting comprising:

a body;

a marking element for making strokes comprising a character on a surface, the marking element being coupled to the body;

a detector for detecting each stroke on the surface, and

a processor coupled to the detector, wherein the detector and processor are disposed within the body, wherein the detector comprises;

a multi-segment photo emitter mounted on the device, wherein said multi-segment photo emitter emits light towards the marking element to illuminate the strokes;

a first photo detector mounted on a first side of the device, wherein said first photo detector (a) has a first field of view that includes the marking element, (b) detects the strokes using the light reflected off the surface, (c) converts the detected strokes into electronic signals, and (d) sends the electronic signals to the processor; and

a second photo detector mounted on a second side of the device, wherein said second photo detector (a) has a second field of view that includes the marking element, (b) detects the strokes using the light reflected off the surface, (c) converts the detected strokes into electronic signals, and (d) sends the electronic signals to the processor.

49. The device according to claim 48, wherein the multi-segment photo emitter is in the shape of a ring, with the center of the ring perpendicular to a z-axis that passes through a center of the marking element and is parallel to a major axis of the device.

50. The device according to claim 48, wherein the first side is opposite the second side.

51. The device according to claim 48, wherein the multi-segment photo emitter, the first photo detector, and the second photo detector are mounted adjacent to the marking element.

52. The device according to claim 48, wherein the first photo detector and the second photo detector detect the strokes at a periodic rate which is adjusted based on input from an active feedback mechanism.

53. The device according to claim 48, wherein the light, the first field of view, and the second field of view are approximately centered upon the marking element.

54. The device according the claim 48, wherein the first field of view overlaps the second field of view.

55. The device according to claim 48, wherein the multi-segment photo emitter emits constant light and the first photo detector and the second photo detector detect diffusely reflected light.

56. The device according to claim 48, wherein the multi-segment photo emitters emits pulsed light and the first photo detector and the second photo detector detect spectrally reflected light.

57. The device according to claim 48, wherein the multi-segment photo emitter emits constant light and the first photo detector and the second photo detector detect spectrally reflected light.

58. The device according to claim 48, wherein the multi-segment photo emitters emits pulsed light and the first photo detector and the second photo detector detect diffusely reflected light.

59. The device according to claim 48, further comprising an active feed-back mechanism, wherein the detector detects the strokes at a periodic rate which is adjusted based on input from the active feedback-mechanism.

60. The device of claim 48, wherein the processor identifies the character by combining the recognized strokes of the character and comparing the combined recognized strokes with a reference set of combined recognized strokes.

61. The device of claim 60, wherein the processor is comprised of a first sub-processor for characterizing each detected stroke as one in a set of reference strokes, and

a second sub-processor for identifying the character, the first and second sub-processors functioning asynchronously.

62. The device of claim 60, wherein the processor comprises a first processor for characterizing each detected stroke as a quadrature element using a set of reference quadrature elements, and a second processor for identifying the character, the first and second processor functioning asynchronously.

63. The device of claim 48, wherein the processor characterizes each detected stroke as one in a set of reference strokes by representing each detected stroke as a polynomial representation, comparing the polynomial representation of each detected stroke with polynomial representations of the reference strokes, and selecting for each detected stroke a reference stroke whose polynomial representation is sufficiently similar to the polynomial representation of the detected stroke.

64. The device of claim 48, wherein the processor characterizes each detected stroke as one in a set of reference strokes by representing each detected stroke as a vector representation, comparing the vector representation of each detected stroke with vector representations of the reference strokes, and selecting for each detected stroke a reference stroke whose vector representation is sufficiently similar to the vector representation of the detected stroke.

65. The device of claim 48, further comprising a character output mechanism for outputting a signal representing the character.

66. The device of claim 48, wherein the detector detects the strokes in the temporal order that the strokes are made.

67. The device according to claim 48, wherein the detector detects quadrature elements using a feed-forward and feed-backward mechanism.

68. The device of claim 48, wherein the processor identifies the character by combining recognized quadrature elements of the character and comparing the combined recognized quadrature elements with a reference set of quadrature data.

69. The device of claim 48, wherein the processor identifies the character by comparing a recognized quadrature element with a reference set of quadrature data.

70. The device of claim 48, wherein the processor characterizes each detected stroke as a quadrature element as one in a set of reference quadrature elements by representing each detected stroke as a quadrature element selected from the group consisting of:

a basis vector, eigenvector, polynomial, Fast Fourier Transform function, and a combination of vector data and function translations.

71. The device of claim 48 further comprising a character storage mechanism for storing a signal representing the character.