Electronic acquisition of a hand formed expression and a context of the expression

• US 8,244,074 B2
• Filed: 10/11/2006
• Issued: 08/14/2012
• Est. Priority Date: 03/18/2005
• Status: Expired due to Fees

• Current Assignee: The Invention Science Fund I, L.L.C. (Intellectual Ventures LLC)
• Original Assignee: The Invention Science Fund I, L.L.C. (Intellectual Ventures LLC)
• Inventors: Cohen, Alexander J., Jung, Edward K. Y., Levien, Royce A., Lord, Robert W., Malamud, Mark A., Rinaldo, John D. Jr.
• Primary Examiner(s): ENTEZARI, MICHELLE M
• Application Number:
  US11/580,217
• Publication Number:
  US 20070080955A1
• Time in Patent Office: 2,134 Days
• Field of Search: 382313-314
• US Class Current: 382/313
• CPC Class Codes: G06F 3/002 Specific input/output arrangements not covered by G06F3/01 - G06F3/16 other optical apparatus G02B27/00
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/03545 Pens or stylus
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 40/169 Annotation, e.g. comment data or footnotes
• CPC Class Codes: G06F 40/174 Form filling; Merging
• CPC Class Codes: G06V 10/12 Details of acquisition arrangements; Constructional details thereof
• CPC Class Codes: G06V 30/1423 the instrument generating sequences of position coordinates corresponding to handwriting preprocessing or recognising digital ink G06V30/32
• CPC Class Codes: G06V 30/32 Digital ink
• CPC Class Codes: H04L 67/04 specially adapted for terminals or networks with limited capabilities; specially adapted for terminal portability

Abstract

Embodiments include an apparatus, device, method, and computer program product. In an embodiment, a device includes a handheld writing device that includes a writing element, and a writing detector module operable to generate information indicative of a handwriting movement of the writing element. The device also includes a context detector module operable to generate information indicative of a content portion of a document proximate to the handwriting movement of the writing element.

Claims

1. An electronic pen comprising:

a generally elongated body;

a writing element extending from the body;

a writing detector module operable to generate information indicative of a hand driven movement of the writing element;

a context detector module operable to generate information indicative of a content portion of a document proximate to the hand driven movement of the writing element, the information further including arbitrary content unrelated to the content portion of the document, the arbitrary content being indicative of an incidental element capable of distinguishing the document via an image capture device operable as the writing element moves, wherein the incidental element is proximate to the hand driven movement, the information further including additional content generated from the content portion of the document proximate to the hand driven movement of the writing element, wherein the additional content includes at least processed data from the information indicative of the content portion of the document proximate to the handwriting movement of the writing element; and

an external transmitting sensor coupled to the body operable to transmit the information indicative of the hand-driven movement of the writing element.

2. The electronic pen of claim 1, wherein the writing element extending from the body includes a writing element extending from the body and configured to make a visible mark.

3. The electronic pen of claim 1, wherein the writing detector module operable to generate information indicative of a hand driven movement of the writing element includes a writing detector module operable to generate information indicative of a hand driven movement of the writing element in contact with a surface of a document.

4. The electronic pen of claim 3, wherein the writing detector module operable to generate information indicative of a hand driven movement of the writing element in contact with a surface of a document includes a writing detector module operable to generate information indicative of a hand driven movement of the writing element in contact with a surface of a paper document.

5. The electronic pen of claim 3, wherein the writing detector module operable to generate information indicative of a hand driven movement of the writing element in contact with a surface of a document includes a writing detector module operable to generate information indicative of a hand driven movement of the writing element in contact with a surface displaying an electronic document.

6. The electronic pen of claim 3, wherein the context detector module operable to generate information indicative of a content portion of a document proximate to the hand driven movement of the writing element includes a context detector module operable to generate information indicative of a content portion of a page of a multipage document proximate to the handwriting movement of the writing element.

7. The electronic pen of claim 1, wherein the context detector module operable to generate information indicative of a content portion of a document proximate to the hand driven movement of the writing element includes a context detector module operable to generate information indicative of a hand generated writing movement of the writing element.

8. The electronic pen of claim 1, further comprising:

a communications module operable to transmit the information indicative of a handwriting movement of a writing element and to transmit the information indicative of at least a portion of a document proximate to the writing movement of the writing element.

9. The electronic pen of claim 1, further comprising:

a communications module.

10. The electronic pen of claim 9, wherein the communications module includes an operability to transmit at least one of the information indicative of a hand driven movement of a writing element and the information indicative of at least a portion of a document proximate to the hand driven movement of the writing element.

11. The electronic pen of claim 9, wherein the communications module includes an operability to transmit at least one of a processed information corresponding to the information indicative of a handwriting movement of a writing element, and a processed information corresponding to the information indicative of at least a portion of a document proximate to the writing movement of the writing element.

12. The electronic pen of claim 9, wherein the communications module includes a communications module operable to transmit wirelessly.

13. The electronic pen of claim 1, further comprising:

a digital storage medium.

31. The electronic pen of claim 1, wherein the image capture device captures the image of a phrase as the moving writing element is creating the phrase.

32. The electronic pen of claim 1, wherein the image capture device captures the image of a word only as the moving writing element is creating the word.

33. The electronic pen of claim 1, wherein the processed data from the information indicative of the content portion of the document proximate to the handwriting movement of the writing element includes saved information indicative of the content portion of the document proximate to the handwriting movement in a digital storage medium physically associated with the handheld device.

14. An apparatus comprising:

a stylus configured to write on a surface;

a position sensor operable to generate information indicative of one or more handwriting movements of the stylus relative to the surface; and

an imaging element operable to generate information representing at least a portion of a content displayed from the surface proximate to the handwriting movements, the portion of the content including information being indicative of an arbitrary unrelated incidental element capable of distinguishing the handwriting movements, the imaging element operable to render textural information by an optical character resolution process as the writing element moves, wherein the incidental element is proximate to the one or more handwriting movements, the information further including additional content generated from the portion of the content displayed from the surface proximate to the handwriting movements, wherein the additional content includes at least processed data from the portion of the content displayed from the surface proximate to the handwriting movements.

15. A method comprising:

marking at least a portion of a document in response to a hand driven movement of a handheld marking device;

generating in the handheld marking device a first data set indicative of the marking; and

generating in the handheld marking device a second data set indicative of a content portion of the document and additional information proximate to the marking, the additional information being indicative of an arbitrary incidental element unrelated to the document, wherein the arbitrary incidental element is proximate to the marking, and capable of distinguishing the document, the generating including;

transmitting the information indicative of the hand-driven movement of the writing element via an external transmitting sensor coupled to an image capture device operable to render textural information by an optical character resolution process as the handheld marking device moves; and

generating additional content from the content portion of the document, wherein the additional content includes at least processed data from the content portion of the document.

16. The method of claim 15, further comprising:

saving the first data set indicative of the marking in a digital storage medium physically associated with the handheld device.

17. The method of claim 15, further comprising:

saving the second data set indicative of a content portion of the document proximate to the marking in a digital storage medium physically associated with the handheld device.

18. The method of claim 15, further comprising:

transmitting from the handheld marking device the first data set indicative of the marking and the second data set indicative of a content portion of the document proximate to marking.

19. The method of claim 15, further comprising:

transmitting the first data set indicative of the marking from the handheld device.

20. The method of claim 19, wherein the transmitting the first data set indicative of the marking from the handheld device includes transmitting processed data corresponding to the first data set indicative of the marking from the handheld device.

21. The method of claim 15, further comprising:

transmitting the second data set indicative of a content portion of the document proximate to marking from the handheld device.

22. The method of claim 21, wherein the transmitting the second data set indicative of a content portion of the document proximate to marking from the handheld device includes transmitting processed data corresponding to the second data set indicative of a content portion of the document proximate to marking from the handheld device.

23. A system comprising:

a context detecting apparatus operable to generate information indicative of a content portion of a document via an image capture device operable to render textural information by an optical character resolution process as the context detecting apparatus moves, the information further including additional content generated from the content portion of the document, wherein the additional content includes at least processed data from the content portion of the document;

an external transmitting sensor coupled to the context detecting apparatus operable to transmit the textural information; and

an annotating apparatus operable to generate information indicative of a user created expression associated with the content portion of the document, the user-created expression being indicative of an arbitrary unrelated incidental element capable of distinguishing the document, wherein the incidental element is proximate to the user created expression.

24. The system of claim 23, wherein the context detecting apparatus operable to generate information indicative of a content portion of a document includes a handheld context detecting apparatus operable to generate information indicative of a content portion of a document.

25. The system of claim 23, wherein the context detecting apparatus operable to generate information indicative of a content portion of a document includes a human wearable context detecting apparatus operable to generate information indicative of a content portion of a document.

26. The system of claim 25, wherein the human wearable context detecting apparatus operable to generate information indicative of a content portion of a document includes eye glasses having a context detecting apparatus operable to generate information indicative of a content portion of a document.

27. The system of claim 23, wherein the annotating apparatus operable to generate information indicative of a user-created expression associated with the content portion of the document includes a human handheld annotating apparatus operable to generate information indicative of a user-created expression associated with the content portion of the document.

28. The system of claim 27, wherein the human handheld annotating apparatus operable to generate information indicative of a user-created expression associated with the content portion of the document includes a handheld annotating apparatus operable to generate information indicative of a verbal user expression associated with the content portion of the document.

29. The system of claim 27, wherein the human handheld annotating apparatus operable to generate information indicative of a user-created expression associated with the content portion of the document includes a handheld annotating apparatus operable to generate information indicative of a handwritten user expression associated with the content portion of the document.

30. The system of claim 27, wherein the human handheld annotating apparatus operable to generate information indicative of a user-created expression associated with the content portion of the document includes a handheld annotating apparatus operable to generate an information indicative of a predetermined user expression associated with the content portion of the document.