Electronic acquisition of a hand formed expression and a context of the expression

• US 8,340,476 B2
• Filed: 03/18/2005
• Issued: 12/25/2012
• Est. Priority Date: 03/18/2005
• Status: Expired due to Fees

• Current Assignee: The Invention Science Fund I, L.L.C. (Intellectual Ventures LLC)
• Original Assignee: The Invention Science Fund I, L.L.C. (Intellectual Ventures LLC)
• Inventors: Cohen, Alexander J., Jung, Edward K. Y., Levien, Royce A., Lord, Robert W., Malamud, Mark A., Rinaldo, John D. Jr.
• Primary Examiner(s): ENTEZARI, MICHELLE M
• Application Number:
  US11/083,567
• Publication Number:
  US 20060209051A1
• Time in Patent Office: 2,839 Days
• Field of Search: 382313-314
• US Class Current: 382/313
• CPC Class Codes: G06F 3/002 Specific input/output arrangements not covered by G06F3/01 - G06F3/16 other optical apparatus G02B27/00
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/03545 Pens or stylus
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 40/169 Annotation, e.g. comment data or footnotes
• CPC Class Codes: G06F 40/174 Form filling; Merging
• CPC Class Codes: G06V 10/12 Details of acquisition arrangements; Constructional details thereof
• CPC Class Codes: G06V 30/1423 the instrument generating sequences of position coordinates corresponding to handwriting preprocessing or recognising digital ink G06V30/32
• CPC Class Codes: G06V 30/32 Digital ink
• CPC Class Codes: H04L 67/04 specially adapted for terminals or networks with limited capabilities; specially adapted for terminal portability

Abstract

Embodiments include an apparatus, device, method, and computer program product. In an embodiment, a device includes a handheld writing device that includes a writing element, and a writing detector module operable to generate information indicative of a handwriting movement of the writing element. The device also includes a context detector module operable to generate information indicative of a content portion of a document proximate to the handwriting movement of the writing element.

Claims

1. A handheld writing device comprising:

a writing element;

a writing detector module operable to generate information indicative of a handwriting movement of the writing element; and

a context detector module operable to;

[a ]generate information indicative of a content portion of a document proximate to the handwriting movement by the writing element via an image capture device operable to render textural structural information by an optical character resolution process as the writing element moves;

[b ]transmit the information indicative of the content portion of the document proximate to the handwriting movement;

[c ]acquire contextual information via a verbal user expression associated with the content portion of the document, wherein the acquire contextual information includes context of the content portion; and

[d ]generate additional content of the information indicative of the content portion of the document proximate to the handwriting movement of the writing element, wherein the additional content includes at least processed data of the information indicative of the content portion of the document proximate to the handwriting movement of the writing element.

2. The handheld writing device of claim 1, wherein the writing element includes a marking element.

3. The handheld writing device of claim 1, wherein the writing element includes a non-marking element.

4. The handheld writing device of claim 1, wherein the writing detector module operable to generate information indicative of a handwriting movement of the writing element includes an electrical circuit operable to generate information indicative of a handwriting movement of the writing element.

5. The handheld writing device of claim 1, wherein the context detector module operable to generate information indicative of a content portion of a document proximate to the handwriting movement of the writing element includes an electrical circuit operable to generate information indicative of a content portion of a document proximate to the handwriting movement of the writing element.

6. The handheld writing device of claim 1, wherein the writing detector module operable to generate information indicative of a handwriting movement of the writing element includes a writing detector module operable to generate information indicative of a handwriting movement by the writing element generated by an activity of writing by hand.

7. The handheld writing device of claim 1, wherein the writing detector module operable to generate information indicative of a handwriting movement by the writing element includes a writing detector module operable to generate information indicative of a handwriting movement by the writing element in response to a hand generated activity.

8. The handheld writing device of claim 1,

wherein the writing detector module includes the context detector module.

9. The handheld writing device of claim 1, wherein the context detector module operable to generate information indicative of a content portion of a document proximate to the handwriting movement by the writing element includes a context detector module operable to generate information indicative of a printed content portion of a document proximate to the handwriting movement by the writing element.

10. The handheld writing device of claim 1, wherein the information indicative of a content portion of a document proximate to the handwriting movement of the writing element includes information indicative of a textual content portion of a document proximate to the handwriting movement of the writing element.

11. The handheld writing device of claim 1, wherein the information indicative of a content portion of a document proximate to the handwriting movement of the writing element includes information indicative of a human language content portion of a document proximate to the handwriting movement of the writing element.

12. The handheld writing device of claim 1, wherein the information indicative of a content portion of a document proximate to the handwriting movement of the writing element includes information indicative of a human readable content portion of a document proximate to the handwriting movement of the writing element.

13. The handheld writing device of claim 1, wherein the information indicative of a content portion of a document proximate to the handwriting movement of the writing element includes information indicative of a graphical content portion of a document proximate to the handwriting movement of the writing element.

14. The handheld writing device of claim 1,

further comprising:

a recognition module operable to generate information indicative of a human readable content of a hand driven movement of the writing element in

response to the information indicative of the hand driven movement of the writing element.

23. The hand-held writing device of claim 1, further comprising a voice activated switch to capture an image of the content portion.

24. The handheld writing device of claim 1, wherein the additional information from the information indicative of the content portion of the document proximate to the handwriting movement of the writing element includes the location of the content portion.

25. The handheld writing device of claim 1, wherein the additional information from the information indicative of the content portion of the document proximate to the handwriting movement of the writing element includes the textual structure of the content portion.

26. The handheld writing device of claim 1, wherein the processed data of the information indicative of the content portion of the document proximate to the handwriting movement of the writing element includes saved the information indicative of the content portion of the document proximate to the handwriting movement in a digital storage medium physically associated with the handheld device.

15. A method comprising:

generating in a handheld device a first information indicative of a handwriting movement of a writing element physically associated with the handheld device;

generating in the handheld device a second information indicative of a content portion of a document proximate to the handwriting movement of the writing element via an image capture device operable to render textural structural information by an optical character resolution process as the writing element moves;

transmitting the second information indicative of the content portion of the document proximate to the handwriting movement;

acquiring contextual information via a verbal user expression associated with the content portion of the document, wherein the acquiring contextual information includes context of the content portion; and

generating additional content of the second information indicative of the content portion of the document proximate to the handwriting movement of the writing element, wherein the additional content includes at least processed data of the information indicative of the content portion of the document proximate to the handwriting movement of the writing element.

22. The method of claim 15, further comprising:

capturing segments of the handwriting movement;

saving each of the segments of the handwriting movement as a separate Image; and

stitching each separate image together to form an output.

16. A computer program product including a computer storage medium product embodied in code and operable to execute a computer process in a handheld computing device, the computer process comprising:

generating in the handheld computing device a first data set indicative of a handwriting movement of a writing element physically associated with the handheld device;

generating in the handheld device a second data set indicative of a content portion of a document proximate to the handwriting movement of the writing element via an image capture device operable to render textural structural information by an optical character resolution process as the writing element moves;

transmitting the second data set indicative of the content portion of the document proximate to the handwriting movement;

acquiring contextual information via a verbal user expression associated with the content portion of the document, wherein the acquiring contextual information includes context of the content portion; and

generating additional content of the second data set indicative of the content portion of the document proximate to the handwriting movement of the writing, wherein the additional content includes at least processed data of the information indicative of the content portion of the document proximate to the handwriting movement of the writing element.

17. The computer program product of claim 16, further comprising:

saving in a digital storage medium physically associated with the handheld device the first data set indicative of a handwriting movement and the second data set indicative of a content portion of the document proximate to the handwriting movement.

18. The computer program product of claim 16, further comprising:

transmitting the first data set indicative of the handwriting movement.

19. The computer program product of claim 16, further comprising:

transmitting the second data set indicative of a content portion of the document proximate to the handwriting movement.

20. The computer program product of claim 19, wherein the computer storage medium product includes a computer program product.

21. The computer program product of claim 18, wherein the first data set indicative of the handwriting movement is transmitted after batching the first data set indicative of the handwriting movement with an additional data set.