Electronic acquisition of a hand formed expression and a context of the expression

• US 8,290,313 B2
• Filed: 10/11/2006
• Issued: 10/16/2012
• Est. Priority Date: 03/18/2005
• Status: Expired due to Fees

• Current Assignee: The Invention Science Fund I, L.L.C. (Intellectual Ventures LLC)
• Original Assignee: The Invention Science Fund I, L.L.C. (Intellectual Ventures LLC)
• Inventors: Cohen, Alexander J., Jung, Edward K. Y., Levien, Royce A., Lord, Robert W., Malamud, Mark A., Rinaldo, John D. Jr.
• Primary Examiner(s): ENTEZARI, MICHELLE M
• Application Number:
  US11/580,218
• Publication Number:
  US 20070075989A1
• Time in Patent Office: 2,197 Days
• Field of Search: None
• US Class Current: 382/321
• CPC Class Codes: G06F 3/002 Specific input/output arrangements not covered by G06F3/01 - G06F3/16 other optical apparatus G02B27/00
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/03545 Pens or stylus
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 40/169 Annotation, e.g. comment data or footnotes
• CPC Class Codes: G06F 40/174 Form filling; Merging
• CPC Class Codes: G06V 10/12 Details of acquisition arrangements; Constructional details thereof
• CPC Class Codes: G06V 30/1423 the instrument generating sequences of position coordinates corresponding to handwriting preprocessing or recognising digital ink G06V30/32
• CPC Class Codes: G06V 30/32 Digital ink
• CPC Class Codes: G09G 2380/04 Electronic labels
• CPC Class Codes: G09G 2380/14 Electronic books and readers
• CPC Class Codes: G09G 3/008 forming an image on an image carrier by relative movement of a writing unit to the image carrier, e.g. on a photoconductive rotating belt, or on an electronic blackboard
• CPC Class Codes: H04L 67/04 specially adapted for terminals or networks with limited capabilities; specially adapted for terminal portability

Abstract

Embodiments include an apparatus, device, method, and computer program product. In an embodiment, a device includes a handheld writing device that includes a writing element, and a writing detector module operable to generate information indicative of a handwriting movement of the writing element. The device also includes a context detector module operable to generate information indicative of a content portion of a document proximate to the handwriting movement of the writing element.

Claims

1. An apparatus having operability to:

a) mark a document with a marking;

b) generate information indicative of a movement of the apparatus;

c) generate a context using at least two context detectors, the context including;

an indicator of a position of the marking relative to an existing character of the document; and

information indicative of at least two types of content existing in at least two planes, the content capable of distinguishing the document, wherein the generation of information indicative of content existing in at least two planes includes detecting the content in each of the at least two planes substantially simultaneously; and

d) save the marking, the information indicative of a movement of the apparatus, and the context within the apparatus.

2. The apparatus of claim 1, wherein the existing character of the document includes at least one existing letter displayed by the document.

3. The apparatus of claim 1, wherein the existing character of the document includes at least one existing word displayed by the document.

4. The apparatus of claim 1, wherein the existing character of the document includes at least one existing image displayed by the document.

11. The apparatus of claim 4, wherein the at least one existing image displayed by the document is captured as the apparatus is moved toward the document.

12. The apparatus of claim 4, wherein the at least one existing image displayed by the document is captured as the apparatus is moved away from the document.

13. The apparatus of claim 4 wherein the at least one image displayed by the document is captured by the apparatus being activated by a motion sensitive switch.

14. The apparatus of claim 4 wherein the at least one image displayed by the document is captured by the apparatus being activated by a voice activatable switch.

17. The apparatus of claim 1, wherein the information indicative of a movement of the apparatus includes information indicative of velocity, acceleration and angular acceleration of the apparatus.

18. The apparatus of claim 17, wherein the information indicative of velocity, acceleration and angular acceleration of the apparatus is operable to recognize human readable content in the marking.

19. The apparatus of claim 18, wherein the information operable to recognize human readable content in the marking is operable to recognize the handwritten words “

My favorite movie line”

in the marking.

20. The apparatus of claim 1, wherein the content existing in at least two planes, the content capable of distinguishing the document, includes content existing in at least a first plane and content existing in at least a second plane, the content existing in at least the first plane differing from the content existing in at least the second plane.

21. The apparatus of claim 1, wherein the document is a skin surface of a human body, and wherein the content existing in at least two planes, the content capable of distinguishing the document, comprises a landmark of a skin surface of a human body, the landmark of a skin surface of a human body including a mole, the mole having an asymmetrical lateral profile in at least a first plane, the mole having at least a portion that is elevated above the skin surface in at least a second plane, and wherein the marking comprises an incision line for a surgical procedure, the incision line proximate to the mole.

22. The apparatus of claim 1, wherein the content existing in at least two planes, the content capable of distinguishing the document, comprises a book title printed on a spine of a book in at least a first plane and an image printed on a cover of the book in at least a second plane.

5. A device comprising:

a pen configured to mark a marking relative to an existing character of a document based, at least in part, on a marking position relative to the existing character of the document, wherein the device generates information indicative of a movement of the device, and wherein the marking and an indicator of the marking position are saved within the pen; and

at least two context detector modules operable to capture at least two types of content existing in a displayed content portion of the document, the content portion existing in at least two planes, the content portion capable of distinguishing the document, wherein the capturing at least two types of content existing in the content portion existing in at least two planes includes detecting the at least two types of content existing in the content portion in each of the at least two planes substantially simultaneously;

wherein at least one of the at least two types of content is captured as a result of activating one of a motion sensitive switch or a voice activated switch.

15. The device of claim 5, wherein the context detector module is operable to capture at least one image of a displayed content portion of the document as the device moves toward the document.

16. The device of claim 5, wherein the context detector module is operable to capture at least one image of a displayed content portion of the document as the device moves away from the document.

6. A method comprising:

discharging a marking substance on a surface proximate to a character displayed by the surface;

generating information corresponding to a mark formed on the surface by the discharged marking substance;

generating information indicative of a movement corresponding to the discharging of the marking substance;

generating, using two or more context detectors, information indicative of at least two types of content in a content portion of the surface proximate to the movement, the content portion comprising one or more physical attributes existing in at least two planes, the content capable of distinguishing the surface, wherein the generation of information indicative of two types of content in content existing in at least two planes includes detecting the content in each of the at least two planes substantially simultaneously;

generating an indication of a position of the mark formed on the surface relative to the character displayed by the surface; and

saving data corresponding to the mark and the indication of a position of the mark formed on the surface relative to the character displayed by the surface.

7. The method of claim 6, wherein the discharging a marking substance on a surface proximate to a character displayed by the surface includes:

discharging a marking substance on a surface of a document proximate to a character displayed by the surface.

8. The method of claim 6, wherein the discharging a marking substance on a surface proximate to a character displayed by the surface includes:

discharging a marking substance on a surface of an object proximate to a character displayed by the surface.

9. The method of claim 6, wherein the generating information corresponding to a mark formed on the surface by the discharged marking substance includes:

generating information corresponding to at least one of a visible trace, a line, a spot, and/or an impression formed on the surface by the discharged marking substance.

10. A device comprising:

means for discharging a marking substance on a surface proximate to a character displayed by the surface;

means for generating information corresponding to a mark formed on the surface by the discharged marking substance;

means for generating information indicative of a movement corresponding to the discharging of a marking substance on the surface;

means for generating information indicative of at least two types of content in a content portion of an object proximate to the hand driven movement of the writing element, the content portion comprising one or more physical attributes of the object existing in at least two planes, the content capable of distinguishing the surface, wherein the generation of information indicative of two types of content existing in at least two planes includes detecting the content in each of the at least two planes substantially simultaneously;

means for capturing an image of a displayed content portion while moving towards the displayed content portion or away from the displayed content portion;

means for generating an indication of a position of the mark formed on the surface relative to the character displayed by the surface; and

means of saving data corresponding to the mark and the indication of a position of the mark formed on the surface relative to the character displayed by the surface.