Method, apparatus, and manufacture for document writing and annotation with virtual ink

• US 8,971,568 B1
• Filed: 10/08/2013
• Issued: 03/03/2015
• Est. Priority Date: 10/08/2012
• Status: Active Grant

• Current Assignee: Gerard Dirk Smits
• Original Assignee: Gerard Dirk Smits
• Inventors: Smits, Gerard Dirk
• Primary Examiner(s): Chu, Randolph I
• Application Number:
  US14/048,954
• Time in Patent Office: 511 Days
• Field of Search: None
• US Class Current: 382/100
• CPC Class Codes: G06F 21/00 Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
• CPC Class Codes: G06F 3/03545 Pens or stylus
• CPC Class Codes: G06F 3/0418 for error correction or compensation, e.g. based on parallax, calibration or alignment
• CPC Class Codes: G06F 3/042 by opto-electronic means
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06V 40/376 Acquisition

Abstract

A method, apparatus, and manufacture for writing and annotation is provided. An image is provided on a surface. Each time invisible ink is deposited on the surface, the location of the invisible ink deposited on the surface is detected before the invisible ink vanishes from the surface. The invisible ink automatically vanishes from the surface in less than one minute from the time that the invisible ink is deposited on the surface. Location information is stored based on each detected location. The projected image is modified based, at least in part, on the stored location information.

Claims

1. A method, comprising:

providing an image on a surface;

each time invisible ink is deposited on the surface by a stylus;

detecting a location of the invisible ink deposited on the surface before the invisible ink vanishes from the surface, wherein the invisible ink automatically vanishes from the surface in less than one minute from the time that the invisible ink is deposited on the surface;

storing location information based on each detected location; and

modifying the provided image based, at least in part, on the stored location information.

2. The method of claim 1, wherein the location information includes, for each time the invisible ink is deposited on the surface, the location on a screen relative to the surface that the invisible ink was deposited on the surface, and the time that the invisible ink was deposited on the surface.

3. The method of claim 1, wherein the surface is at least one of glass, plastic, fabric, or paper.

4. The method of claim 1, further comprising providing an undo function to enable the modifications to the image based on the stored location information to be undone.

5. The method of claim 1, further comprising further modifying the image to compensate for at least one of eye alignment precision errors in the image or surface detection visual parallax errors in the image.

6. The method of claim 1, wherein the invisible ink is invisible to visible light but visible to infrared until the invisible ink vanishes, wherein detecting the location of the invisible ink deposited on the surface is accomplished via infrared.

7. The method of claim 1, wherein the invisible ink is invisible to visible light but visible to ultraviolet until the invisible ink vanishes, wherein detecting the location of the invisible ink deposited on the surface is accomplished via ultraviolet.

8. The method of claim 1, wherein the provided image is provided by a mobile device on a screen of the mobile device, and wherein the surface is the screen of the mobile device.

9. The method of claim 1, wherein the provided image is provided by a mobile device on a screen of the mobile device, and wherein the surface is over the screen of the mobile device.

10. The method of claim 1, wherein modifying the provided image includes providing a visual mark-up on the surface aligned with the location on which the invisible ink was deposited on the surface.

12. The method of claim 1, further comprising,

making a determination as to whether the deposited ink has touched an object in the image or in a pre-existing image on the surface;

if it is determined that the deposited ink has touched an object on the image or on a pre-existing image on the surface associating a visual mark-up with the touched object.

13. The method of claim 12, wherein the image is a 3D image, and wherein the object is 3D object.

14. The method of claim 1, further comprising requiring at least one security confirmation to be successful prior to providing the image.

15. The method of claim 14, wherein the at least one security confirmation includes requiring a signature to be made with the invisible ink and verifying the signature based on a comparison of the signature to at least one of velocity, acceleration, or deceleration of a personal signature of an authorized user.

16. The method of claim 14, wherein providing the image is accomplished with a projection device, and wherein the at least one security confirmation includes verifying a unique identification associated with the projection device.

17. The method of claim 14, wherein providing the image requires decryption of a file that is associated with the image, and wherein the decryption of the file requires a decryption key that is in a unique invisible code on a screen in an embedded pattern within an optical structure that is part of a retro-reflective surface element.

18. The method of claim 1, wherein providing the image on the surface includes projecting the image onto the surface.

19. The method of claim 18, wherein projecting the image onto the surface is accomplished such that, each time the surface is moved, the position of the projected image is unchanged relative to the surface.

11. The method of 10, wherein the provided visual mark-up is adjustable in at least one of color, shape, translucency, duration, or motion.

20. An apparatus, comprising:

at least one processor that is configured to enable actions, including;

providing image control to provide an image on a surface; and

receiving, each time invisible ink is deposited on the surface by a stylus;

a detected location of the invisible ink deposited on the surface before the invisible ink vanishes from the surface, wherein the invisible ink automatically vanishes from the surface in less than one minute from the time that the invisible ink is deposited on the surface; and

a memory that is configured to store location information based on each detected location,

wherein the at least one processor is further configured to enable modifying the provided image based, at least in part, on the stored location information.

21. The apparatus of claim 20 further comprising:

a detector that is configured to detect the location of the invisible ink and provide the detected location to the processor.

22. The apparatus of claim 21, wherein the detector is at least one of an infrared camera, an infrared sensor, an ultraviolet camera, or an ultraviolet sensor.

23. The apparatus of claim 20, wherein the processor is further configured such that at least one security confirmation is required to be successful prior to providing the image.

24. The apparatus of claim 20, further comprising:

a projector that is arranged to project the image onto the surface based on the image control.

25. The apparatus of claim 24, wherein the projector is arranged to project the image such that, each time the surface is moved, the position of the projected image is unchanged relative to the surface.

26. The apparatus of claim 24, wherein the projector is a head-mounted projector that projects the image onto the surface such that only a viewer wearing the head-mounted projector can see the projected image.

27. The apparatus of claim 26, wherein the head-mounted projector includes a first projector and a second projector, the first projector is arranged such that only a first eye of the viewer can see a first eye image projected by the first projector, and wherein the second projector is arranged so that only a second eye of the viewer can see a second eye image projected by the second projector, such that the provided image is the effective image seen by the viewer based on the first eye image seen by the first eye of the viewer and the second eye image seen by the second eye of the viewer.

28. A non-transitive processor-readable storage medium that arranged to encode processor-executable code, which, when executed by one or more processors, enables actions, comprising:

providing image control to provide an image on a surface;

receiving, each time invisible ink is deposited on the surface by a stylus;

a detected location of the invisible ink deposited on the surface before the invisible ink vanishes from the surface, wherein the invisible ink automatically vanishes from the surface in less than one minute from the time that the invisible ink is deposited on the surface;

storing location information based on each detected location; and

modifying the provided image based, at least in part, on the stored location information.

29. The non-transitive processor-readable storage medium of claim 28, the actions further comprising requiring at least one security confirmation to be successful prior to providing the image.

30. The non-transitive processor-readable storage medium of claim 29, wherein the security confirmation includes at least one of biometric signature verification, verifying a unique device identification, or requiring decryption using an encryption key present in a unique invisible code.