Method, apparatus, and manufacture for document writing and annotation with virtual ink

• US 9,501,176 B1
• Filed: 03/02/2015
• Issued: 11/22/2016
• Est. Priority Date: 10/08/2012
• Status: Active Grant

• Current Assignee: Gerard Dirk Smits
• Original Assignee: Gerard Dirk Smits
• Inventors: Smits, Gerard Dirk
• Primary Examiner(s): Chu, Randolph I
• Application Number:
  US14/636,062
• Time in Patent Office: 631 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 21/00 Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
• CPC Class Codes: G06F 3/03545 Pens or stylus
• CPC Class Codes: G06F 3/0418 for error correction or compensation, e.g. based on parallax, calibration or alignment
• CPC Class Codes: G06F 3/042 by opto-electronic means
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06V 40/376 Acquisition

Abstract

A method, apparatus, and manufacture for writing and annotation is provided. An image is provided on a surface. In one embodiment, each time invisible ink is deposited on the surface, the location of the invisible ink deposited on the surface is detected before the invisible ink vanishes from the surface. In another embodiment, when a tip of a stylus is in contact with a location of an image on the surface, employing three or more light detectors to detect light at the location. The detected light is employed to determine a position and an orientation of the tip of the stylus and the location on the surface, and modifying the image based on the stored information.

Claims

1. A method for using an optical pen, comprising:

when a tip of a stylus is in contact with a location of an image on a surface, employing three or more line of sight light detectors on the optical pen to separately detect reflected light from three points around the location, wherein the three or more line of sight light detectors are positioned at angular locations offset along the directional orientation of the stylus formed by the tip, and wherein the detected reflected light is provided by one or more remotely located light sources that are separate from the optical pen;

employing the detected lights from line of sight detectors to determine a position and an orientation of the tip of the stylus and the location on the surface;

storing information for the position, orientation and location; and

modifying the image based on the stored information.

2. The method of claim 1, further comprising employing an amount of pressure on the tip in contact with the surface to determine when to detect light at the location.

3. The method of claim 1, wherein the three or more line of sight light detectors employ spatial filtering to determine the information.

4. The method of claim 1, further comprising employing one or more line of sight light sources on the stylus to project light onto the location on the surface, wherein the projected light is reflected by the location for detection by the light detectors.

5. The method of claim 1, further comprising:

employing the line of sight light detectors to detect light emitted from the location on the surface towards the stylus when the tip is in contact with the surface; and

employing one or more patterns in the detected light to determine the location on the surface where the tip of the stylus is in contact.

6. The method of claim 1, further comprising providing haptic feedback to the stylus based at least on pressure applied to the tip.

7. The method of claim 1, further comprising employing a remote light source that is arranged to project the image onto the surface.

8. An optical pen for editing an image, comprising:

a stylus, further comprising;

three or more line of sight light detectors arranged around a tip, wherein the tip is arranged to touch the image on a surface; and

a hardware processor that is arranged to performs actions, including;

when a tip of a stylus is in contact with a location of an image on a surface, employing the three or more line of sight light detectors on the optical pen to separately detect reflected light from three points around the location, wherein the three or more line of sight light detectors are positioned at angular locations offset along the directional orientation of the stylus formed by the tip, and wherein the detected reflected light is provided by one or more remotely located light sources that are separate from the optical pen;

employing the detected lights from line of sight detectors to determine a position and an orientation of the tip of the stylus and the location on the surface;

storing information in the hardware memory for the position, orientation and location of the stylus tip; and

modifying the image based on the stored information.

9. The apparatus of claim 8, further comprising employing an amount of pressure on the tip in contact with the surface to determine when to detect light at the location.

10. The apparatus of claim 8, wherein the three or more line of sight light detectors employ spatial filtering to determine the information.

11. The apparatus of claim 8, further comprising one or more light sources on the stylus, wherein the light sources project light onto the location on the surface, and wherein the projected light is reflected by the location for detection by the three or more line of sight light detectors.

12. The apparatus of claim 8, further comprising:

employing the three or more line of sight light detectors to detect light emitted from the location on the surface towards the stylus when the tip is in contact with the surface; and

employing one or more patterns in the detected light to determine the location on the surface where the tip of the stylus is in contact.

13. The apparatus of claim 8, further comprising providing haptic feedback to the stylus based at least on pressure applied to the tip.

14. The apparatus of claim 8, further comprising a remote light source that is arranged to project the image onto the surface.

15. A processor-readable non-transitive storage media that is operative to store data and instructions, which, when executed by one or more processors, enables actions with an optical pen, comprising:

when a tip of a stylus is in contact with a location of an image on a surface, employing three or more line of sight light detectors on the optical pen to separately detect reflected light from three points around the location, wherein the three or more line of sight light detectors are positioned at angular locations offset along the directional orientation of the stylus formed by the tip;

employing the detected lights from line of sight detectors to determine a position and an orientation of the tip of the stylus and the location on the surface;

storing information for the position, orientation and location; and

modifying the image based on the stored information.

16. The media of claim 15, further comprising employing one or more light sources on the stylus to project light onto the location on the surface, wherein the projected light is reflected by the location for detection by the three or more line of sight light detectors.

17. The media of claim 15, further comprising:

employing the three or more line of sight light detectors to detect light emitted from the location on the surface towards the stylus when the tip is in contact with the surface; and

employing one or more patterns in the detected light to determine the location on the surface where the tip of the stylus is in contact.

18. The media of claim 15, further comprising employing a remote light source that is arranged to project the image onto the surface.