Optical position determination on any surface

• US 20020163511A1
• Filed: 04/29/2002
• Published: 11/07/2002
• Est. Priority Date: 11/29/2000
• Status: Abandoned Application

• Current Assignee: Anoto Group AB
• Original Assignee: Anoto Group AB
• Inventors: Sekendur, Oral Faith
• Application Number:
  US10/134,112
• Publication Number:
  US 20020163511A1
• Time in Patent Office: Days
• US Class Current: 345/179
• CPC Class Codes: G06F 3/0321 by optically sensing the absolute position with respect to a regularly patterned surface forming a passive digitiser, e.g. pen optically detecting position indicative tags printed on a paper sheet constructional details of pen-shaped pointing devices G06F3/03545, G06F3/03542, G06F3/037
• CPC Class Codes: G06F 3/0325 using a plurality of light emitters or reflectors or a plurality of detectors forming a reference frame from which to derive the orientation of the object, e.g. by triangulation or on the basis of reference deformation in the picked up image
• CPC Class Codes: G06F 3/03542 Light pens for emitting or receiving light
• CPC Class Codes: G06F 3/03545 Pens or stylus

Abstract

The present invention proposes the use of a surface or writing surface such as paper and a moveable element such as a pen or a stylus. The stylus comprises an input means such as a charge-coupled device (CCD) or digital camera, a microcomputer, memory, power supply, and a communications device, whereby the digital camera scans the surface for position-related information to determine the position and/or movement of the stylus relative to the surface. The path of stylus is determined by detecting a sequence of position-related information. An output signal from the digital camera or array of light sensitive elements is sent to a computer or processor and finally output to the user. The output can be in various forms including an image on a computer display or a computer printout. When writing on the surface, handwriting recognition software can be used to convert the handwritten text into a “keyboard-typed” representation.

Claims

1. An apparatus for obtaining and outputting the position and movement of a moveable element on a surface comprising

a. a position-related means for designating position information,

b. an input means for acquiring position information from said position-related means,

c. an output means for outputting position information, and

d. a processing means for obtaining and analyzing position information from said output means.

2. An apparatus according to claim 1 whereby

a. said position-related means comprises at least three points,

b. said at least three points comprising at least two fixed points and at least one movable point,

c. said movable element comprises said at least one movable point, and

d. comprises triangulating means for determining the position and movement of said moveable element.

3. An apparatus according to claim 1 whereby

a. said position-related means comprises at least one pattern, and

b. comprises pattern-matching means for determining the position and movement of said moveable element.

4. An apparatus according to claim 1 whereby said position-related means comprises position-related code means designating two dimensional coordinates of at least one point for determining the position and movement of said moveable element.

5. An apparatus according to claim 1 comprising means for determining the position and movement of said moveable element selected from the group consisting of

a. triangulating means,

b. pattern-matching means,

c. position-related code means, and

d. any combination thereof.

6. An apparatus according to claim 1 comprising a cell phone selected from the group consisting of

a. said cell phone comprising a mouth piece, an ear piece, a display, function keys, a pen, a pen retractor, a keypad, a lower digital camera, memory, a micro computer, power supply and a writing element;

b. said cell phone comprising a mouth piece, an ear piece, a display, function keys, a pen, a pen retractor, a keypad, a lower digital camera, memory, a micro computer, power supply, a writing element, and a lower sensor selected from the group consisting of a track ball, pressure sensor, transducer, gyroscope, and an accelerometer;

c. said cell phone comprising a portable video camera further comprising a mouthpiece that functions as a microphone, an earpiece that functions as a speaker and a display that functions as a camera display;

d. said cell phone comprising a portable video camera further comprising a mouthpiece that functions as a microphone, an earpiece that functions as a speaker and a display that functions as a camera display whereby the camera display flips open and twists for viewing and exposing function keys;

e. said cell phone whereby said cell phone flips open for viewing and exposing function keys and a display;

f. said cell phone in the shape of a stylus; and

g. said cell phone for hand held use.

7. An apparatus according to claim 1 selected from the group consisting of

a. said surface comprising a writing surface;

b. said movable element comprising a writing means for writing on said surface;

c. said surface comprising a substantially two dimensional planar face;

d. said input means is a detector means comprising an array of light sensitive elements for detecting said position-related means and said output means for generating at least one output signal thereof,

e. said processing means comprising means for receiving and processing said at least one output signal form said input means, thereby to determine the position and movement of said movable element relative to said surface;

f. said moveable element comprising said input means, whereby said movable element is movable relative to said surface;

g. means for determining the path of said moveable element by detecting said position-related means;

h. means for determining the path of said moveable element by detecting a sequence of said position-related means;

i. means for analyzing the path of said moveable element;

j. said position-related means comprising position-related data;

k. said position-related means is printed on said surface;

l. said position-related means is written on said surface;

m. said position-related means comprising existing features on said surface;

n. said position-related means comprising existing features obtainable by said input means;

o. at least part of said position-related means is outside said surface;

p. said surface comprising a writing surface and said movable element comprising a writing means for writing on said surface and whereby said writing means comprises said at least part of said position-related means;

q. said apparatus intended for use with a computer;

r. said apparatus comprising a computer display;

s. said apparatus comprising a computer printer;

t. said apparatus comprising a computer;

u. said apparatus comprising a digital camera;

v. said apparatus comprising writing means for writing on said surface;

w. said apparatus comprising a microcomputer;

x. said apparatus comprising a display;

y. said apparatus comprising function keys;

z. said apparatus comprising an illumination means;

aa. said processing means comprising a computer;

bb. said apparatus comprising a writing means for writing on said surface and further comprising an original hard copy means for forming an original hard copy made by said writing means on said surface;

cc. said surface made of a material selected from the group consisting of paper, plastic, glass, metal, synthetic fiber, synthetic material, natural material, and a paper like substance;

dd. said position-related means comprising a reflecting means for reflecting said position-related means to said input means;

ee. said moveable element selected from the group consisting of a stylus shaped moveable element for hand held use, a hand held moveable element, a cell phone and a stylus shaped cell phone;

ff. said surface comprising an overlay means for overlaying said surface with said position-related means;

gg. said surface comprising an overlay means for overlaying said surface with said position-related means selected from the group consisting of carbon paper, film, template, plate and sheet;

hh. said position-related means comprising a selective reflecting means for selectively reflecting at least one selected frequency of light;

ii. said input means comprising a selective input means for selectively inputting at least one selected frequency of light;

jj. said input means comprising a selective input means for selectively inputting said position-related means;

kk. said apparatus comprising a pressure switch means for turning on said apparatus when pressure is applied to said movable element;

ll. said apparatus comprising a focusing means for focusing said surface;

mm. said movable element comprising a writing means for writing on said surface;

nn. said movable element comprising a self contained optical stylus, a writing means for writing on said surface, a microcomputer, a user interface means for communicating with a user and a device interface means for communicating with other devices;

oo. said input means selected from the group consisting of said input means mounted on said movable element;

said input means mounted on a computer monitor, said input means mounted on a portable computer, said input means mounted to input said position-related means;

pp. said input means selected from the group consisting of an upper input means, a lower input means, an upper input means and a lower input means mounted in the same direction, an upper input means and a lower input means mounted in opposite directions;

qq. said apparatus further comprising an element selected from the group consisting of a track ball, a pressure sensor, a transducer, a gyroscope, and an accelerometer;

rr. said apparatus comprising a cell phone in the shape of a stylus further comprising a mouth piece, an ear piece, a display, function keys, a pen, a pen retractor, a keypad, a digital camera, memory, a micro computer, power supply, and a writing element;

ss. said processing means adapted to produce a digital duplicate of visible marks made on said surface,

tt. said apparatus comprising macro-triangulating means and micro-triangulating means for determining the position and movement of said moveable element;

uu. said apparatus comprising handwriting recognition means for converting handwritten text into a “

keyboard-typed”

representation;

vv. said surface comprising a repetitive pattern that is made unique by marking said repetitive pattern with said movable element comprising a writing element;

ww. said input means comprising an upper digital camera selected from the group consisting of a video camera, a still camera, video conferencing camera and an internet camera;

xx. said input means comprising a lower digital camera selected from the group consisting of a video camera, a still camera, video conferencing camera and an internet camera;

yy. said input means comprising a lower digital camera that can function as an optical computer mouse;

zz. said apparatus comprising an “

ink-well”

like interface for said output means to communicate with other devices using means selected form the group consisting of optical means, wireless means and electronic means;

aaa. said input means comprising a digital camera and further comprising mirrors mounted at angles to detect the position of said movable element using triangulation, pattern-matching and/or position-related code;

bbb. said input means comprising a digital camera and further comprising a split-screen to capture different views of said surface whereby said different views are calibrated and compared with each other to determine movement and position of the writing element;

ccc. said apparatus comprising two digital cameras to capture different views of the surface whereby the frames of said two digital cameras are calibrated and compared with each other to determine movement and position of said movable element.

8. A device for producing an electronic duplicate of markings made upon a surface comprising

a. a position-related means for designating position information,

b. a movable element,

c. a detector means for detecting said position-related means and for generating at least one output signal thereof, and

d. a processing means for receiving and processing said at least one output signal, thereby to determine the position of said movable element relative to said surface.

9. A device according to claim 8 whereby

a. said position-related means comprises at least three points,

b. said at least three points comprise least two fixed points and at least one movable point,

c. said movable element comprises said at least one movable point, and

d. said device further comprises triangulating means for determining the position and movement of said moveable element.

10. A device according to claim 8 whereby

c. said position-related means comprises at least one pattern, and

d. said device further comprises pattern-matching means for determining the position and movement of said moveable element.

11. A device according to claim 8 whereby said position-related means comprises position-related code means designating two dimensional coordinates of at least one point for determining the position and movement of said moveable element.

12. A device according to claim 8 comprising means for determining the position and movement of said moveable element selected from the group consisting of

e. triangulating means,

f. pattern-matching means,

g. position-related code means, and

h. any combination thereof.

13. A device according to claim 8 comprising a cell phone selected from the group consisting of

a. said cell phone comprising a mouth piece, an ear piece, a display, function keys, a pen, a pen retractor, a keypad, a lower digital camera, memory, a micro computer, power supply and a writing element;

b. said cell phone comprising a mouth piece, an ear piece, a display, function keys, a pen, a pen retractor, a keypad, a lower digital camera, memory, a micro computer, power supply, a writing element, and a lower sensor selected from the group consisting of a track ball, pressure sensor, transducer, gyroscope, and an accelerometer;

c. said cell phone comprising a portable video camera further comprising a mouthpiece that functions as a microphone, an earpiece that functions as a speaker and a display that functions as a camera display;

d. said cell phone comprising a portable video camera further comprising a mouthpiece that functions as a microphone, an earpiece that functions as a speaker and a display that functions as a camera display whereby the camera display flips open and twists for viewing and exposing function keys;

e. said cell phone whereby said cell phone flips open for viewing and exposing function keys and a display;

f. said cell phone in the shape of a stylus; and

g. said cell phone for hand held use.

14. A device according to claim 8 selected from the group consisting of

a. said surface comprising a writing surface;

b. said movable element comprising a writing means for writing on said surface;

c. said surface comprising a substantially two dimensional planar face;

d. said detector means comprises an array of light sensitive elements for detecting said position-related means and for generating said at least one output signal thereof;

e. said processing means comprising means for receiving and processing said at least one output signal form said detector means, thereby to determine the position and movement of said movable element relative to said surface;

f. said moveable element comprising said detector means, whereby said movable element is movable relative to said surface;

g. means for determining the path of said moveable element by detecting said position-related means;

h. means for determining the path of said moveable element by detecting a sequence of said position-related means;

i. means for analyzing the path of said moveable element;

j. said position-related means comprising position-related data;

k. said position-related means is printed on said surface;

l. said position-related means is written on said surface;

m. said position-related means comprising existing features on said surface;

n. said position-related means comprising existing features obtainable by said detector means;

O. at least part of said position-related means is outside said surface;

p. said surface comprising a writing surface and said movable element comprising a writing means for writing on said surface and whereby said writing means comprises said at least part of said position-related means;

q. said apparatus intended for use with a computer;

r. said apparatus comprising a computer display;

s. said apparatus comprising a computer printer;

t. said apparatus comprising a computer;

u. said apparatus comprising a digital camera;

v. said apparatus comprising writing means for writing on said surface;

w. said apparatus comprising a microcomputer;

x. said apparatus comprising a display;

y. said apparatus comprising function keys;

z. said apparatus comprising an illumination means;

aa. said processing means comprising a computer;

bb. said apparatus comprising a writing means for writing on said surface and further comprising an original hard copy means for forming an original hard copy made by said writing means on said surface;

cc. said surface made of a material selected from the group consisting of paper, plastic, glass, metal, synthetic fiber, synthetic material, natural material, and a paper like substance;

dd. said position-related means comprising a reflecting means for reflecting said position-related means to said detector means;

ee. said moveable element selected from the group consisting of a stylus shaped moveable element for hand held use, a hand held moveable element, a cell phone and a stylus shaped cell phone;

ff. said surface comprising an overlay means for overlaying said surface with said position-related means;

gg. said surface comprising an overlay means for overlaying said surface with said position-related means selected from the group consisting of carbon paper, film, template, plate and sheet;

hh. said position-related means comprising a selective reflecting means for selectively reflecting at least one selected frequency of light;

ii. said detector means comprising a selective detecting means for selectively detecting at least one selected frequency of light;

jj. said detector means comprising a selective detecting means for selectively detecting said position-related means;

kk. said apparatus comprising a pressure switch means for turning on said apparatus when pressure is applied to said movable element;

ll. said apparatus comprising a focusing means for focusing said surface;

mm. said movable element comprising a writing means for writing on said surface;

nn. said movable element comprising a self contained optical stylus, a writing means for writing on said surface, a microcomputer, a user interface means for communicating with a user and a device interface means for communicating with other devices;

oo. said detector means selected from the group consisting of said detector means mounted on said movable element;

said detector means mounted on a computer monitor, said detector means mounted on a portable computer, said detector means mounted to detect said position-related means;

pp. said detector means selected from the group consisting of an upper detector means, a lower detector means, an upper detector means and a lower detector means mounted in the same direction, an upper detector means and a lower detector means mounted in opposite directions;

qq. said apparatus further comprising an element selected from the group consisting of a track ball, a pressure sensor, a transducer, a gyroscope, and an accelerometer;

rr. said apparatus comprising a cell phone in the shape of a stylus further comprising a mouth piece, an ear piece, a display, function keys, a pen, a pen retractor, a keypad, a digital camera, memory, a micro computer, power supply, and a writing element;

ss. said processing means adapted to produce a digital duplicate of visible marks made on said surface,

tt. said apparatus comprising macro-triangulating means and micro-triangulating means for determining the position and movement of said moveable element;

uu. said apparatus comprising handwriting recognition means for converting handwritten text into a “

keyboard-typed”

representation;

vv. said surface comprising a repetitive pattern that is made unique by marking said repetitive pattern with said movable element comprising a writing element;

ww. said detector means comprising an upper digital camera selected from the group consisting of a video camera, a still camera, video conferencing camera and an internet camera;

xx. said detector means comprising a lower digital camera selected from the group consisting of a video camera, a still camera, video conferencing camera and an internet camera;

yy. said detector means comprising a lower digital camera that can function as an optical computer mouse;

zz. said apparatus comprising an “

ink-well”

like interface for said detector means to communicate with other devices using means selected form the group consisting of optical means, wireless means and electronic means;

aaa. said detector means comprising a digital camera and further comprising mirrors mounted at angles to detect the position of said movable element using triangulation, pattern-matching and/or position-related code;

bbb. said detector means comprising a digital camera and further comprising a split-screen to capture different views of said surface whereby said different views are calibrated and compared with each other to determine movement and position of the writing element;

ccc. said apparatus comprising two digital cameras to capture different views of the surface whereby the frames of said two digital cameras are calibrated and compared with each other to determine movement and position of said movable element.

16. An apparatus according to claim 15 whereby

a. said position-related means comprises at least three points,

b. said at least three points comprising at least two fixed points and at least one movable point,

c. said movable element comprises said at least one movable point, and

d. said apparatus further comprises triangulating means for determining the position and movement of said moveable element.

17. An apparatus according to claim 15 whereby

a. said position-related means comprises at least one pattern, and

b. said apparatus further comprises pattern-matching means for determining the position and movement of said moveable element.

18. An apparatus according to claim 15 whereby said position-related means comprises position-related code means designating two dimensional coordinates of at least one point for determining the position and movement of said moveable element.

19. An apparatus according to claim 15 comprising means for obtaining and outputting the position and movement of a moveable element on a surface selected from the group consisting of

a. triangulating means,

b. pattern-matching means,

c. position-related code means, and

d. any combination thereof.

20. An apparatus according to claim 15 selected from the group consisting of

a. said surface comprising a writing surface;

b. said movable element comprising a writing means for writing on said surface;

c. said surface comprising a substantially two dimensional planar face;

d. said sensor means comprising an array of light sensitive elements for sensing said position-related means and for generating said at least one output signal thereof;

e. said processing means comprising means for receiving and processing said at least one output signal form said position sensor means, thereby to determine the position and movement of said movable element relative to said surface;

f. said moveable element comprising said position sensor means, whereby said movable element is movable relative to said surface;

g. means for determining the path of said moveable element by sensing said position-related means;

h. means for determining the path of said moveable element by sensing a sequence of said position-related means;

i. means for analyzing the path of said moveable element;

j. said position-related means comprising position-related data;

k. said position-related means is printed on said surface;

l. said position-related means is written on said surface;

m. said position-related means comprising existing features on said surface;

n. said position-related means comprising existing features obtainable by said position sensor means;

o. at least part of said position-related means is outside said surface;

p. said surface comprising a writing surface and said movable element comprising a writing means for writing on said surface and whereby said writing means comprises said at least part of said position-related means;

q. said apparatus intended for use with a computer;

r. said apparatus comprising a computer display;

s. said apparatus comprising a computer printer;

t. said apparatus comprising a computer;

u. said apparatus comprising a digital camera;

v. said apparatus comprising writing means for writing on said surface;

w. said apparatus comprising a microcomputer;

x. said apparatus comprising a display;

y. said apparatus comprising function keys;

z. said apparatus comprising an illumination means;

aa. said processing means comprising a computer;

bb. said apparatus comprising a writing means for writing on said surface and further comprising an original hard copy means for forming an original hard copy made by said writing means on said surface;

cc. said surface made of a material selected from the group consisting of paper, plastic, glass, metal, synthetic fiber, synthetic material, natural material, and a paper like substance;

dd. said position-related means comprising a reflecting means for reflecting said position-related means to said position sensor means;

ee. said moveable element selected from the group consisting of a stylus shaped moveable element for hand held use, a hand held moveable element, a cell phone and a stylus shaped cell phone;

ff. said surface comprising an overlay means for overlaying said surface with said position-related means;

gg. said surface comprising an overlay means for overlaying said surface with said position-related means selected from the group consisting of carbon paper, film, template, plate and sheet;

hh. said position-related means comprising a selective reflecting means for selectively reflecting at least one selected frequency of light;

ii. said position sensor means comprising a selective sensing means for selectively sensing at least one selected frequency of light;

jj. said position sensor means comprising a selective sensing means for selectively sensing said position-related means;

kk. said apparatus comprising a pressure switch means for turning on said apparatus when pressure is applied to said movable element;

ll. said apparatus comprising a focusing means for focusing said surface;

mm. said movable element comprising a writing means for writing on said surface;

nn. said movable element comprising a self contained optical stylus, a writing means for writing on said surface, a microcomputer, a user interface means for communicating with a user and a device interface means for communicating with other devices;

oo. said position sensor means selected from the group consisting of said position sensor means mounted on said movable element;

said position sensor means mounted on a computer monitor, said position sensor means mounted on a portable computer, said position sensor means mounted to sense said position-related means;

pp. said position sensor means selected from the group consisting of an upper position sensor means, a lower position sensor means, an upper position sensor means and a lower position sensor means mounted in the same direction, an upper position sensor means and a lower position sensor means mounted in opposite directions;

qq. said apparatus further comprising an element selected from the group consisting of a track ball, a pressure sensor, a transducer, a gyroscope, and an accelerometer;

rr. said apparatus comprising a cell phone selected from the group consisting of

said cell phone comprising a mouthpiece, an earpiece, a display, function keys, a pen, a pen retractor, a keypad, a lower digital camera, memory, a microcomputer, power supply and a writing element;

said cell phone comprising a mouthpiece, an earpiece, a display, function keys, a pen, a pen retractor, a keypad, a lower digital camera, memory, a microcomputer, power supply, a writing element, and a lower sensor selected from the group consisting of a track ball, pressure sensor, transducer, gyroscope, and an accelerometer;

said cell phone comprising a portable video camera further comprising a mouthpiece that functions as a microphone, an earpiece that functions as a speaker and a display that functions as a camera display;

said cell phone comprising a portable video camera further comprising a mouthpiece that functions as a microphone, an earpiece that functions as a speaker and a display that functions as a camera display whereby the camera display flips open and twists for viewing and exposing function keys;

said cell phone whereby said cell phone flips open for viewing and exposing function keys and a display;

said cell phone in the shape of a stylus; and

said cell phone for hand held use;

ss. said processing means adapted to produce a digital duplicate of visible marks made on said surface,

tt. said apparatus comprising macro-triangulating means and micro-triangulating means for determining the position and movement of said moveable element;

uu. said apparatus comprising handwriting recognition means for converting handwritten text into a “

keyboard-typed”

representation;

vv. said surface comprising a repetitive pattern that is made unique by marking said repetitive pattern with said movable element comprising a writing element;

ww. said position sensor means comprising an upper digital camera selected from the group consisting of a video camera, a still camera, video conferencing camera and an internet camera;

xx. said position sensor means comprising a lower digital camera selected from the group consisting of a video camera, a still camera, video conferencing camera and an internet camera;

yy. said position sensor means comprising a lower digital camera that can function as an optical computer mouse;

zz. said apparatus comprising an “

ink-well”

like interface for said position sensor means to communicate with other devices using means selected form the group consisting of optical means, wireless means and electronic means;

aaa. said position sensor means comprising a digital camera and further comprising mirrors mounted at angles to sense the position of said movable element using triangulation, pattern-matching and/or position-related code;

bbb. said position sensor means comprising a digital camera and further comprising a split-screen to capture different views of said surface whereby said different views are calibrated and compared with each other to determine movement and position of the writing element;

ccc. said apparatus comprising two digital cameras to capture different views of the surface whereby the frames of said two digital cameras are calibrated and compared with each other to determine movement and position of said movable element.

15. An apparatus comprising position sensor means for sensing position on a surface further comprising:

a. a position-related means for designating position information,

b. a movable element,

c. a processing means for receiving and processing said at least one output signal, thereby to determine the position of said movable element relative to said surface,

Whereby said position sensor means senses said position-related means and generates at least one output signal thereof.