Drawing, writing and pointing device

• US 20030178493A1
• Filed: 03/05/2003
• Published: 09/25/2003
• Est. Priority Date: 09/11/2000
• Status: Active Grant

• Current Assignee: Seiko Epson Corporation (Seiko Group)
• Original Assignee: New Index AS (Seiko Group)
• Inventors: Njolstad, Tormod
• Granted Patent: US 7,083,100 B2
• Time in Patent Office: Days
• US Class Current: 235/472.03
• CPC Class Codes: G06F 3/014 Hand-worn input/output arrangements, e.g. data gloves
• CPC Class Codes: G06F 3/0325 using a plurality of light emitters or reflectors or a plurality of detectors forming a reference frame from which to derive the orientation of the object, e.g. by triangulation or on the basis of reference deformation in the picked up image
• CPC Class Codes: G06F 3/03545 Pens or stylus
• CPC Class Codes: G06F 3/042 by opto-electronic means
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06F 3/14 Digital output to display device ; Cooperation and interconnection of the display device with other functional units
• CPC Class Codes: G09G 3/002 to project the image of a two-dimensional display, such as an array of light emitting or modulating elements or a CRT

Abstract

Drawing, writing or pointing device (1) for use with data presentations, computer supported work or other interactive use of a computer. The device (1) comprises one or more code patterns (3). Each code pattern (3) is adapted to be detected by a camera and recognition system (13). A camera and recognition system (13) adapted to cooperate with the device (1) records one or more images (31). At least one image contains one or more code patterns (3) from the device(s). Drawing, writing and pointing devices (1) are used in a system (40) and in a method for drawing, writing and pointing at data presentations, computer supported work and other interactive use of a computer where the position and orientation of the device (1) is determined and transferred to a computer. The drawing, writing and pointing device (1) is also used in a method for continuously recording the position and orientation of one or more marked objects, where the marking comprises one or more code patterns (3).

Claims

1. Drawing, writing or pointing device (1) for data presentations, computer supported work or other interactive use of a computer,

characterized in that the device (1) comprises one or more code patterns (3) adapted to be detected by a camera and recognition system.

2. Device according to claim 1,

wherein the device (1) comprises one or two ends (2) with code patterns (3).

3. Device according to claim 1,

designed as a pointing glove (6) adapted to be placed on one or more fingers or a hand.

4. Device according to claim 3,

wherein the code patterns (3) are placed on the fingertip(s) (7), joints or knuckles of the pointing glove (6).

5. Device according to claim 1,

wherein one or both ends of the device has a triangular, conical or fasetted pointed end (2) comprising code patterns (3) in order to ensure continuous detection of the code pattern and the position and orientation of the device.

6. Device according to one of the claims 1-5,

comprising an activation mechanism/device (8) such that the code patterns (3) may be activated-/controlled by the user of the device.

7. Device according to claim 6,

where the activation mechanism/device (8) comprises additional code patterns (33) partially maskable by the user and/or additional code patterns (34) which may be used to determine left or right rotation of the device (1) around its own axis, in order to signal and activate the selection made by the user.

8. Device according to claim 6,

where the activation mechanism/device (8) comprises a control device (29) placed e.g. on a remote control unit (28), on a keyboard of a computer (18) or on the device (1) itself.

9. Device according to claim 6,

where the activation mechanism/device (8) comprises a compressible air chamber (9) which upon compression displaces an inner cylinder with a tip (10), such that the code pattern (3) on a partially transparent outer tip (2) obtains a contrast coloured background for easier detection.

10. Device according to claim 6,

where the activation mechanism/device (8) comprises an inner tip (10) and an outer cover (2) axially displacable relative to each other upon activation, and that the code pattern (4) on the inner tip and a stripe pattern (5) on the outer cover in an initial position are aligned right above each other, i.e. in register, forming a resulting code pattern (6), and where upon displacement of the inner tip or the outer cover, e.g. the distance of half a stripe, another resulting code pattern (7) appears.

11. Device according to claim 6

where one end of the activation mechanism/device comprises a spring suspended tip with additional code patterns (33), the tip beeing mechanically coupled to a through-going centre pin of the device such that the tip and the centre pin are displaced along the centre line of the device (1) and that the effective stripe pattern in one end of the device is altered when the tip is pressed towards a surface.

12. Device according to claim 11,

wherein the other end of the centre pin comprises a tip with additional code patterns (33) so that the effective stripe pattern in the other end of the device (1) is altered when the tip is pressed against a surface.

13. Device according to claim 6

where the activation mechanism/device (8) comprises one or more infrared light emitting diodes (11) arranged individually or in one or more rows, with corresponding battery(ies) (21), where the diode(s) upon activation causes different patterns to appear on the partially transparent outer tip (2).

14. Device according to claim 13

comprising also means (12) for modulating the infrared light, so that the resulting modulated light contains information regarding pen type, colour, user or other similar information or a combination of more of these characteristics so that the desired function and user may be identified.

15. Device according to claim 6

where the activation mechanism/function (8) comprises means (12,32,19) for transferring radio signals adapted to exchange information on e.g. pen type, colour, users or combinations of more of the characteristics.

16. Camera and recognition system (13) for cooperation with the drawing, writing or pointing device(s) (1) at data presentations, computer supported office work or other interactive use of a computer according to one or more of the claims 1-15,

characterized in that the camera is adapted to record one or more images (31) wherein at least one image contains one or more code paterns (3) from on or more drawing, writing or pointing devices (1)

17. System according to claim 16,

comprising position determining means (15) for the determination of the position of the devices (1) in one or more images (31).

18. System according to claim 17

where the position determining means (15) comprises filtering means (16) for the filtering of the points in the images (31).

19. System according to claim 17

where the position determining means (15) comprises filtering means (16) for filtering of horisontal lines in the images (31).

20. System according to claim 16 or 19,

where the filtering means (16) comprises filters, preferably matched filters, adapted to the code patterns (3), and subsequent maximum value detection, thereby identifying the positions and/or orientations of the pattern(s) (3) of the device (1).

21. System according to one of the claims 16-20

comprising data transferring means (17) for transferring signals to a computer (18) in that the signals comprise information regarding the posistion and/or orientation of the device(s).

22. System according to claim 21,

where the signals are in a standard format of input data to a computer, in particular such as from a computer mouse device.

23. System for drawing, writing and pointing suitable for data presentations and computer supported office work comprising

drawing, writing or pointing devices (1),

camera and recognition devices (13),

computer devices (18,20) with display unit(s) (20) comprising at least one projection screen, computer display, flat screen or virtual screen (14)

characterized in that

the camera (13) is adapted to record one or more images (31) in which at least one image contains one or more code patterns from one or more drawing, writing or pointing device(s) (1,6),

the tip(s) (2) of the drawing, writing or pointing device(s) (1, 6) comprising one or more code patterns (3) which are detectable and whose position(s) can be determined by the camera and recognition system (13), and that

position determining information is transferred continuously to the computer devices (18,20) so that one or more users may draw, write or point at the projection unit (20) in replacement of computer mouse devices or similar devices.

24. Method of drawing, writing or pointing at data presentations, computer supported office work or other interactive use of a computer comprising the following steps:

displaying one or more images (31) on a computer screen, a data or video projection screen, a flat screen or a virtual screen (14) using a computer (18,20) or projector (20),

providing image recordings of the displayed images using a camera and recognition system (13)

characterized in the further steps of

processing the images recorded using the camera and recognition system (13) and/or software in the computer (18,20) and/or the projector (20) in order for in each recorded image to record drawing, writing or pointing movements in the form of changes in the position(s) and/or orientation(s) of drawing, writing or pointing devices (1,6) comprising one or more code patterns (3) in front of the computer/video projection screen, flat screen, or the virtual screen.

25. Method according to claim 24,

characterized in that the processing of the images further comprises

storing the recorded positions and orientations of the drawing, writing and pointing devices in a data storage unit.

26. Method according to claim 24 or 25

characterized in that the processing of the recorded images further comprises

altering the displayed images depending on the recorded positions and orientations of the drawing, writing or pointing device(s).

27. Method according to one of the claims 24-26

characterized in that the processing of the recorded images further comprises

using a multiple of determined positions and/or orientations of the device(s) (1,6), based on a multiple of recorded images, for the further determination of the direction and/or velocity of movement of the device(s) (1,6).

28. Method according to claim 26 or 27

characterized in that the processing of the recorded images further comprises

altering the appearance or execution of the program in the computer depending on one or more of;

the recorded positions, orientations, directions of movement or velocities of movement of the devices (1,6).

29. Method for continuously recording the position and orientation of one or more marked objects in an area comprising the following steps:

providing recorded images of the area using a camera and recognition system (13)

characterized in the further steps of

processing the recorded images using the camera and recognition system (13) and/or software in the computer (18,20), in order to in each image to record the position(s) and/or orientation(s) of one or more marked objects, the marking comprising one or more code patterns.

30. Method according to claim 29,

characterized in that the processing of the recorded images further comprises

storing the recorded positions and orientations of one or more marked objects where the marking comprises one or more code patterns in a data storage unit.

31. Method according to one of the claims 29-30,

characterized in that the processing of the images further comprises

using a multiple of recorded positions and orientations of the marked objects, based on a multiple of recorded images, for the further calculation of the direction and/or velocity of movement of the marked objects.

32. Method according to claim 30 or 31,

characterized in that the processing of the recorded images further comprises

altering the appearance or execution of the program in the computer depending on one or more of;

the recorded positions, orientations, directions of movement or velocities of movement of the marked objects.