Drawing, writing and pointing device

• US 7,083,100 B2
• Filed: 09/07/2001
• Issued: 08/01/2006
• Est. Priority Date: 09/11/2000
• Status: Expired due to Term

• Current Assignee: Seiko Epson Corporation (Seiko Group)
• Original Assignee: New Index AS (Seiko Group)
• Inventors: Njølstad, Tormod
• Primary Examiner(s): Frech, Karl D.
• Application Number:
  US10/363,530
• Publication Number:
  US 20030178493A1
• Time in Patent Office: 1,789 Days
• Field of Search: 235/462.13, 235/462.45, 235/472.01, 235/462.01, 235/472.02, 235/472.03
• US Class Current: 235/462.13
• CPC Class Codes: G06F 3/014 Hand-worn input/output arrangements, e.g. data gloves
• CPC Class Codes: G06F 3/0325 using a plurality of light emitters or reflectors or a plurality of detectors forming a reference frame from which to derive the orientation of the object, e.g. by triangulation or on the basis of reference deformation in the picked up image
• CPC Class Codes: G06F 3/03545 Pens or stylus
• CPC Class Codes: G06F 3/042 by opto-electronic means
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06F 3/14 Digital output to display device ; Cooperation and interconnection of the display device with other functional units
• CPC Class Codes: G09G 3/002 to project the image of a two-dimensional display, such as an array of light emitting or modulating elements or a CRT

Abstract

Drawing, writing or pointing device (1) for use with data presentations, computer supported work or other interactive use of a computer. The device (1) comprises one or more code patterns (3). Each code pattern (3) is adapted to be detected by a camera and recognition system (13). A camera and recognition system (13) adapted to cooperate with the device (1) records one or more images (31). At least one image contains one or more code patterns (3) from the device(s). Drawing, writing and pointing devices (1) are used in a system (40) and in a method for drawing, writing and pointing at data presentations, computer supported work and other interactive use of a computer where the position and orientation of the device (1) is determined and transferred to a computer. The drawing, writing and pointing device (1) is also used in a method for continuously recording the position and orientation of one or more marked objects, where the marking comprises one or more code patterns (3).

Claims

1. Drawing, writing or pointing device (1) for data presentations, computer supported work or other interactive use of a computer,

characterized in that the device (1) exhibits one or more code patterns (3) adapted to be remotely detected by a camera and recognition system.

2. Device according to claim 1, wherein the device (1) comprises one or two ends (2) with code patterns (3).

3. Device according to claim 1, designed as a pointing glove (6) adapted to be placed on one or more fingers or a hand.

4. Device according to claim 3, wherein the code patterns (3) are placed on the fingertips (7), joints or knuckles of the pointing glove (6).

5. Device according to claim 1, wherein one or both ends of the device has a triangular, conical or fasetted pointed end (2) comprising code patterns (3) in order to ensure continuous detection of the code pattern and the position and orientation of the device.

6. Device according to claim 1, comprising an activation mechanism/device (8) such that the code patterns (3) may be activated/controlled by the user of the device.

7. Device according to claim 6, where the activation mechanism/device (8) activates additional code patterns (33) partially maskable by the user and/or additional code patterns (34) which may be used to determine left or right rotation of the device (1) around its own axis, in order to signal and activate the selection made by the user.

8. Device according to claim 6, where the activation mechanism/device (8) comprises a control device (29) placed on a remote control unit (28), on a keyboard of a computer (18) or on the device (1) itself.

9. Device according to claim 6, where the activation mechanism/device (8) comprises a compressible air chamber (9) which upon compression displaces an inner cylinder with a tip (10), such that the code pattern (3) on a partially transparent outer tip (2) obtains a contrast coloured background for easier detection.

10. Device according to claim 6, where the activation mechanism/device (8) comprises an inner tip (10) and an outer cover (2) axially displacable relative to each other upon activation, and that the code pattern (4) on the inner tip and a stripe pattern (5) on the outer cover in an initial position are aligned right above each other, forming a resulting code pattern (6), and where upon displacement of the inner tip or the outer cover, another resulting code pattern (7) appears.

11. Device according to claim 6 where one end of the activation mechanism/device comprises a spring suspended tip with additional code patterns (33), the tip being mechanically coupled to a through-going centre pin of the device such that the tip and the centre pin are displaced along the centre line of the device (1) and that the effective stripe pattern in one end of the device is altered when the tip is pressed towards a surface.

12. Device according to claim 11, wherein the other end of the centre pin comprises a tip with additional code patterns (33) 50 that the effective stripe pattern in the other end of the device (1) is altered when the tip is pressed against a surface.

13. Device according to claim 6 where the activation mechanism/device (8) comprises at least one infrared light emitting diode (11) arranged individually or in one or more rows, with corresponding battery (21), where the at least one diode upon activation causes different patterns to appear on the partially transparent outer tip (2).

14. Device according, to claim 13 comprising also means (12) for modulating the infrared light, so that the resulting modulated light contains information regarding pen type, colour, user or other similar information or a combination of more of these characteristics so that the desired function and user may be identified.

15. Device according to claim 6 where the activation mechanism/function (8) comprises means (12,32,19) for transferring radio signals adapted to exchange information.

16. Camera and recognition system (13) for cooperation with at least one drawing, writing or pointing device (1) at data presentations, computer supported office work or other interactive use of a computer according to claim 1,

characterized in that the camera is adapted to record one or more images (31) wherein at least one image contains one or more code patterns (3) from said at least one drawing, writing or pointing device (1).

17. System according to claim 16, comprising position determining means (15) for the determination of the position of the devices (1) in one or more images (31).

18. System according to claim 17 where the position determining means (15) comprises filtering means (16) for the filtering of the points in the images (31).

19. System according to claim 17 where the position determining means (15) comprises filtering means (16) for filtering of horisontal lines in the images (31).

20. System according to claim 16, where the filtering means (16) comprises filters, preferably matched filters, adapted to the code patterns (3), and subsequent maximum value detection, thereby identifying the positions or orientations of the patterns (3) of the device (1).

21. System according to claim 16, comprising data transferring means (17) for transferring signals to a computer (18) in that the signals comprise information regarding the position or orientation of the at least one device.

22. System according to claim 21, where the signals are in a standard format of input data to a computer, in particular such as from a computer mouse device.

23. Device according to claim 1,

comprising means for generating projected code patterns on a screen using a light source and optical elements.

24. Method of drawing, writing or pointing at data presentations, computer supported office work or other interactive use of a computer comprising the following steps:

displaying one or more images (31) on a computer screen, a data or video projection screen, a flat screen or a virtual screen (14) using a computer (18,20) or projector (20);

providing image recordings of the displayed images using a camera and recognition system (13)

characterized in the further steps of

processing the images recorded using the camera and recognition system (13) or software in the computer (18,20) or the projector (20) in order for in each recorded image to record drawing, writing or pointing movements in the form of changes in the position or orientation of drawing, writing or pointing devices (1,6) comprising one or more code patterns (3) in front of the computer/video projection screen, flat screen, or the virtual screen.

25. Method according to claim 24,

characterized in that the processing of the images further comprises

storing the recorded positions and orientations of the drawing, writing and pointing devices in a data storage unit.

26. Method according to claim 24, characterized in that the processing of the recorded images further comprises

altering the displayed images depending on the recorded positions and orientations of the drawing, writing or pointing devices.

27. Method according to claim 26,

characterized in that the processing of the recorded images further comprises

altering the appearance or execution of the program in the computer depending on one or more of;

the recorded positions, orientations, directions of movement or velocities of movement of the devices (1,6).

28. Method according to claim 24, characterized in that the processing of the recorded images further comprises

using a multiple of determined positions or orientations of the devices (1,6), based on a multiple of recorded images, for the further determination of the direction or velocity of movement of the devices (1,6).

29. Method for continuously recording the position and orientation of one or more marked objects in an area comprising the following steps:

providing recorded images of the area with said one or more marked objects using a camera and recognition system (13) such that said image contains information related to the mark on said marked objects

characterized in the further steps of

processing the recorded images of said area using the camera and recognition system (13) or software in the computer (18,20), a part of said image being formed by said one or more marked objects in the area in order to in each image to record the position or orientation of one or more marked objects, the marking comprising one or more code patterns.

30. Method according to claim 29,

characterized in that the processing of the recorded images further comprises

storing the recorded positions and orientations of one or more marked objects where the marking comprises one or more code patterns in a data storage unit.

31. Method according to claim 30, characterized in that the processing of the recorded images further comprises

altering the appearance or execution of the program in the computer depending on one or more of;

the recorded positions, orientations, directions of movement or velocities of movement of the marked objects.

32. Method according to claim 29,

characterized in that the processing of the images further comprises

using a multiple of recorded positions and orientations of the marked objects, based on a multiple of recorded images, for the further calculation of the direction or velocity of movement of the marked objects.

33. Method according to claim 29,

comprising generating projected code patterns on a screen using a light source and optical elements.

34. System for drawing, writing and pointing suitable for data presentations and computer supported office work comprising

drawing, writing or pointing devices (1),

camera and recognition devices (13),

computer devices (18,20) with display units (20) comprising at least one projection screen, computer display, flat screen or virtual screen (14)

characterized in that

the camera (13) is adapted to record one or more images (31) in which at least one image contains one or more code patterns from at least one drawing, writing or pointing device (1,6),

the tip (2) of the at least one drawing, writing or pointing device (1,6) comprising one or more, code patterns (3) which are detectable and whose position(s) can be determined by the camera and recognition system (13), and that position determining information is transferred continuously to the computer devices (18,20) 50 that one or more users may draw, write or point at the projection unit (20) in replacement of computer mouse devices.

35. System according to claim 34,

comprising means for generating projected code patterns on a screen using a light source and optical elements.