Three dimensional pointing device monitored by computer vision

• US 5,297,061 A
• Filed: 05/19/1993
• Issued: 03/22/1994
• Est. Priority Date: 05/19/1993
• Status: Expired due to Fees

• Current Assignee: University of Maryland
• Original Assignee: University of Maryland
• Inventors: Dementhon, Daniel F., Fujii, Yukio
• Primary Examiner(s): Teska, Kevin J.
• Application Number:
  US08/063,489
• Time in Patent Office: 307 Days
• Field of Search: 364/559, 364/514, 364/516, 364/424.02, 340/706
• US Class Current: 345/180
• CPC Class Codes: G06F 3/0325 using a plurality of light emitters or reflectors or a plurality of detectors forming a reference frame from which to derive the orientation of the object, e.g. by triangulation or on the basis of reference deformation in the picked up image
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors

Abstract

A pointing device (28), comprising light sources (30) and monitored by a computer vision task running in a microcontroller (108) and a computer (24). The computer vision task computes the spatial position and orientation of the pointing device, and enables an operator to control virtual three dimensional objects (73) on the display (22) of the computer. Images (78) are captured by a video camera (20) and digitized, and only image rows that contain bright pixels (80) from the light sources are processed. The light sources are the tips of optic fibers (30) guiding light from a laser diode (32), and an optical filter (46) on the camera is matched to the wavelength of the laser diode.

Claims

1. In an apparatus for monitoring the position and orientation of a rigid three-dimensional (3-D) object in space comprising:

an electronic camera having a reference coordinate frame and producing an analog video signal, the analog video signal being composed of scanning lines, each scanning line corresponding to a single line of an image of the electronic camera;

a plurality of light sources rigidly attached to the rigid 3-D object, these lights projecting bright spots onto the image of the camera;

processing means for processing the analog video signal and determining a list of positions of the bright spots;

computing means including memory means, output means and display means;

the computing means including pose computing means for computing a translation vector and a rotation matrix of the rigid 3-D object in the reference coordinate frame of the electronic camera and transmitting these quantities to the output means;

the output means modifying images on the display means;

the improvement comprising in the processing means;

a first electronic means for generating a timing signal synchronous to the beginning of the scanning line,

a clock generator for producing a clock signal,

a second electronic means for digitizing the scanning line into a string of digitized values, at the rate of the clock signal,

thresholding means for thresholding the string of digitized values into a string of thresholded digitized values in which values higher than a given threshold value are set to a logical high value and values lower than a given brightness threshold value are set to a logical low value,

a third electronic means for producing a flag signal which is set to a logical high value when all elements of the string of thresholded digitized values are set to a logical low value and which is reset by the synchronous timing signal,

a counter driven by the clock signal and reset by the synchronous timing signal for generating address numbers,

a second memory means for;

i) storing strings of thresholded digitized values to memory cells addressed by the address numbers, and (ii) storing an additional data when the flag signal is set to a logical high value,

and the processing means further including means for;

i) recalling the strings of thresholded digitized values from the memory cells and skipping a string of thresholded digitized values when the additional data indicates that these digitized values are all set to a logical low value, ii) calculating from recalled strings of thresholded digitized values the positions of centers of the bright spots, and iii) computing the position and orientation of the object from the positions of the centers of the bright spots.

2. The improvement of the apparatus of claim 1, wherein the brightness threshold value is provided by the processing means.

3. The improvement of the apparatus of claim 1, further comprising a direct signal path between the second electronic means for digitizing the scanning lines and the second memory means, whereby the digitized video signal is stored into the second memory means without being subjected to the thresholding means.

4. The apparatus of claim 3, wherein the threshold value is provided by the processing means.

5. The improvement of the apparatus of claim 1, further comprising the light sources which are secondary light sources obtained by guiding light produced by a primary light source through optic fibers.

6. The apparatus of claim 5 wherein the means for attaching the light sources to the object is a hand held structure.

7. The apparatus of claim 5 wherein the device is battery powered.

8. The improvement of the apparatus of claim 1, wherein the display means includes control of a virtual cursor on the display means which is a part of a computer display, the computing means further including means for computing the virtual cursor at a position transformed from the translation vector of the light sources such that the virtual cursor is displayed in a vicinity of an edge of the computer display when the light sources are positioned in a vicinity of a boundary of a field of view of the camera, whereby the transformed position of the virtual cursor on the computer display provides visual feedback about the proximity of the light sources to the boundary of the field of view of the camera.

9. In an apparatus for monitoring the position and orientation of a rigid three-dimensional (3-D) object in space comprising:

an electronic camera having a reference coordinate frame and producing an analog video signal, the analog video signal being composed of scanning lines, each scanning line corresponding to a single line of an image of the electronic camera;

a plurality of light sources rigidly attached to the rigid 3-D object, these lights projecting bright spots onto the image of the camera;

processing means for processing the analog video signal and determining a list of positions of the bright spots;

computing means including memory means, output means and display means;

the computing means including pose computing means for computing a translation vector and a rotation matrix of the rigid 3-D object in the reference coordinate frame of the electronic camera and transmitting these quantities to the output means;

the output means modifying images on the display means;

wherein the improvement comprising in the processing means;

a first electronic means for generating a timing signal synchronous to the beginning of the scanning line,

a clock generator for producing a clock signal,

a second electronic means for digitizing the scanning line into a string of digitized values, at the rate of the clock signal,

thresholding means for thresholding the strings of digitized values into strings of thresholded digitized values in which the values higher than a given brightness threshold value are set to a logical high value and the values lower than a given threshold value are set to a logical low value,

a third electronic means for producing an edge timing pulse corresponding to an occurrence of transition from a logical low value to a logical high value and from a logical high value to a logical low value in the strings of thresholded digitized values,

a counter driven by the clock signal and reset by the synchronous signal for generating an address number,

a second memory means for sequentially storing the address number at each occurrence of the edge timing pulse,

the processing means further including means for;

i) recalling each address number from the memory means and for calculating from all the address numbers the positions of centers of the bright spots, and ii) computing the position and orientation of the object from the positions of centers of the bright spots, and iii) controlling the brightness threshold value.

10. The improvement of the apparatus of claim 9, wherein the display means includes control of a virtual cursor on the display means which is a part of a computer display, the computing means further including means for computing the virtual cursor at a position transformed from the translation vector of the light sources such that the virtual cursor is displayed in a vicinity of an edge of the computer display when the light sources are positioned in a vicinity of a boundary of a field of view of the camera, whereby the transformed position of the virtual cursor on the computer display provides visual feedback about the proximity of the light sources to the boundary of the field of view of the camera.

11. The improvement of the apparatus of claim 9, further comprising the light sources which are secondary light sources obtained by guiding light produced by a primary light source through optic fibers.

12. The apparatus of claim 9 wherein the means for attaching the light sources to the object is a hand held structure.

13. The apparatus of claim 9 wherein the device is battery powered.