WIRELESS POSITION SENSING IN THREE DIMENSIONS USING ULTRASOUND

• US 20090187374A1
• Filed: 01/22/2009
• Published: 07/23/2009
• Est. Priority Date: 01/22/2008
• Status: Active Grant

• Current Assignee: Performance Designed Products LLC
• Original Assignee: Performance Designed Products LLC
• Inventors: Lavache, Steven, Baxter, Richard
• Granted Patent: US 8,639,471 B2
• Time in Patent Office: Days
• US Class Current: 702/152
• CPC Class Codes: G01S 11/16 using difference in transit time between electrical and acoustic signals
• CPC Class Codes: G01S 5/16 using electromagnetic waves other than radio waves
• CPC Class Codes: G01S 5/18 using ultrasonic, sonic, or infrasonic waves
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0325 using a plurality of light emitters or reflectors or a plurality of detectors forming a reference frame from which to derive the orientation of the object, e.g. by triangulation or on the basis of reference deformation in the picked up image
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/043 using propagating acoustic waves

Abstract

According to various embodiments, the invention provides a method for establishing communication between a computer program in a computing system and a controller held in a hand of a user, comprising: capturing image data in a capture area in front of a display, the capturing configured to identify a two-dimensional location of the controller in the capture area; and capturing one-way sound communication between a location proximate to the display and a location at the controller, the sound communication being used to compute a third-dimensional location of the controller in the capture area.

Claims

2. A system for establishing communication between a computer program and a controller, comprising:

an image capture device for capturing image data in a capture area in front of a display;

a sound capture device for capturing sound data in the capture area; and

a computing system for executing the computer program, the computing system being connected to the display, the image capture device, and the sound capture device;

the controller to be held by a user with a single hand, the controller including;

a body with a first end and a second end;

a grip area at about the first end;

an input pad defined between the first end and the second end for entering input data to be transmitted to the computing system;

a spherical-type section defined at the second end for facilitating image recognition by the image capture device; and

a sound emitting device near the second end, the sound emitting device directed away from the user when held and configured for one-way sound communication with the sound capture device;

wherein

the computer program identifies a two-dimensional location of the controller based on the captured image data and a third-dimensional location of the controller based on the captured sound data.

3. A system for establishing communication between a computer program and a controller, comprising:

a first set and a second set of light emitters facing a capture area in front of a display, the first and second set of light emitters located near the display;

a sound capture device for capturing sound data in the capture area; and

a computing system for executing the computer program, the computing system being connected to the display, and the sound capture device;

the controller to be held by a user with a single hand, the controller including;

a body with a first end and a second end;

a grip area at about the first end;

an input pad defined between the first end and the second end for entering input data to be transmitted to the computing system;

an image capture device near the second end for capturing image data of an area where the first and second set of light emitters are located; and

a sound emitting device near the second end, the sound emitting device directed away from the user when held and configured for one-way sound communication with the sound capture device;

wherein

the computer program identifies a two-dimensional location of the controller based on the captured image data and a third-dimensional location of the controller based on the captured sound data.

4. A computer program embedded in a computer-readable storage medium, when executed by one or more processors, for establishing communication between a computer program in a computing system and a controller held in a hand of a user, the computer program comprising:

program instructions for capturing image data in a capture area in front of a display, the capturing configured to identify a two-dimensional location of the controller in the capture area; and

program instructions for capturing one-way sound communication between a location proximate to the display and a location at the controller, the sound communication being used to compute a third-dimensional location of the controller in the capture area.

5. A method for determining the position of a mobile controller, the position to be determined relative to a fixed unit, wherein the mobile controller comprises an IR light source and an ultrasonic transducer, wherein the fixed unit comprises a camera and a first ultrasonic transducer, the method comprising the steps of:

determining the angular location of the IR light source relative to the fixed unit using the camera;

determining the distance between the mobile controller and the fixed unit by measuring the time of flight of ultrasound between them; and

determining the 3D position of the controller, relative to the fixed unit, by combining the angular location with the distance.

1. A method for establishing communication between a computer program in a computing system and a controller held in a hand of a user, comprising:

capturing image data in a capture area in front of a display, the capturing configured to identify a two-dimensional location of the controller in the capture area; and

capturing one-way sound communication between a location proximate to the display and a location at the controller, the sound communication being used to compute a third-dimensional location of the controller in the capture area.

6. A method according to claim 5, further comprising:

altering the illumination of the IR light source on receipt of a signal by the mobile controller from the fixed unit; and

distinguishing the IR light source from other IR light sources by detecting the altered illumination of the IR light source using the camera.

7. A method according to claim 5, wherein the fixed unit further comprises a second ultrasonic transducer, the method further comprising:

determining an approximate location for the mobile component by comparing the time of flight of ultrasound between the mobile controller and the first and second ultrasonic transducers respectively; and

comparing the approximate location to the image detected by the camera to determine which IR light source belongs to the mobile controller.

8. A method according to claim 5, further comprising:

tracking the course of the IR light source detected by the camera over time; and

using the course to distinguish the IR light source from background light sources when they are close to each other.

9. A method according to claim 5, further comprising:

averaging the time of flight of ultrasound between the mobile controller and fixed unit over two or more readings, for reducing the effect of errors on the determined distance.

10. A method according to claim 9, wherein the mobile controller further comprises an accelerometer, the method comprising:

determining the speed of the mobile controller using the accelerometer; and

reducing the number of readings averaged when the controller is moving quickly.

11. A method according to claim 5, further comprising:

correcting the image detected by the camera to remove distortion caused by the camera.

12. A method according to claim 5, wherein the fixed unit further comprises means for sensing the angle of tilt of the camera, the method comprising:

detecting the angle of tilt of the camera from an upright position; and

correcting the information determined from the images captured by the camera for the angle of tilt of the camera.

13. A method according to claim 5, further comprising filtering the measured distance readings by:

comparing a reading to be tested against the last valid reading to determine an average speed; and

rejecting the reading to be tested if the average speed is above a maximum allowable speed.

13-1. A method according to claim 5, further comprising filtering the measured distance readings by:

comparing a reading to be tested against the last two valid reading to determine an average acceleration; and

rejecting the reading to be tested if the average speed is above a maximum allowable acceleration.

14. A method according to claim 5, further comprising filtering the measured distance readings by:

calculating the tangential speed of the mobile controller by measuring its movement with the camera and taking as its range the last valid distance reading;

comparing a reading to be tested against the last valid reading to determine an average radial speed;

calculating an average overall speed by combining the calculated tangential speed and average radial speed; and

rejecting the reading to be tested if the average speed is above a maximum allowable speed.

14-2. A controller for use with a position sensing apparatus, the controller comprising an IR light source, an ultrasonic transducer and an RF transmitter, the controller being adapted so that in use the IR light source is illuminated and the RF transmitter is used to send information about ultrasonic signals detected by the ultrasonic transducer.

15. Position sensing apparatus comprising an ultrasonic transmitter, an ultrasonic receiver, an infrared transmitter, a 2-dimensional infrared receiver such as a camera, a timer and trigger means for activating the timer, the apparatus being divided into a mobile and a fixed component separate from one-another and the ultrasonic and infrared components being located adjacent one-another in heterogeneous pairs on each respective component, the apparatus further including image processing means for processing the image received by the infrared receiver and control means for coordinating the functions of the said receivers, transmitters and timer, the apparatus being arranged in use to transmit an ultrasonic pulse between the ultrasonic transmitter and receiver and substantially simultaneously to start the timer, the timer being stopped when the ultrasonic pulse has been received thereby measuring the time of flight of the ultrasonic energy between the ultrasonic transmitter and receiver and the apparatus further being arranged to determine a 2-dimensional relative angular position of the infrared transmitter and receiver by processing the image received by the infrared receiver, whereby the ultrasonic distance measurement and infra-red angular measurements together provides a 3-dimensional position fix.

16. Position sensing apparatus according to claim 15, further comprising a second infrared transmitter in a known spatial relationship to the first infrared transmitter, the image processing means being adapted to distinguish the first and second infrared transmitters from the background by searching for the known relationship between two detected infrared sources.

17. Position sensing apparatus according to claim 15, wherein the infrared transmitter is of a known size, the image processing means being adapted to calculate an approximate range for the infrared transmitter by measuring its detected size, the apparatus being arranged to compare the approximate range with the ultrasonic distance measurement to validate the received infrared source as being the infrared transmitter.