Method and device for three-dimensional sensing

• US 8,139,029 B2
• Filed: 03/07/2007
• Issued: 03/20/2012
• Est. Priority Date: 03/08/2006
• Status: Expired due to Fees

• Current Assignee: Navisense
• Original Assignee: Navisense
• Inventors: Boillot, Marc, McIntosh, Jason
• Primary Examiner(s): Awad, Amr
• Assistant Examiner(s): Bukowski, Kenneth
• Application Number:
  US11/683,410
• Publication Number:
  US 20070211022A1
• Time in Patent Office: 1,840 Days
• Field of Search: 345/156, 345/158, 345/173, 345175-177, 178/18.04, 178/18.09, 178/19.02, 178/19.05
• US Class Current: 345/158
• CPC Class Codes: G06F 3/0325 using a plurality of light emitters or reflectors or a plurality of detectors forming a reference frame from which to derive the orientation of the object, e.g. by triangulation or on the basis of reference deformation in the picked up image
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/043 using propagating acoustic waves

Abstract

An apparatus (100) and method is provided that identifies and tracks a relative location and movement of an object in a three-dimensional space. The sensing unit includes a processor (122) for communicating a coordinate information of the object within the three-dimensional space. The method includes emitting a pulse from a first transducer (101), estimating a time of flight from a reflection signal received by a second transducer (102), and determining a location and relative movement of the object from the time of flight measurements. The sensing unit can provide touchless control via touchless finger depression actions, finger slide actions, finger release actions, and finger hold actions.

Claims

1. An apparatus comprising:

a touchless sensing unit to generate a three-dimensional ultrasonic sensing space and track a relative location and movement of an object in the three-dimensional ultrasonic sensing space; and

a controller communicatively coupled to said touchless sensing unit for communicating a coordinate of said object within said three-dimensional space, wherein the sensing unit operates to;

emit an ultrasonic pulse from a first ultrasonic transducer configured to transmit the ultrasonic pulse that reflects off the object in said three-dimensional space and generates a reflected ultrasonic signal;

digitally sample the reflected ultrasonic signal to produce a sampled reflected ultrasonic signal;

suppress a ringing distortion of the sampled reflected ultrasonic signal via digital signal processing

wherein the ringing distortion is a result of the close proximity interference and residual ringing of the emitted ultrasonic pulse;

and wherein the suppressing comprises subtracting a contribution of the ringing portion of the emitted ultrasonic pulse from the reflected signal to remove a ring-down effect;

estimate a time of flight between when said ultrasonic pulse was transmitted from said first ultrasonic transducer and when the suppressed reflected ultrasonic signal was received for a plurality of ultrasonic transducers configured to receive the reflected ultrasonic signal;

calculate a phase differential between the suppressed reflected ultrasonic signal and a previously received reflected ultrasonic signal both received at a same ultrasonic transducer for the plurality of ultrasonic transducers; and

determine a location and relative displacement of said object by weighting said time of flight measurements and with said phase differential measurements for the plurality of ultrasonic transducers from repeated operation.

2. The apparatus of claim 1,

wherein a member of said sensing unit has ultrasonic sensing elements, wherein a first sensing element, a second sensing element, and a third ultrasonic sensing element are symmetrically spaced.

3. The apparatus of claim 1, wherein a member of said sensing unit has at least one ultrasonic sensing element, wherein a first and a second member are spaced at approximately 90 degrees, a second and third member are spaced at approximately 90 degrees, and the third and the first member are spaced at approximately 180 degrees to one another, wherein the sensing unit is less than 10 inches along any length.

4. The apparatus of claim 1, further comprising a center ultrasonic sensing element positioned at a center location that is approximately equidistant from at least one more sensing unit.

5. The apparatus of claim 1, further comprising one more sensing element that is at least one of an acoustic microphone, an acoustic speaker, an ultrasonic receiver, an ultrasonic emitter, a combination acoustic microphone and speaker, an omni-directional ultrasonic transducer, a directional ultrasonic transducer, a micro-electromechanical (MEMS) microphone, an imaging element, an infra-red element, and a camera element.

6. A method for identifying and tracking a relative location and movement of an object in a three-dimensional space comprising:

emitting a first ultrasonic pulse from a first ultrasonic transducer configured to transmit the ultrasonic pulse at a first time and reflect off an object in said three-dimensional space and generate a first reflected ultrasonic signal;

emitting a second ultrasonic pulse from the first ultrasonic transducer configured to transmit the ultrasonic pulse at a second time and reflect off the object in said three-dimensional space and generate a second reflected ultrasonic signal;

digitally sampling the first and second reflected ultrasonic signal to produce first and second sampled reflected ultrasonic signals;

suppressing a ringing distortion of the first and second reflected ultrasonic signals via digital signal processing;

wherein the ringing distortion is a result of the close proximity interference, residual ringing of the first and second emitted ultrasonic pulses and an overlap with a ring-down of the first and second emitted ultrasonic pulse;

and wherein the suppressing comprises subtracting a contribution of the ringing portion of the first and second emitted ultrasonic pulses from the first and second reflected ultrasonic signals to remove the ring-down effect;

wherein a contribution of the emitted ultrasonic signal is subtracted from the reflected signal according to a symmetry of sensing elements that transmit and receive the ultrasonic signals to suppress the ring-down

estimating a time of flight between when said second ultrasonic pulse was transmitted from said first ultrasonic transducer and when the second reflected ultrasonic signal was received for a plurality of ultrasonic transducers configured to receive the second reflected ultrasonic signal;

calculating a phase differential between the first reflected ultrasonic signal and the second ultrasonic reflected signal both received at a same ultrasonic transducer for the plurality of ultrasonic transducers;

determining a location and relative displacement of said object by weighting said time of flight measurements and with said phase differential measurements for the plurality of ultrasonic transducers;

wherein the object is within a predetermined range from the first transducer and plurality of transducers.

7. The method of claim 6, further comprising:

converting said location and relative movement to a coordinate object; and

conveying said coordinate object to a program for controlling a user interface.

8. The method of claim 7, further comprising wirelessly transmitting said coordinate object to a communication device.

9. The method of claim 6, further comprising:

tracking a finger within said three-dimensional space;

detecting a finger action within said three-dimensional space; and

performing a user interface command in response to said finger action.

10. The method of claim 9, further comprising at least one of:

determining a length of time a finger stays at a current location; and

determining if a finger moves away and returns to the current location within a time frame.

11. The method of claim 9, further comprising detecting a touchless finger depression action, a touchless finger slide action, a touchless finger release action, and a touchless finger hold action.

12. The method of claim 9, further comprising:

from the first ultrasonic transducer, emitting a first ultrasonic signal from a first direction at a first time;

from at least one pair of ultrasonic transducers, detecting a first and second ultrasonic reflection of said signal off a finger from said first direction;

calculating a first and second phase differential from said first ultrasonic signal for an ultrasonic transducer of each said pair;

determining a location of said finger from first and second time of flight (TOF) measurements calculated at said pair of ultrasonic transducers; and

updating said location using said phase differentials from each said pair.

13. The method of claim 12, further comprising:

repeating the steps of emitting, detecting, and determining a TOF and said phase differentials for multiple directions at multiple times for generating a plurality of finger locations and relative displacements;

correlating said possible finger locations for determining a finger position having a highest likelihood of producing said reflections from said multiple directions; and

updating said finger locations using said relative displacements through a combinational weighting of said locations and said differentials.

14. The method of claim 6, further comprising:

calculating

a first TOF of a signal emitted at a first time, and

a first phase differential between the signal emitted at the first time and a previously received reflected signal both detected at a first sensor;

calculating

a second TOF of a signal emitted at the first time, and

a second phase differential between said signal emitted at said first time and a previously received reflected signal both detected at a second sensor; and

calculating

a third TOF of a signal emitted at the first time, and

a third phase differential between said signal emitted at said first time and a previously received reflected signal both detected at a third sensor.

15. The method of claim 14, further comprising

calculating a first estimate of a location of said object from a first locus of points produced from said first TOF;

calculating a second estimate of a location of said object from a second locus of points produced from said second TOF;

calculating a third estimate of a location of said object from a third locus of points produced from said third TOF;

creating at least three complex surfaces as a function of said first, second, and third loci of points as an estimate of a location; and

identifying an intersection of said at least three complex surfaces corresponding to a coarse location of said object; and

applying said first, second, and third phase differential to said location for updating said coarse location of said object to a fine location of said object.

16. The method of claim 14, further comprising:

estimating a coarse location of said object using an estimate of said object'"'"'s location from said TOFs when said object is at a far distance from said sensing unit;

updating said coarse location using a relative displacement of said object using said phase differences when said object is at a close distance from said sensing unit;

employing a combinational weighting of location and relative displacement as said object moves between said far distance and said close distance.

17. The method of claim 16, wherein a phase differential corresponds to a relative displacement of said object and a TOF corresponds to a location of said object, such that a base coordinate of said object can be referenced at a location when said object is at a far distance from said sensing unit, and said base coordinate is updated using said relative displacement as said object nears closer to said sensing unit.

18. The method of claim 17, wherein said base coordinate is updated at one of a faster or slower time interval in accordance with the proximity of the object to the sensing unit.

19. A mobile device having a sensing unit comprising:

an array of sensing elements arranged for said communication device to identify and track touchless finger movements;

at least one sensing element within a boundary of a display of the communication device; and

a controller communicatively coupled to said array and said at least one sensing element for conveying touchless controls to a user interface of said communication device in response to said finger movements

where the controller by way of the at least one sensing element;

emits a sequence of ultrasonic pulses from a first ultrasonic transducer configured to transmit the sequence of ultrasonic pulses that reflect off an object to produce reflected ultrasonic signals;

digitally sample the reflected ultrasonic signals to produce sampled reflected ultrasonic signals;

suppress a ringing distortion of the sampled reflected ultrasonic signals via digital signal processing;

wherein the ringing distortion is a result of the close proximity interference, residual ringing of the emitted ultrasonic pulse and an overlap with a ring-down of the emitted ultrasonic pulses;

and wherein the suppressing comprises subtracting a contribution of the ringing portion of the emitted ultrasonic pulses from the reflected signals to remove a ring-down effect;

estimates a time of flight between when one ultrasonic pulse of the sequence was transmitted from said first ultrasonic transducer and when the reflected ultrasonic signal generated from a reflection of said one ultrasonic pulse off an object in said three-dimensional space was received for a plurality of ultrasonic transducers configured to receive the reflection of said one ultrasonic pulse;

calculates a phase differential between the reflected ultrasonic signal and a previously received ultrasonic reflected signal at each ultrasonic transducer for the plurality of ultrasonic transducers; and

determines a location and relative displacement of said object f-rein by weighting said time of flight measurement with said phase differential measurement for the plurality of ultrasonic transducers, wherein the sensing unit is less than 10 inches along any length.

20. The mobile device of claim 19, wherein said at least one sensing element emits a direct pulse to said array to estimate an angle of the display of said communication device.