Ultrasonic depth imaging

• US 9,945,946 B2
• Filed: 09/11/2014
• Issued: 04/17/2018
• Est. Priority Date: 09/11/2014
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Dokmanic, Ivan, Tashev, Ivan J., Soemo, Thomas M.
• Primary Examiner(s): Alsomiri, Isam A
• Assistant Examiner(s): Armstrong, Jonathan D
• Application Number:
  US14/484,128
• Publication Number:
  US 20160077206A1
• Time in Patent Office: 1,314 Days
• Field of Search: 367 99
• CPC Class Codes: G01S 15/89 for mapping or imaging
• CPC Class Codes: G01S 3/74 Multi-channel systems specially adapted for direction-finding, i.e. having a single antenna system capable of giving simultaneous indications of the directions of different signals systems in which the directions of different signals are determined sequentially and displayed simultaneously G01S3/04, G01S3/14
• CPC Class Codes: G01S 7/526 Receivers

Abstract

Examples are disclosed herein that relate to depth imaging techniques using ultrasound One example provides an ultrasonic depth sensing system configured to, for an image frame, emit an ultrasonic pulse from each of a plurality of transducers, receive a reflection of each ultrasonic pulse at a microphone array, perform transmit beamforming and also receive beamforming computationally after receiving the reflections, form a depth image, and output the depth image for the image frame.

Claims

1. An ultrasonic depth sensing system configured to output a depth image, the ultrasonic depth sensing system comprising

a transducer array comprising a plurality of transducers;

a microphone array comprising a plurality of microphones; and

a computing device comprising a logic subsystem and a storage subsystem including instructions executable by the logic subsystem to control the ultrasonic depth sensing system to,

for an image frame,

emit an ultrasonic pulse from each of a plurality of transducers,

for each ultrasonic pulse, receive a reflection of the ultrasonic pulse at a microphone array, wherein receiving the reflection comprises receiving one or more echoes corresponding to the reflection,

perform transmit beamforming and receive beamforming computationally after receiving the reflections,

form the depth image based upon echoes detected by the receive beamforming, wherein forming the depth image comprises, for a selected echo, performing sound source localization and comparing a direction determined by the sound source localization to a direction of the receive beamforming, and assigning a depth value determined for the selected echo to a pixel of a depth image if the direction determined by the sound source localization is within a threshold distance of the direction determined by the receive beamforming; and

output the depth image for the image frame.

2. The system of claim 1, wherein the instructions are executable to emit the ultrasonic pulse from each of the plurality of transducers by timing emission of the ultrasonic pulses based upon a frequency of the ultrasonic pulses.

3. The system of claim 1, wherein the instructions are executable to perform transmit beamforming by computing transmit beams for each pixel of the depth image after receipt of each reflection of each ultrasonic pulse from all of the plurality of transducers.

4. The system of claim 1, wherein the instructions are executable to emit the ultrasonic pulse from each of the plurality of transducers by emitting a single ultrasonic pulse from each transducer for the image frame.

5. The system of claim 1, wherein the instructions are executable to detect the one or more echoes by detecting a plurality of echoes from the direction of the receive beamforming and determining a candidate list of echoes from the plurality of echoes based upon strength, and wherein the selected echo is from the candidate list of echoes.

6. The system of claim 5, wherein the instructions are executable to determine depth values for two or more echoes of the candidate list of echoes, and if the two or more echoes of the candidate list are from directions within a threshold distance of the direction of the receive beamforming, then assign the depth value to the pixel of the depth image by assigning a smallest depth value of the two or more echoes that are from directions within the threshold distance.

7. The system of claim 1, wherein the instructions are further executable to determine depth values for a plurality of pixels of the depth image.

8. A method of ultrasonic depth sensing, comprising:

for an image frame,

emitting an ultrasonic pulse from each of a plurality of transducers;

for each ultrasonic pulse, receiving a reflection of the ultrasonic pulse at a microphone array;

performing receive beamforming to detect one or more echoes from a direction determined by the receive beamforming;

for a selected echo of the one or more echoes, performing sound source localization using a reflection corresponding to the selected echo;

comparing a direction determined by the sound source localization to the direction determined by the receive beamforming;

assigning a depth value determined for the selected echo to a pixel of a depth image if the direction determined by the sound source localization is within a threshold distance of the direction determined by the receive beamforming; and

outputting the depth image for the image frame.

9. The method of claim 8, wherein detecting the one or more echoes comprises detecting a plurality of echoes and determining a candidate list of echoes from the plurality of echoes, and wherein the selected echo is selected from the candidate list of echoes.

10. The method claim 9, further comprising determining depth values for two or more echoes of the candidate list of echoes.

11. The method of claim 10, wherein if the two or more echoes of the candidate list of echoes are from directions within the threshold distance, then assigning the depth value to the pixel of the depth image comprises assigning a smallest depth value of the two or more echoes.

12. The method of claim 8, further comprising not assigning the depth value to the pixel of the depth image if the direction determined by the sound source localization is not within the threshold distance of the direction determined by the receive beamforming.

13. The method of claim 8, further comprising performing receive beamforming for each direction of an array of directions to determine depth values for a plurality of pixels of a depth image.

14. The method of claim 8, further comprising performing transmit beamforming after receiving the reflection of all ultrasonic pulses for the image frame at the microphone array.

15. The method of claim 8, wherein emitting the ultrasonic pulse from each of the plurality of transducers comprises timing an emission of the ultrasonic pulses based upon a frequency of the ultrasonic pulses.

16. The method of claim 15, wherein emitting the ultrasonic pulse from each of the plurality of transducers comprises emitting a single ultrasonic pulse from each transducer for the image frame.

17. A method of ultrasonic depth sensing, comprising:

for an image frame,

emitting an ultrasonic pulse from each of a plurality of transducers;

for each ultrasonic pulse, receiving a reflection of the ultrasonic pulse at a microphone array;

performing transmit beamforming after receiving the reflection of all ultrasonic pulses for the image frame at the microphone array;

performing receive beamforming to detect one or more echoes from a direction of the receive beamforming;

for a selected echo of the one or more echoes, performing sound source localization using a reflection corresponding to the selected echo;

comparing a direction determined by the sound source localization to the direction of the receive beamforming;

assigning a depth value determined for the selected echo to a pixel of a depth image if the direction determined by the sound source localization is within a threshold distance of the direction of the receive beamforming;

not assigning the depth value to the pixel if the direction determined by the sound source localization is not within the threshold distance of the direction of the receive beamforming; and

outputting the depth image for the image frame.

18. The method of claim 17, wherein detecting the one or more echoes comprises detecting a plurality of echoes from the direction of the receive beamforming and determining a candidate list of echoes from the plurality of echoes based upon strength, and wherein the selected echo is selected from the candidate list of echoes.

19. The method of claim 18, further comprising determining depth values for two or more echoes of the candidate list of echoes, wherein if the two or more echoes of the candidate list are from directions within the threshold distance, then assigning the depth value to the pixel of the depth image comprises assigning a smallest depth value of the two or more echoes that are from directions within the threshold distance.