METHOD, COMPUTER READABLE STORAGE MEDIUM AND SYSTEM FOR LOCALIZING ACOUSTIC SOURCE

• US 20110135102A1
• Filed: 03/22/2010
• Published: 06/09/2011
• Est. Priority Date: 12/04/2009
• Status: Active Grant

• Current Assignee: TECO Electric & Machinery Company Limited (TECO Electric & Machinery Pte Ltd.)
• Original Assignee: TECO Electric & Machinery Company Limited (TECO Electric & Machinery Pte Ltd.)
• Inventors: Huang, Hsin-Chieh, Kao, Chih-Wei, Lin, Wen-Kuo
• Granted Patent: US 8,363,848 B2
• Time in Patent Office: Days
• US Class Current: 381/58
• CPC Class Codes: G01S 3/8083 determining direction of source
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: H04N 7/142 Constructional details of the terminal equipment, e.g. arrangements of the camera and the display
• CPC Class Codes: H04R 27/00 Public address systems circuits for preventing acoustic reaction H04R3/02; circuits for distributing signals to loudspeakers H04R3/12; monitoring or testing arrangements for public address systems H04R29/007; amplifiers H03F

Abstract

A system for localizing an acoustic source is provided. This system includes a microphone apparatus, an audio processing apparatus, a photographing apparatus, and a decision apparatus. The microphone apparatus receives an acoustic signal and generates at least one received audio signal. The audio processing apparatus generates first location information based on the at least one received audio signal. The decision apparatus generates depth information based on at least one image captured by the photographing apparatus. According to the first location information, the at least one captured image, and the depth information, the decision apparatus determines a location corresponding to the source of the acoustic signal.

Claims

1. An acoustic source localization system, comprising:

a microphone apparatus for receiving an acoustic signal in a region and generating at least one received audio signal;

an audio processing apparatus for generating first location information based on the at least one received audio signal;

a photographing apparatus for photographing the region and generating at least one captured image; and

a decision apparatus for generating depth information of the region based on the at least one captured image and determining a source location corresponding to the acoustic signal based on the first location information, the at least one captured image, and the depth information.

2. The acoustic source localization system of claim 1, wherein the microphone apparatus comprises plural microphones, each of the microphones respectively receives the acoustic signal and respectively generates one of the received audio signals;

the audio processing apparatus generates the first location information based on at least one difference between the received audio signals.

3. The acoustic source localization system of claim 1, wherein the decision apparatus performs image recognition on the at least one captured image to generate second location information;

according to the first location information, the second location information, and the depth information, the decision apparatus determines the source location corresponding to the acoustic signal.

4. The acoustic source localization system of claim 3, wherein the decision apparatus performs image recognition on the at least one captured image based on the first location information.

5. The acoustic source localization system of claim 3, wherein the photographing apparatus comprises two image capturers and the depth information is a depth map.

6. The acoustic source localization system of claim 5, wherein the depth map has a horizontal axis, a vertical axis, and a depth axis, the axes are substantially perpendicular to each other;

the first location information comprises a horizontal coordinate on the horizontal axis, the second location information comprises a vertical coordinate on the vertical axis;

after determining a depth coordinate on the depth axis corresponding to the horizontal coordinate and the vertical coordinate, the decision apparatus labels the source location by the horizontal coordinate, the vertical coordinate, and the depth coordinate.

7. The acoustic source localization system of claim 3, further comprising:

a database therein storing plural default acoustic source images, the decision apparatus performs image recognition on the at least one captured image based on the default acoustic source images.

8. The acoustic source localization system of claim 7, wherein if the decision apparatus finds out, from the at least one captured image, a target image corresponding to one of the default acoustic source images, the decision apparatus generates the depth information according to the target image and the corresponding default acoustic source image.

9. The acoustic source localization system of claim 8, wherein each of the default acoustic source images is respectively a three-dimensional image.

10. An acoustic source localization method, comprising the steps:

(a) receiving an acoustic signal in a region and generating at least one received audio signal;

(b) generating first location information based on the at least one received audio signal;

(c) photographing the region and generating at least one captured image; and

(d) generating depth information of the region based on the at least one captured image and determining a source location corresponding to the acoustic signal based on the first location information, the at least one captured image, and the depth information.

11. The acoustic source localization method of claim 10, wherein in step (d), second location information is generated by performing image recognition on the at least one captured image;

the source location corresponding to the acoustic signal is determined according to the first location information, the second location information, and the depth information.

12. The acoustic source localization method of claim 11, wherein in step (d), image recognition is performed on the at least one captured image based on the first location information.

13. The acoustic source localization method of claim 11, wherein the depth information is a depth map.

14. The acoustic source localization method of claim 13, wherein the depth map has a horizontal axis, a vertical axis, and a depth axis, the axes are substantially perpendicular to each other;

the first location information comprises a horizontal coordinate on the horizontal axis, the second location information comprises a vertical coordinate on the vertical axis;

in step (d), after a depth coordinate on the depth axis corresponding to the horizontal coordinate and the vertical coordinate is determined, the source location is represented by the horizontal coordinate, the vertical coordinate, and the depth coordinate.

15. The acoustic source localization method of claim 11, wherein in step (d), image recognition is performed on the at least one captured image based on plural default acoustic source images.

16. The acoustic source localization method of claim 15, wherein step (d) comprises:

from the at least one captured image, finding out a target image corresponding to one of the default acoustic source images; and

generating the depth information according to the target image and the corresponding default acoustic source image.

17. A computer readable storage medium having stored therein a acoustic source localization program for executing instruction comprising:

a first instruction for generating first location information based on at least one received audio signal;

a second instruction for generating depth information based on at least one captured image; and

a third instruction for determining a source location corresponding to an acoustic signal based on the first location information, the at least one captured image, and the depth information.

18. The computer readable storage medium in claim 17, wherein when the third instruction is executed, image recognition is performed on the at least one captured image to generate second location information;

the source location corresponding to the acoustic signal is determined according to the first location information, the second location information, and the depth information.

19. The computer readable storage medium in claim 18, wherein when the third instruction is executed, image recognition is performed on the at least one captured image based on the first location information.

20. The computer readable storage medium in claim 17, wherein the depth information is a depth map.