Apparatus, method and program for processing acoustic signal, and recording medium in which acoustic signal, processing program is recorded

• US 20060215854A1
• Filed: 09/27/2005
• Published: 09/28/2006
• Est. Priority Date: 03/23/2005
• Status: Active Grant

• Current Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Original Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Inventors: Koga, Toshiyuki, Suzuki, Kaoru
• Granted Patent: US 7,711,127 B2
• Time in Patent Office: Days
• US Class Current: 381/98
• CPC Class Codes: G10L 21/0272 Voice signal separating
• CPC Class Codes: H04R 3/005 for combining the signals of two or more microphones specially adapted for hearing aids H04R25/407
• CPC Class Codes: H04S 7/40 Visual indication of stereophonic sound image

Abstract

An acoustic signal processing apparatus includes an acoustic signal input device, a frequency resolution device, a two-dimensional data generating device, a graphics detection device, a sound source candidate information generating device, and a sound source information generating device. The sound source information generating device generates sound source information including at least one of the number of sound sources, the spatial existence range of the sound source, an existence period of the voice, a frequency component configuration of the voice, amplitude information on the voice, and symbolic contents of the voice based on the sound source candidate information and corresponding information which are generated by the sound source candidate information generating device.

Claims

1. An acoustic signal processing apparatus comprising:

an acoustic signal input device configured to input n acoustic signals including voice from a sound source, the n acoustic signals being detected at n different points (n is a natural number 3 or more);

a frequency resolution device configured to resolve each of the acoustic signals into a plurality of frequency components to obtain n pieces of frequency resolved information including phase information of each frequency component;

a two-dimensional data generating device configured to compute phase difference between a pair of pieces of frequency resolved information in each frequency component with respect to m pairs of pieces of frequency resolved information different from each other in the n pieces of frequency resolved information (m is a natural number 2 or more), the two-dimensional data generating device generating m pieces of two-dimensional data in which a frequency function is set at a first axis and a function of the phase difference is set at a second axis;

a graphics detection device configured to detect predetermined graphics from each piece of the two-dimensional data;

a sound source candidate information generating device configured to generate sound source candidate information including at least one of the number of a plurality of sound source candidates, a spatial existence range of each sound source candidate, and the frequency component of the acoustic signal from each sound source candidate based on each of the detected graphics, the sound source candidate information generating device generating corresponding information indicating a corresponding relationship between the pieces of sound source candidate information; and

a sound source information generating device configured to generate sound source information including at least one of the number of sound sources, the spatial existence range of the sound source, an existence period of the voice, a frequency component configuration of the voice, amplitude information on the voice, and symbolic contents of the voice based on the sound source candidate information and corresponding information which are generated by the sound source candidate information generating device.

2. An acoustic signal processing apparatus according to claim 1, wherein the two-dimensional data is a set of coordinate values of points determined by the frequency component and the phase difference, the frequency component and the phase difference being located on a two-dimensional coordinate system in which scalar multiplication of the frequency is set at the first axis and scalar multiplication of the phase difference is set at the second axis.

3. An acoustic signal processing apparatus according to claim 1, wherein the two-dimensional data is a set of coordinate values of points determined by the frequency component and the phase difference, the frequency component and the phase difference being located on the two-dimensional coordinate system in which the scalar multiplication of the frequency is set at the first axis and arrival time difference derived from the phase difference is set at the second axis.

4. An acoustic signal processing apparatus according to claim 1, wherein the graphics detection device detects a line as the graphics.

5. An acoustic signal processing apparatus according to claim 1, wherein the two-dimensional data is a set of coordinate values of points determined by the frequency component and the phase difference, the frequency component and the phase difference being located on the two-dimensional coordinate system having the first axis the second axis,

the graphics detection device includes a voting device which votes the each point in a vote space by linear Hough transform, and the graphics detection device detects the line by detecting a maximum position from a vote distribution generated by the vote up to the predetermined number of high-order votes, the vote becomes values not lower than a predetermined threshold in the maximum position.

6. An acoustic signal processing apparatus according to claim 1, wherein the two-dimensional data is a set of coordinate values of points determined by the frequency component and the phase difference, the frequency component and the phase difference being located on the two-dimensional coordinate system having the first axis the second axis,

the graphics detection device includes a voting device which votes the each point in a predetermined direction, and the graphics detection device detects the line by detecting a maximum position from a vote distribution generated by the vote up to the predetermined number of high-order votes, the vote becomes values not lower than a predetermined threshold in the maximum position.

7. An acoustic signal processing apparatus according to claim 5, wherein the voting device votes a fixed value per the one point.

8. An acoustic signal processing apparatus according to claim 5, wherein the voting device votes a numerical value per the one point, the numerical value being computed from a power value of the frequency corresponding to the point.

9. An acoustic signal processing apparatus according to claim 5, wherein the graphics detection device determines the maximum position only at a position on the vote space corresponding to the line passing through a particular position on the two-dimensional coordinate system, when the graphics detection device detects the maximum position which captures the votes not lower than the predetermined threshold from the vote distribution in the line.

10. An acoustic signal processing apparatus according to claim 5, wherein the graphics detection device computes a total value of the votes corresponding to the line group, the lines in the line group having the same gradient as the line, the lines in the line group being parallel to one another, the lines in the line group being separated apart from one another by a predetermined distance computed according to the gradient, and the graphics detection device determines the maximum position in which the total value becomes values not lower than a predetermined threshold, when the graphics detection device detects the maximum position which captures the vote not lower than the predetermined threshold from the vote distribution in the line.

11. An acoustic signal processing apparatus according to claim 1, wherein the sound source candidate information generating device estimates continuity in a time axis direction in each of the sound source candidate, and the sound source candidate information generating device generates the corresponding information by causing the sound source candidates to correspond to each other, the total vote value becoming the maximum value in the sound source candidates.

12. An acoustic signal processing apparatus according to claim 5, wherein the sound source candidate information generating device estimates the total vote value in the time axis direction of the graphics detected by the graphics detection device in each of the sound source candidate, and the sound source candidate information generating device generates the corresponding information by causing the sound source candidates to correspond to each other, the total vote value becoming the maximum value in the sound source candidates.

13. An acoustic signal processing apparatus according to claim 1, wherein the sound source candidate information generating device estimates the continuity in the time axis direction in each of the sound source candidate, and the sound source candidate information generating device generates the corresponding information by causing the sound source candidates to correspond to each other, continuous periods becoming the same time in the sound source candidates.

14. An acoustic signal processing apparatus according to claim 1, wherein the sound source candidate information generating device estimates resemblance between each sound source candidate and other sound source candidates, and the sound source candidate information generating device generates the corresponding information by causing the sound source candidates to correspond to each other, the sound source candidates having the resemble frequency components.

15. An acoustic signal processing apparatus according to claim 1, wherein the sound source information generating device generates the sound source information by computing a space range through which at least both the spatial existence range of the sound source indicated by first sound source candidate information and the spatial existence range of the sound source indicated by second sound source candidate information pass commonly, the sound source information indicating the spatial existence range of the sound source, the spatial existence range of the sound source indicated by the first sound source candidate information and the spatial existence range of the sound source indicated by the second sound source candidate information being caused to correspond to according to the corresponding information generated by the sound source candidate information generating device.

16. An acoustic signal processing apparatus according to claim 1, wherein the sound source information generating device generates the sound source information by searching a space coordinate having the smallest error from a predetermined table, the sound source information indicating the spatial existence range of the sound source, the space coordinate having the smallest error simultaneously satisfying at least a first sound source direction and a second sound source direction, the first sound source direction and the second sound source direction being caused to correspond to according to the corresponding information generated by the sound source candidate information generating device, the first sound source direction being estimated from first graphics corresponding to the first sound source candidate information, the second sound source direction being estimated from second graphics corresponding to the second sound source candidate information.

17. An acoustic signal processing apparatus according to claim 1, wherein the sound source information generating device selects a pair by comparing at least a first sound source direction and a second sound source direction, the pair capturing the source sound from the front-most surface, the first sound source direction and the second sound source direction being caused to correspond to according to the corresponding information generated by the sound source candidate information generating device, the first sound source direction being estimated from first graphics corresponding to the first sound source candidate information, the second sound source direction being estimated from second graphics corresponding to the second sound source candidate information, and the sound source information generating device generates the sound source information from the acoustic signal corresponding to the selected pair or the frequency resolved information, the sound source information indicating the amplitude information of the voice.

18. An acoustic signal processing apparatus according to claim 1, wherein the sound source information generating device selects a pair by comparing at least a first sound source direction and a second sound source direction, the pair being located farthest away from other sound sources being selected, the first sound source direction and the second sound source direction being caused to correspond to according to the corresponding information generated by the sound source candidate information generating device, the first sound source direction being estimated from first graphics corresponding to the first sound source candidate information, the second sound source direction being estimated from second graphics corresponding to the second sound source candidate information, and the sound source information generating device generates the sound source information from the acoustic signal corresponding to the selected pair or the frequency resolved information, the sound source information indicating the amplitude information of the voice.

19. An acoustic signal processing apparatus according to claim 1, further comprising:

a user interface device for which a user confirms and changes setting information on apparatus operation.

20. An acoustic signal processing apparatus according to claim 1, further comprising:

a user interface device for which the user stores and reads the setting information on the apparatus operation.

21. An acoustic signal processing apparatus according to claim 1, further comprising:

a user interface device configured to display the two-dimensional data or the graphics to the user.

22. An acoustic signal processing apparatus according to claim 1, further comprising:

a device configured to display the sound source information to the user.

23. An acoustic signal processing method, comprising:

inputting n acoustic signals including voice from a sound source, the n acoustic signals being captured at n different points (n is a natural number 3 or more);

resolving each of the acoustic signals into a plurality of frequency components to obtain n pieces of frequency resolved information including phase information of each frequency component;

computing phase difference between a pair of pieces of frequency resolved information in each frequency component with respect to m pairs of pieces of frequency resolved information different from each other in the n pieces of frequency resolved information (m is a natural number 2 or more), and generating m pieces of two-dimensional data in which a frequency function is set at a first axis and a function of the phase difference is set at a second axis;

detecting predetermined graphics from each piece of the two-dimensional data;

generating sound source candidate information including at least one of the number of a plurality of sound source candidates, a spatial existence range of each sound source candidate, and the frequency component of the acoustic signal from each sound source candidate based on each of the detected graphics, and generating corresponding information indicating a corresponding relationship between the pieces of sound source candidate information; and

generating sound source information including at least one of the number of sound sources, the spatial existence range of the sound source, an existence period of the voice, a frequency component configuration of the voice, amplitude information on the voice, and symbolic contents of the voice based on the sound source candidate information and corresponding information which are generated by the sound source candidate information generating device.

24. An acoustic signal processing program, recorded on a computer readable storage medium, the program comprising:

means for instructing a computer to input n acoustic signals including voice from a sound source, the n acoustic signals being captured at n different points (n is a natural number 3 or more);

means for instructing the computer to resolve each of the acoustic signals into a plurality of frequency components to obtain n pieces of frequency resolved information including phase information of each frequency component;

means for instructing the computer to compute phase difference between a pair of pieces of frequency resolved information in each frequency component with respect to m pairs of pieces of frequency resolved information different from each other in the n pieces of frequency resolved information (m is a natural number 2 or more), and to generate device generating m pieces of two-dimensional data in which a frequency function is set at a first axis and a function of the phase difference is set at a second axis;

means for instructing the computer to detect graphics which is previously determined from each piece of the two-dimensional data;

means for instructing the computer to generate sound source candidate information including at least one of the number of a plurality of sound source candidates, a spatial existence range of each sound source candidate, and the frequency component of the acoustic signal from each sound source candidate based on each of the detected graphics, and to generate corresponding information indicating a corresponding relationship between the pieces of sound source candidate information; and

means for instructing the computer to generate sound source information including at least one of the number of sound sources, the spatial existence range of the sound source, an existence period of the voice, a frequency component configuration of the voice, amplitude information on the voice, and symbolic contents of the voice based on the sound source candidate information and corresponding information which are generated by the sound source candidate information generating device.

25. A computer-readable recording medium in which an acoustic signal processing program according to claim 24 is recorded.