Sound enhancement method, device, program and recording medium

• US 9,191,738 B2
• Filed: 12/19/2011
• Issued: 11/17/2015
• Est. Priority Date: 12/21/2010
• Status: Active Grant

• Current Assignee: Nippon Telegraph and Telephone Corporation
• Original Assignee: Nippon Telegraph and Telephone Corporation
• Inventors: Niwa, Kenta, Sakauchi, Sumitaka, Furuya, Kenichi, Haneda, Yoichi
• Primary Examiner(s): Tran, Thang
• Application Number:
  US13/996,302
• Publication Number:
  US 20130287225A1
• Time in Patent Office: 1,429 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G10L 2021/02082 the noise being echo, reverberation of the speech
• CPC Class Codes: G10L 2021/02166 Microphone arrays; Beamforming
• CPC Class Codes: G10L 21/0232 Processing in the frequency domain
• CPC Class Codes: H04R 2430/03 Synergistic effects of band splitting and sub-band processing
• CPC Class Codes: H04R 3/00 Circuits for transducers , loudspeakers or microphones
• CPC Class Codes: H04R 3/005 for combining the signals of two or more microphones specially adapted for hearing aids H04R25/407

Abstract

A sound enhancement technique that uses transfer functions a_i,g of sounds that come from each of one or more positions/directions that are assumed to be sound sources arriving at each microphone to obtain a filter for a position that is a target of sound enhancement, where i denotes a direction and g denotes a distance for identifying each of the positions. Each of the transfer functions a_i,g is represented by sum of a transmission characteristic of a direct sound that directly arrives from the position determined by the direction i and the distance g and a transmission characteristic of one or more reflected sounds produced by reflection of the direct sound off an reflective object. A filter that corresponds to the position that is the target of sound enhancement is applied to frequency-domain signals transformed from M picked-up sounds picked up with M microphones to obtain a frequency-domain output signal.

Claims

1. A sound enhancement method of obtaining a frequency-domain output signal in which a sound from a desired position determined by a direction and a distance is enhanced by applying, for each frequency, a filter enhancing a sound from the position to frequency-domain signals transformed from M picked-up sounds picked up with M microphones, where M is an integer greater than or equal to two, the method comprising:

a filter design step of using a transmission characteristic a_i,g of a sound that comes from each of one or a plurality of positions that are assumed to be sound sources and arrives at each of the microphones to obtain the filter for each frequency for a position that is a target of a sound enhancement, where i denotes a direction and g denotes a distance for identifying each of the positions; and

a filter applying step of applying the filter obtained at the filter design step to the frequency-domain signals for each frequency to obtain the output signal;

wherein each of the transmission characteristics a_i,g is represented by the sum of a transmission characteristic of a direct sound that comes from the position determined by the direction i and the distance g and directly arrives at the M microphones and a transmission characteristic of one or more reflected sounds, the one or more reflected sounds being produced by reflection of the direct sound off an reflective object and arriving at the M microphones.

2. The sound enhancement method according to claim 1, wherein each of the transmission characteristics a_i,g is the sum of a steering vector of the direct sound and each steering vector of the one or more reflected sounds whose decays due to reflection and arrival time differences with respect to the direct sound are corrected.

3. The sound enhancement method according to claim 1, wherein each of transmission characteristics a_i,g is obtained by measurement in a real environment.

4. The sound enhancement method according to any one of claims 1 to 3,

wherein the filter design step obtains, for each frequency, the filter that minimizes the power of sounds from positions other than the position that is the target of sound enhancement.

5. The sound enhancement method according to any one of claims 1 to 3,

wherein the filter design step obtains, for each frequency, the filter that maximizes the signal-to-noise ratio of a sound from the position that is the target of sound enhancement.

6. The sound enhancement method according to any one of claims 1 to 3,

wherein the filter design step obtains, for each frequency, the filter that minimizes the power of sounds from positions other than the one or plurality of positions that are assumed to be sound source positions while a filter coefficient for one of the M microphones is fixed at a constant value.

7. The sound enhancement method according to any one of claims 1 to 3,

wherein the filter design step obtains, for each frequency, the filter that minimizes the power of sounds from the positions other than the position that is the target of sound enhancement and one or more suppression points on conditions that (1) the filter passes sounds in all frequency bands from the position that is the target of sound enhancement and that (2) the filter suppresses sounds in all frequency bands from the one or more suppression points.

8. The sound enhancement method according to any one of claims 1 to 3,

wherein the filter design step normalizes a transmission characteristic a_s,h of a sound from the position in a direction i=s at distance g=h that is the target of sound enhancement to obtain the filter for each frequency.

9. The sound enhancement method according to any one of claims 1 to 3, wherein the filter design step uses a spatial correlation matrix represented by the transmission characteristics a_i,g corresponding to the positions other than the position that is the target of sound enhancement to obtain the filter for each frequency.

10. The sound enhancement method according to any one of claims 1 to 3,

wherein the filter design step obtains, for each frequency, the filter that minimizes the power of sounds from positions other than the position that is the target of sound enhancement on condition that the filter reduces decay of a sound from the position that is the target of sound enhancement to a predetermined amount or less.

11. The sound enhancement method according to any one of claims 1 to 3,

wherein the filter design step uses a spatial correlation matrix represented by a frequency-domain signal to obtain the filter for each frequency, the frequency-domain signal being obtained by transforming a signal obtained by observation with a microphone array to a frequency domain.

12. The sound enhancement method according to any one of claims 1 to 3, wherein the filter design step uses a spatial correlation matrix represented by the transmission characteristics a_i,g corresponding to each position included in one or a plurality of positions that are assumed to be sound source positions to obtain the filter for each frequency.

28. A non-transitory computer-readable recording medium having recorded thereon a computer program for causing a computer to execute the steps of the sound enhancement method according to claim 1 or 15.

13. A sound enhancement apparatus obtaining a frequency-domain output signal in which a sound from a desired position determined by a direction and a distance is enhanced by applying, for each frequency, a filter enhancing a sound from the position to frequency-domain signals transformed from M picked-up sounds picked up with M microphones, where M is an integer greater than or equal to two, the apparatus comprising:

a filter design section using a transmission characteristic a_i,g of a sound that comes from each of one or a plurality of positions that are assumed to be sound sources and arrives at each of the microphones to obtain the filter for each frequency for a position that is a target of a sound enhancement, where i denotes a direction and g denotes a distance for identifying each of the positions; and

a filter applying section applying the filter obtained by the filter design section to the frequency-domain signals for each frequency to obtain the output signal;

wherein each of the transmission characteristics a_i,g is represented by the sum of a transmission characteristic of a direct sound that comes from the position determined by the direction i and the distance g and directly arrives at the M microphones and a transmission characteristic of one or more reflected sounds, the one or more reflected sounds being produced by reflection of the direct sound off an reflective object and arriving at the M microphones.

14. The sound enhancement apparatus according to claim 13, further comprising one or more reflective objects providing each of the reflected sounds to the M microphones.

15. A sound enhancement method of obtaining a frequency-domain output signal in which a sound from a desired direction is enhanced by applying, for each frequency, a filter enhancing a sound from the direction to frequency-domain signals transformed from M picked-up sounds picked up with M microphones, where M is an integer greater than or equal to two, the method comprising:

a filter design step of using a transmission characteristic a_φ

of a sound that comes from each of one or a plurality of directions φ

that are assumed to be directions from which sounds come and arrives at each of the microphones to obtain the filter for each frequency for a direction that is a target of a sound enhancement; and

a filter applying step of applying the filter obtained at the filter design step to the frequency-domain signals for each frequency to obtain the output signal;

wherein each of the transmission characteristics a_φ

is represented by the sum of a transmission characteristic of a direct sound that comes from the direction φ and

directly arrives at the M microphones and a transmission characteristic of one or more reflected sounds, the one or more reflected sounds being produced by reflection of the direct sound off an reflective object and arriving at the M microphones.

16. The sound enhancement method according to claim 15, wherein each of the transmission characteristics a_φ

is the sum of a steering vector of the direct sound and each steering vector of the one or more reflected sounds whose decays due to reflection and arrival time differences with respect to the direct sound are corrected.

17. The sound enhancement method according to claim 15, wherein each of the transmission characteristics a_φ

is obtained by measurement in a real environment.

18. The sound enhancement method according to any one of claims 15 to 17,

wherein the filter design step obtains, for each frequency, the filter that minimizes the power of sounds from directions other than the direction that is the target of sound enhancement.

19. The sound enhancement method according to any one of claims 15 to 17,

wherein the filter design step obtains, for each frequency, the filter that maximizes the signal-to-noise ratio of a sound from the direction that is the target of sound enhancement.

20. The sound enhancement method according to any one of claims 15 to 17,

wherein the filter design step obtains, for each frequency, the filter that minimizes the power of sounds from the one or plurality of directions that are assumed to be directions from which sounds come, while a filter coefficient for one of the M microphones is fixed at a constant value.

21. The sound enhancement method according to any one of claims 15 to 17,

wherein the filter design step obtains, for each frequency, the filter that minimizes the power of sounds from the directions other than the direction that is the target of sound enhancement and one or more null directions on conditions that (1) the filter passes sounds in all frequency bands from the direction that is the target of sound enhancement and that (2) the filter suppresses sounds in all frequency bands from the one or more null directions.

22. The sound enhancement method according to any one of claims 15 to 17,

wherein the filter design step normalizes a transmission characteristic a_s of a sound from the position in the direction φ

=s that is the target of sound enhancement to obtain the filter for each frequency.

23. The sound enhancement method according to any one of claims 15 to 17, wherein the filter design step uses a spatial correlation matrix represented by the transmission characteristics a_φ

corresponding to directions other than the directions that is the target of sound enhancement to obtain the filter for each frequency.

24. The sound enhancement method according to any one of claims 15 to 17,

wherein the filter design step obtains, for each frequency, the filter that minimizes the power of sounds from directions other than the direction that is the target of sound enhancement on condition that the filter reduces decay of a sound from the direction that is the target of sound enhancement to a predetermined amount or less.

25. The sound enhancement method according to any one of claims 15 to 17,

wherein the filter design step uses a spatial correlation matrix represented by a frequency-domain signal to obtain the filter for each frequency, the frequency-domain signal being obtained by transforming a signal obtained by observation with a microphone array to a frequency domain.

26. A sound enhancement apparatus obtaining a frequency-domain output signal in which a sound from a desired direction is enhanced by applying, for each frequency, a filter enhancing a sound from the direction to frequency-domain signals transformed from M picked-up sounds picked up with M microphones, where M is an integer greater than or equal to two, the apparatus comprising:

a filter design section using a transmission characteristic a_φ

of a sound that comes from each of one or a plurality of directions .phi. that are assumed to be directions from which sounds come and arrives at each of the microphones to obtain the filter for each frequency for a direction that is a target of a sound enhancement; and

a filter applying section applying the filter obtained by the filter design section to the frequency-domain signals for each frequency to obtain the output signal;

wherein each of the transmission characteristics a_φ

is represented by the sum of a transmission characteristic of a direct sound that comes from the direction φ and

directly arrives at the M microphones and a transmission characteristic of one or more reflected sounds, the one or more reflected sounds being produced by reflection of the direct sound off an reflective object and arriving at the M microphones.

27. The sound enhancement apparatus according to claim 26, further comprising one or more reflective objects providing each of the reflected sounds to the M microphones.