Systems, methods, apparatus, and computer program products for enhanced intelligibility

• US 8,538,749 B2
• Filed: 11/24/2008
• Issued: 09/17/2013
• Est. Priority Date: 07/18/2008
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Visser, Erik, Toman, Jeremy
• Primary Examiner(s): Godbold, Douglas
• Application Number:
  US12/277,283
• Publication Number:
  US 20100017205A1
• Time in Patent Office: 1,758 Days
• Field of Search: 704200-201, 704226-228, 704500-504
• US Class Current: 704/228
• CPC Class Codes: G10L 19/00 Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis in musical instruments G10H
• CPC Class Codes: G10L 2021/02082 the noise being echo, reverberation of the speech
• CPC Class Codes: G10L 2021/02087 the noise being separate speech, e.g. cocktail party
• CPC Class Codes: G10L 2021/02166 Microphone arrays; Beamforming
• CPC Class Codes: G10L 21/02 Speech enhancement, e.g. noise reduction or echo cancellation reducing echo effects in line transmission systems H04B3/20; echo suppression in hands-free telephones H04M9/08
• CPC Class Codes: G10L 21/0208 Noise filtering
• CPC Class Codes: G10L 21/0232 Processing in the frequency domain
• CPC Class Codes: G10L 25/18 the extracted parameters being spectral information of each sub-band
• CPC Class Codes: H04R 1/1083 Reduction of ambient noise active noise reduction per se G10K11/175; protective devices for the ear, e.g. providing acoustic protection A61F11/06
• CPC Class Codes: H04R 2430/03 Synergistic effects of band splitting and sub-band processing
• CPC Class Codes: H04R 3/005 for combining the signals of two or more microphones specially adapted for hearing aids H04R25/407

Abstract

Techniques described herein include the use of equalization techniques to improve intelligibility of a reproduced audio signal (e.g., a far-end speech signal).

Claims

1. A method comprising:

performing a spatially selective processing operation on a first input, wherein the first input is a multichannel sensed audio signal input, to produce a source signal and a noise reference;

filtering a second input, wherein the second input is a reproduced audio signal input, to obtain a first plurality of time-domain subband signals;

filtering the noise reference to obtain a second plurality of time-domain subband signals;

based on information from the first plurality of time-domain subband signals, calculating a plurality of first subband power estimates;

based on information from the second plurality of time-domain subband signals, calculating a plurality of second subband power estimates; and

based on information from the plurality of first subband power estimates and on information from the plurality of second subband power estimates, boosting at least one frequency subband of the reproduced audio signal input relative to at least one other frequency subband of the reproduced audio signal input.

2. The method of claim 1, further comprising filtering a second noise reference that is based on information from the multichannel sensed audio signal input to obtain a third plurality of time-domain subband signals, and

wherein said calculating a plurality of second subband power estimates is based on information from the third plurality of time-domain subband signals.

3. The method of claim 2, wherein the second noise reference is an unseparated sensed audio signal.

4. The method of claim 3, wherein said calculating a plurality of second subband power estimates includes:

based on information from the second plurality of time-domain subband signals, calculating a plurality of first noise subband power estimates;

based on information from the third plurality of time-domain subband signals, calculating a plurality of second noise subband power estimates; and

identifying the minimum among the calculated plurality of second noise subband power estimates, and

wherein the values of at least two among the plurality of second subband power estimates are based on the identified minimum.

5. The method of claim 2, wherein the second noise reference is based on the source signal.

6. The method of claim 2, wherein said calculating a plurality of second subband power estimates includes:

based on information from the second plurality of time-domain subband signals, calculating a plurality of first noise subband power estimates; and

based on information from the third plurality of time-domain subband signals, calculating a plurality of second noise subband power estimates, and

wherein each of the plurality of second subband power estimates is based on the maximum of (A) a corresponding one of the plurality of first noise subband power estimates and (B) a corresponding one of the plurality of second noise subband power estimates.

7. The method of claim 1, wherein said performing a spatially selective processing operation includes concentrating energy of a directional component of the multichannel sensed audio signal input into the source signal.

8. The method of claim 1,

wherein the multichannel sensed audio signal input includes a directional component and a noise component, and

wherein said performing a spatially selective processing operation includes separating energy of the directional component from energy of the noise component such that the source signal contains more of the energy of the directional component than each channel of the multichannel sensed audio signal input does.

9. The method of claim 1,

wherein said filtering the reproduced audio signal input to obtain a first plurality of time-domain subband signals includes obtaining each among the first plurality of time-domain subband signals by boosting a gain of a corresponding subband of the reproduced audio signal input relative to other subbands of the reproduced audio signal input.

10. The method of claim 1,

wherein said method includes, for each of the plurality of first subband power estimates, calculating a ratio of the first subband power estimate and a corresponding one of the plurality of second subband power estimates;

and

wherein said boosting at least one frequency subband of the reproduced audio signal input relative to at least one other frequency subband of the reproduced audio signal input includes, for each of the plurality of first subband power estimates, applying a gain factor based on the corresponding calculated ratio to a corresponding frequency subband of the reproduced audio signal.

11. The method of claim 10,

wherein said boosting at least one frequency subband of the reproduced audio signal input relative to at least one other frequency subband of the reproduced audio signal input includes filtering the reproduced audio signal input using a cascade of filter stages, and

wherein, for each of the plurality of first subband power estimates, said applying a gain factor to a corresponding frequency subband of the reproduced audio signal input comprises applying the gain factor to a corresponding filter stage of the cascade.

12. The method of claim 10, wherein, for at least one of the plurality of first subband power estimates, a current value of the corresponding gain factor is constrained by at least one bound that is based on a current level of the reproduced audio signal.

13. The method of claim 10, wherein said method includes, for at least one of the plurality of first subband power estimates, smoothing a value of the corresponding gain factor over time according to a change in the value of the corresponding ratio over time.

14. The method of claim 1,

wherein said method includes performing an echo cancellation operation on a plurality of microphone signals to obtain the multichannel sensed audio signal,

wherein said performing an echo cancellation operation is based on information from an audio signal that results from said boosting at least one frequency subband of the reproduced audio signal input relative to at least one other frequency subband of the reproduced audio signal.

15. A method of processing a reproduced audio signal, said method comprising performing each of the following acts within a device that is configured to process audio signals:

performing a spatially selective processing operation on a multichannel sensed audio signal to produce a source signal and a noise reference;

for each of a plurality of subbands of the reproduced audio signal, calculating a first subband power estimate;

for each of a plurality of subbands of the noise reference, calculating a first noise subband power estimate;

for each of a plurality of subbands of a second noise reference that is based on information from the multichannel sensed audio signal, calculating a second noise subband power estimate;

for each of the plurality of subbands of the reproduced audio signal, calculating a second subband power estimate that is based on a maximum of the corresponding first and second noise subband power estimates; and

based on information from the plurality of first subband power estimates and on information from the plurality of second subband power estimates, boosting at least one frequency subband of the reproduced audio signal relative to at least one other frequency subband of the reproduced audio signal.

16. The method according to claim 15, wherein the second noise reference is an unseparated sensed audio signal.

17. The method according to claim 15, wherein the second noise reference is based on the source signal.

18. An apparatus comprising:

a spatially selective processing filter configured to perform a spatially selective processing operation on a first input, wherein the first input is a multichannel sensed audio signal input, to produce a source signal and a noise reference;

a first subband signal generator configured to filter a second input, wherein the second input is a reproduced audio signal input, to obtain a first plurality of time-domain subband signals;

a second subband signal generator configured to filter the noise reference to obtain a second plurality of time-domain subband signal;

a first subband power estimate calculator configured to calculate a plurality of first subband power estimates based on information from the first plurality of time-domain subband signals;

a second subband power estimate calculator configured to calculate a plurality of second subband power estimates based on information from the second plurality of time-domain subband signals; and

a subband filter array configured to boost at least one frequency subband of the reproduced audio signal input-relative to at least one other frequency subband of the reproduced audio signal input, based on information from the plurality of first subband power estimates and on information from the plurality of second subband power estimates.

19. The apparatus according to claim 18,

wherein said method includes a third subband signal generator configured to filter a second noise reference that is based on information from the multichannel sensed audio signal input to obtain a third plurality of time-domain subband signals, and

wherein said second subband power estimate calculator is configured to calculate the plurality of second subband power estimates based on information from the third plurality of time-domain subband signals.

20. The apparatus according to claim 19, wherein the second noise reference is an unseparated sensed audio signal.

21. The apparatus according to claim 19, wherein the second noise reference is based on the source signal.

22. The apparatus according to claim 19,

wherein said second subband power estimate calculator is configured to calculate (A) a plurality of first noise subband power estimates based on information from the second plurality of time-domain subband signals and (B) a plurality of second noise subband power estimates based on information from the third plurality of time-domain subband signals, and

wherein said second subband power estimate calculator is configured to calculate each of the plurality of second subband power estimates based on the maximum of (A) a corresponding one of the plurality of first noise subband power estimates and (B) a corresponding one of the plurality of second noise subband power estimates.

23. The apparatus according to claim 18,

wherein the multichannel sensed audio signal input includes a directional component and a noise component, and

wherein said spatially selective processing filter is configured to separate energy of the directional component from energy of the noise component such that the source signal contains more of the energy of the directional component than each channel of the multichannel sensed audio signal input does.

24. The apparatus according to claim 18, wherein said first subband signal generator is configured to obtain each among the first plurality of time-domain subband signals by boosting a gain of a corresponding subband of the reproduced audio signal input relative to other subbands of the reproduced audio signal.

25. The apparatus according to claim 18,

wherein said apparatus includes a subband gain factor calculator configured to calculate, for each of the plurality of first subband power estimates, a ratio of the first subband power estimate and a corresponding one of the plurality of second subband power estimates;

and

wherein said subband filter array is configured to apply a gain factor based on the corresponding calculated ratio, for each of the plurality of first subband power estimates, to a corresponding frequency subband of the reproduced audio signal.

26. The apparatus according to claim 25,

wherein said subband filter array includes a cascade of filter stages, and

wherein said subband filter array is configured to apply each of the plurality of gain factors to a corresponding filter stage of the cascade.

27. The apparatus according to claim 25, wherein said subband gain factor calculator is configured to constrain a current value of the corresponding gain factor, for at least one of the plurality of first subband power estimates, by at least one bound that is based on a current level of the reproduced audio signal.

28. The apparatus according to claim 25, wherein said first subband gain factor calculator is configured to smooth a value of the corresponding gain factor over time, for at least one of the plurality of first subband power estimates, according to a change in the value of the corresponding ratio over time.

29. A non-transitory computer-readable medium comprising instructions which when executed by a processor cause the processor to:

perform a spatially selective processing operation on a first input, wherein the first input is a multichannel sensed audio signal input, to produce a source signal and a noise reference;

filter a second input, wherein the second input is a reproduced audio signal input, to obtain a first plurality of time-domain subband signals;

filter the noise reference to obtain a second plurality of time-domain subband signals;

based on information from the first plurality of time-domain subband signals, calculate a plurality of first subband power estimates;

based on information from the second plurality of time-domain subband signals, calculate a plurality of second subband power estimates; and

based on information from the plurality of first subband power estimates and on information from the plurality of second subband power estimates, boost at least one frequency subband of the reproduced audio signal input relative to at least one other frequency subband of the reproduced audio signal.

30. The computer-readable medium according to claim 29,

wherein said medium includes instructions which when executed by a processor cause the processor to filter a second noise reference that is based on information from the multichannel sensed audio signal input to obtain a third plurality of time-domain subband signals, and

wherein said instructions which when executed by a processor cause the processor to calculate a plurality of second subband power estimates, when executed by the processor cause the processor to calculate the plurality of second subband power estimates based on information from the third plurality of time-domain subband signals.

31. The computer-readable medium according to claim 30, wherein the second noise reference is an unseparated sensed audio signal.

32. The computer-readable medium according to claim 30, wherein the second noise reference is based on the source signal.

33. The computer-readable medium according to claim 30,

wherein said instructions which when executed by a processor cause the processor to calculate a plurality of second subband power estimates include instructions which when executed by a processor cause the processor to:

based on information from the second plurality of time-domain subband signals, calculate a plurality of first noise subband power estimates; and

based on information from the third plurality of time-domain subband signals, calculate a plurality of second noise subband power estimates, and

wherein said instructions which when executed by a processor cause the processor to calculate a plurality of second subband power estimates, when executed by the processor cause the processor to calculate each of the plurality of second subband power estimates based on the maximum of (A) a corresponding one of the plurality of first noise subband power estimates and (B) a corresponding one of the plurality of second noise subband power estimates.

34. The computer-readable medium according to claim 29, wherein the multichannel sensed audio signal input includes a directional component and a noise component, and

wherein said instructions which when executed by a processor cause the processor to perform a spatially selective processing operation include instructions which when executed by a processor cause the processor to separate energy of the directional component from energy of the noise component such that the source signal contains more of the energy of the directional component than each channel of the multichannel sensed audio signal input does.

35. The computer-readable medium according to claim 29, wherein said instructions which when executed by a processor cause the processor to filter the reproduced audio signal input to obtain a first plurality of time-domain subband signals include instructions which when executed by a processor cause the processor to obtain each among the first plurality of time-domain subband signals by boosting a gain of a corresponding subband of the reproduced audio signal input relative to other subbands of the reproduced audio signal.

36. The computer-readable medium according to claim 29, wherein said medium includes instructions which when executed by a processor cause the processor to calculate, for each of the plurality of first subband power estimates, a gain factor based on a ratio of (A) the first subband power estimate and (B) a corresponding one of the plurality of second subband power estimates;

and

wherein said instructions which when executed by a processor cause the processor to boost at least one frequency subband of the reproduced audio signal input relative to at least one other frequency subband of the reproduced audio signal input include instructions which when executed by a processor cause the processor to apply, for each of the plurality of first subband power estimates, a gain factor based on the corresponding calculated ratio to a corresponding frequency subband of the reproduced audio signal input.

37. The computer-readable medium according to claim 36, wherein said instructions which when executed by a processor cause the processor to boost at least one frequency subband of the reproduced audio signal input relative to at least one other frequency subband of the reproduced audio signal input include instructions which when executed by a processor cause the processor to filter the reproduced audio signal input using a cascade of filter stages, and

wherein said instructions which when executed by a processor cause the processor to apply, for each of the plurality of first subband power estimates, a gain factor to a corresponding frequency subband of the reproduced audio signal input include instructions which when executed by a processor cause the processor to apply the gain factor to a corresponding filter stage of the cascade.

38. The computer-readable medium according to claim 36, wherein said instructions which when executed by a processor cause the processor to calculate a gain factor include instructions which when executed by a processor cause the processor to constrain a current value of the corresponding gain factor, for at least one of the plurality of first subband power estimates, by at least one bound that is based on a current level of the reproduced audio signal.

39. The computer-readable medium according to claim 36, wherein said instructions which when executed by a processor cause the processor to calculate a gain factor include instructions which when executed by a processor cause the processor to smooth, for at least one of the plurality of first subband power estimates, a value of the corresponding gain factor over time according to a change in the value of the corresponding ratio over time.

40. An apparatus comprising:

means for performing a spatially selective processing operation on a first input, wherein the first input is a multichannel sensed audio signal input, to produce a source signal and a noise reference;

means for filtering a second input, wherein the second input is a reproduced audio signal input, to obtain a first plurality of time-domain subband signals;

means for filtering the noise reference to obtain a second plurality of time-domain subband signals;

means for calculating a plurality of first subband power estimates based on information from the first plurality of time-domain subband signals;

means for calculating a plurality of second subband power estimates based on information from the second plurality of time-domain subband signals; and

means for boosting at least one frequency subband of the reproduced audio signal input relative to at least one other frequency subband of the reproduced audio signal input, based on information from the plurality of first subband power estimates and on information from the plurality of second subband power estimates.

41. The apparatus according to claim 40,

wherein said apparatus includes means for filtering a second noise reference that is based on information from the multichannel sensed audio signal input to obtain a third plurality of time-domain subband signals, and

wherein said means for calculating a plurality of second subband power estimates is configured to calculate the plurality of second subband power estimates based on information from the third plurality of time-domain subband signals.

42. The apparatus according to claim 41, wherein the second noise reference is an unseparated sensed audio signal.

43. The apparatus according to claim 41, wherein the second noise reference is based on the source signal.

44. The apparatus according to claim 41,

wherein said means for calculating a plurality of second subband power estimates is configured to calculate (A) a plurality of first noise subband power estimates based on information from the second plurality of time-domain subband signals and (B) a plurality of second noise subband power estimates based on information from the third plurality of time-domain subband signals, and

wherein said means for calculating a plurality of second subband power estimates is configured to calculate each of the plurality of second subband power estimates based on the maximum of (A) a corresponding one of the plurality of first noise subband power estimates and (B) a corresponding one of the plurality of second noise subband power estimates.

45. The apparatus according to claim 40,

wherein the multichannel sensed audio signal input includes a directional component and a noise component, and

wherein said means for performing a spatially selective processing operation is configured to separate energy of the directional component from energy of the noise component such that the source signal contains more of the energy of the directional component than each channel of the multichannel sensed audio signal input does.

46. The apparatus according to claim 40, wherein said means for filtering the reproduced audio signal input is configured to obtain each among the first plurality of time-domain subband signals by boosting a gain of a corresponding subband of the reproduced audio signal input relative to other subbands of the reproduced audio signal input.

47. The apparatus according to claim 40,

wherein said apparatus includes means for calculating, for each of the plurality of first subband power estimates, a gain factor based on a ratio of (A) the first subband power estimate and (B) a corresponding one of the plurality of second subband power estimates;

and

wherein said means for boosting is configured to apply a gain factor based on the corresponding calculated ratio, for each of the plurality of first subband power estimates, to a corresponding frequency subband of the reproduced audio signal.

48. The apparatus according to claim 47,

wherein said means for boosting includes a cascade of filter stages, and

wherein said means for boosting is configured to apply each of the plurality of gain factors to a corresponding filter stage of the cascade.

49. The apparatus according to claim 47, wherein said means for calculating a gain factor is configured to constrain a current value of the corresponding gain factor, for at least one of the plurality of first subband power estimates, by at least one bound that is based on a current level of the reproduced audio signal.

50. The apparatus according to claim 47, wherein said means for calculating a gain factor is configured to smooth a value of the corresponding gain factor over time, for at least one of the plurality of first subband power estimates, according to a change in the value of the corresponding ratio over time.