System for dynamic spectral correction of audio signals to compensate for ambient noise

• US 8,913,754 B2
• Filed: 06/05/2012
• Issued: 12/16/2014
• Est. Priority Date: 11/30/2011
• Status: Expired due to Fees

• Current Assignee: Sound Enhancement Technology, LLC
• Original Assignee: Sound Enhancement Technology, LLC
• Inventors: McClain, David
• Primary Examiner(s): Chin, Vivian
• Assistant Examiner(s): HAMID, AMMAR T
• Application Number:
  US13/489,255
• Publication Number:
  US 20130136266A1
• Time in Patent Office: 924 Days
• Field of Search: 381/57, 381 711- 7112, 381 941- 9, 381/86, 381/106, 704/500, 704/E21.001, 455/63, 455/432.3, 700/94
• US Class Current: 381/57
• CPC Class Codes: H03G 3/32 the control being dependent upon ambient noise level or sound level
• CPC Class Codes: H03G 9/005 of digital or coded signals
• CPC Class Codes: H03G 9/025 frequency-dependent volume compression or expansion, e.g. multiple-band systems H03G9/10, H03G9/18 take precedence

Abstract

The present invention features systems for adjusting audio signals by applying a gain to the signal in a spectrally varying manner to compensate for ambient noise in the environment of the listener. The system allows a listener to hear what ought to be heard, over the ambient noise, by applying a gain to the source that varies according to the spectral composition of the noise, rather than cancelling or filtering the noise. The spectral composition of the source is thus preserved in the listener'"'"'s awareness without the removal of the noise signal. After application of these corrective gains to the source, the listener'"'"'s perception of the source sound is as if the noise was not present. Systems may be incorporated into apparatuses including but not limited to mobile phones and music players.

Claims

1. An apparatus (200), effective in producing corrections to an audio signal which is a signal of interest, across a spectrum of frequencies, by applying corrective gains of amplitude to a plurality of frequency components which constitute the audio signal, wherein parallel compression is used to approximate gain curves derived from a psychoacoustic model, wherein the parallel compression is implemented by applying a gain G at a particular frequency component of the audio signal according to a linear compression and then combining this compressed signal with the audio signal, such that the combined parallel compression of the audio signal forms a non-linear compression curve, where the linear compression is found by the equation, G=g0+(P−

T)*(1/R−

1) where G is the gain in dB to be applied to the audio signal, where g0 is a makeup gain in dB, T is a threshold in dB, and R is a compression ratio, and where P is a sound intensity in dB of the audio signal at a frequency component, such that g0, T, and R are parameters that vary the shape of a resulting parallel compression curve, where the psychoacoustic model takes as inputs the audio signal loudness and an ambient noise signal loudness, where the model computes the gain in sound loudness, at a given frequency component, required to correct for the effect of the ambient noise signal, where the parameters g0, T, and R, for a plurality of noise loudness levels, are predetermined by fitting the parallel compression curves against the desired behavior, where a different set of parameters is retrieved at each instant depending on the ambient noise volume at each frequency component, where the dynamically changing noise loudness results in a dynamically changing selection of parameters for the parallel compression module, the apparatus (200) comprising;

(a) a first audio device (202), configured to extract a first audio signal (002) where the first audio signal is a broadband audio signal of interest, the first audio device operatively connected to a microprocessor (201);

(b) a second audio device (203), configured for extracting a second audio signal (003) where the second audio signal (003) is ambient broadband noise audio in the environment, the second audio device (203) operatively connected to the microprocessor (201);

(c) the microprocessor (201), configured to execute applications for;

i. a first frequency analysis module (204), configured to extract a plurality of frequency components from the first audio signal (002), ii. a second frequency analysis module (205), configured to extract a plurality of frequency components from the second audio signal (003), iii. a first power estimation block (206), configured to calculate a sound intensity in decibels of each frequency component of the first audio signal (002), iv. a second power estimation block (207), configured to calculate a sound intensity of decibels of each frequency component of the second audio signal (003), v. a parameter estimation module (211), which determines parameter settings for a parallel compression module, according to information stored in memory, vi. the parallel compression module (209), parameterized by parameters including a threshold T, a compression ratio R, and a makeup gain g0, where said parameters may by dynamically varied, where the parallel compression module is comprised of;

a. a linear compression module (112), which, using the parameters applies gains to the first audio signal according to the equation G=g0+(P−

T)*(1/R+1), where G is the gain in dB to be applied to a frequency component of the first audio signal, P is the sound intensity of the first signal at the frequency component in dB, g0 is the makeup gain in dB, T is the threshold in dB, and R is the compression ratio of the linear compression, b. a summer (110), which combines the resulting amplified first audio signal component with the original first audio signal component, vii. a Frequency Synthesis Module (210), which combines the frequency components of the resulting first audio signal, wherein (i) the first audio device (202) extracts the first audio signal (002), whereupon the first audio signal (002) is then transmitted through a first analog to digital converter (71), thereby converting the first audio signal (002) into digital format, and simultaneously, (ii) the second audio device (203) extracts the second audio signal (003), where said second audio signal (003) is then transmitted through a second analog to digital converter (72) thereby converting the second audio signal (003) into digital format, whereupon the first audio signal x(t) (002) and the second audio signal xo(t) (003) are fed, respectively, through the First Frequency Analysis Module (204) and Second Frequency Analysis Module (205) breaking down each respective audio signal into arrays of frequency components, whereupon, for each frequency component, the first audio signal (002) and the second audio signal (003) are fed, respectively, through the first Power Estimation Block (206) and Second Power Estimation Block (207), resulting in estimates, for each frequency component, and at each instant, for the sound intensity of each signal in decibels, whereupon, for each frequency component, and at each instant, the microprocessor (201), using the power estimates of the frequency components of the first (002) and second (003) audio signals, obtains parameters for parallel compression (211), depending on the sound intensity of the second audio signal (003) at each instant, whereupon the microprocessor (201), applies the parameters to the parallel compression module (209), whereupon the parallel compression module (209) applies linear compression, according to the selected parameters, to each of the frequency components of the first audio signal (002), whereupon the parallel compression module (209) sums each frequency component of the first audio signal (002) with the corresponding compressed audio signal, whereupon the microprocessor (201) repeats the process of estimating the power of both the first audio signal and the second audio signal, selecting compression settings, and applying parallel compression for each frequency component of the first audio signal (002), whereupon the microprocessor (201) reconstitutes the first audio signal (002) by feeding the now parallel compressed frequency components of the parallel compressed first audio signal (002) through the Frequency Synthesis Module (210), thereby obtaining an output Xout (t).

2. The apparatus of claim 1, where the First (204) and Second (205) Frequency Analysis Modules are composed of a digital filter banks (304), (305) of filters and output and the frequency components comprise time-domain arrays of band-limited frequency channels for the first (002) and second (003) audio signals, and where the Frequency Synthesis Module is a summer block that summers together the channels of the audio signals after gains have been applied.

3. The apparatus of claim 2, where the frequency sub-bands correspond to critical bands of hearing.

4. The apparatus of claim 1, where the First and second Frequency Analysis Modules are digital filter banks composed of wavelet filters.

5. The apparatus of claim 1, where the First (204) and Second (205) Frequency Analysis modules are composed of FFT co-processors (411) and the frequency components comprise output Fourier transform components for the first (002) and second (003) audio signals, and where the Frequency Synthesis Module (210) is an IFFT co-processor (410).

6. The apparatus of claim 5, where the frequency components are grouped together to approximate sub-bands correspond to critical bands of hearing.

7. The apparatus of claim 1, where a plurality of desired dynamic range compression curves approximate the ideal compression curves needed to compensate for a plurality of ambient noise signal intensities, so that the derived parallel compression parameters correspond to a plurality of ambient noise loudness levels.

8. The apparatus of claim 1 or claim 7, where discrete sets of parameters are found by dividing the range of possible noise volumes into discrete levels, and where sets of parameters are found for each discrete noise loudness level, where the parameter estimation module (211) is a lookup table, where during real-time processing the ambient noise intensity is measured and used to select the corresponding set of parameters.

9. The apparatus of claim 1 or claim 7, where the parameter estimation module (211) is a polynomial continuous curve defining the relationship between the ambient noise loudness and each of the parameters, where polynomial regression is used to find parameters which define this curve, where these parameters are then stored in memory, so that during processing the parameters can then be computed using a polynomial equation.

10. The apparatus of claim 1 or claim 7, where one or more parameters (e.g. T) is defined as a function of the ambient noise intensity (e.g. T=PN−

T′

, where PN is noise loudness), such that the parameters stored (e.g. T′

) are the parameters of the function defining the relationship between sound intensity (e.g. PN) and the parameter (e.g. T) of the parallel compression.

11. The apparatus of claim 1 or claim 7, where the gain equation is expressed as a function of the difference between signal intensity and noise intensity:

G=g0+(dP−

T′

)/R−

dP where dP is the difference between the signal loudness and noise loudness in decibels, and T′

is the threshold offset from the ambient noise intensity to the threshold T.

12. The apparatus of claim 1, where a fixed set of parameters is used.

13. The apparatus of claim 1, where the fixed parameters fall in the range of 2 to 6 for compression ratio, 15 dB to 25 dB for makeup gain, and −

15 to −

25 dB for threshold offset.

14. The apparatus of claim 1, wherein the apparatus is a mobile phone, media player, or headset.

15. The apparatus of claim 1, wherein the processing of all frequency components of both signals are computed in parallel using multiple processing units, such as on a parallel processing device.

16. The apparatus of claim 1, where the psychoacoustic model comprises a formula which computes the gain as a function of the signal and ambient noise levels which is needed to make the sound appear as loud as if the noise was not present, this formula comprising:

where G is the gain ratio, PSI G is the signal intensity at a frequency in units of power, PNOI SE is the signal intensity of the background noise, PTHRQ is the absolute threshold of hearing, and a=0.2 is a constant.

17. A method of using parallel compression to approximate a desired non-linear dynamic range compression curve, wherein a compression curve defines the output loudness in decibels versus the input loudness in decibels of a signal of interest, wherein the compression is implemented by applying a gain G to the signal of interest according to the following equation:

G=g0+(P−

T)*(1/R−

1) where G is the gain in dB to be applied to a frequency component of the signal of interest, P is the sound intensity of the frequency component in dB, g0 is the makeup gain in dB, T is the threshold in dB, and R is the compression ratio of linear compression, whereupon the linearly compressed signal is added together with the original signal of interest, producing a parallel compressed output signal, where the overall amplitude of the input versus the output signal defines a parallel compression curve, where g0, T, and R are thus parameters that vary the shape of the parallel compression curve, and P is the input loudness of the signal of interest, where the method is comprised of the following steps;

a. obtaining parameters that shape the parallel compression curve, by fitting compression curves using these parameters against a plurality of desired compression curves and selecting the parameters that minimize the difference between the compression curve of the parallel compression module and each of the desired compression curves;

b. storing the optimal sets of parameters in a memory device;

c. during real-time processing, at each time step;

1. retrieving a set of parameters, depending on the desired compression curve to be used, 2. applying the set of parameters, to the parallel compression module, and processing a first audio signal using the time-varying compression parameters in the parallel compression module, where the parallel compression module is performed by;

a. applying linear compression to the first audio signal according to the equation G=g0+(P−

T)*(1/R+1), producing a linearly compressed audio signal, and b. summing the linearly compressed audio signal with the first audio signal.

18. The method of claim 17, where the plurality of desired dynamic range compression curves approximate the ideal compression curves needed to compensate for a plurality of ambient noise signal intensities, so that the derived parallel compression parameters correspond to a plurality of ambient noise loudness levels.

19. The method of claim 18, where discrete sets of parameters are found by dividing the range of possible noise volumes into discrete levels, and where sets of parameters are found for each discrete noise loudness level, where the sets of parallel compression parameters are stored in a lookup table, and where during real-time processing the ambient noise intensity is measured and used to select the corresponding set of parameters.

20. The method of claim 18, where polynomial regression is used to fit a continuous curve between the ambient noise loudness and each of the parameters, where the parameters of this curve are stored, and where during real time processing the ambient noise loudness is measured and applied to the polynomial function to obtain each parameter T, R, and g0.