Equalization for audio mixing

• US 7,039,204 B2
• Filed: 09/18/2002
• Issued: 05/02/2006
• Est. Priority Date: 06/24/2002
• Status: Active Grant

• Current Assignee: Avago Technologies International Sales Pte Limited (Broadcom, Inc.)
• Original Assignee: Agere Systems Incorporated (Broadcom, Inc.)
• Inventors: Baumgarte, Frank
• Primary Examiner(s): Pendleton, Brian T.
• Application Number:
  US10/246,165
• Publication Number:
  US 20030235317A1
• Time in Patent Office: 1,322 Days
• Field of Search: 381/27, 381/17, 381/18, 381/119, 381/98, 381/103, 381/104, 381/107, 381/108, 704/226, 704/233
• US Class Current: 381/119
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 19/008 Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing
• CPC Class Codes: G10L 19/0212 using orthogonal transformation
• CPC Class Codes: H04S 1/007 in which the audio signals are in digital form data reduction aspects thereof based on psychoacoustics G10L19/02
• CPC Class Codes: H04S 2420/03 Application of parametric coding in stereophonic audio systems
• CPC Class Codes: H04S 3/00 Systems employing more than two channels, e.g. quadraphonic H04S5/00, H04S7/00 take precedence
• CPC Class Codes: H04S 5/005 of the pseudo five- or more-channel type, e.g. virtual surround

Abstract

During mixing of an N-channel input signal to generate an M-channel output signal, the mixed channel signals are equalized (e.g., amplified) to maintain the overall energy/loudness level of the output signal substantially equal to the overall energy/loudness level of the input signal. In one embodiment, the N input channel signals are converted to the frequency domain on a frame-by-frame basis, and the overall spectral loudness of the N-channel input signal is estimated. After mixing the spectral components for the N input channel signals (e.g., using weighted summation), the overall spectral loudness of the resulting M mixed channel signals is also estimated. A frequency-dependent gain factor, which is based on the two loudness estimates, is applied to the spectral components of the M mixed channel signals to generate M equalized mixed channel signals. The M-channel output signal is generated by converting the M equalized mixed channel signals to the time domain.

Claims

1. A method for mixing an N-channel input audio signal to generate an M-channel output audio signal, where N and Mare positive integers, the method comprising the steps of:

(a) transforming the N-channel input audio signal from a time domain into a frequency domain;

(b) generating a first spectral loudness measure for the N-channel input audio signal;

(c) mixing N frequency-domain input channel signals corresponding to the N-channel input audio signal to generate M frequency-domain mixed channel signals;

(d) generating a second spectral loudness measure for the M frequency-domain mixed channel signals;

(e) equalizing the M frequency-domain mixed channel signals using a frequency-dependent gain factor that is based on the first and second spectral loudness measures to generate M equalized mixed channel signals; and

(f) transforming the M equalized mixed channel signals from the frequency domain into the time domain to generate the M-channel output audio signal.

2. The invention of claim 1, wherein step (e) comprises the step of amplifying the M frequency-domain mixed channel signals using the frequency-dependent gain factor that is based on the first and second spectral loudness measures to maintain the overall spectral loudness of the M-channel output audio signal substantially equal to the overall spectral loudness of the N-channel input audio signal.

3. The invention of claim 1, wherein the first and second spectral loudness measures are smoothed over time.

4. The invention of claim 1, wherein:

step (a)(

     1) comprises the step of applying a short-time Fourier transfonn (STFT) to each frame of each input channel signal; and

step (d)(2) comprises the step of applying an inverse STFT (ISTFT) to each frame of each equalized mixed channel signal.

5. The invention of claim 4, wherein consecutive frames of each input channel signal overlap in time.

6. The invention of claim 5, wherein:

step (b) comprises the step of mixing based on weighted summation to generate the M frequency-domain mixed channel signals from the N frequency-domain input channel signals;

step (e) comprises the step of amplifying the M frequency-domain mixed channel signals using the frequency-dependent gain factor that is based on the first and second spectral loudness measures to maintain the overall spectral loudness of the M-channel output audio signal substantially equal to the overall spectral loudness of the N-channel input audio signal;

the first and second spectral loudness measures are smoothed over time;

when N>

M, the N-channel input audio signal is down-mixed to generate the M-channel output audio signal; and

when N<

M, the N-channel input audio signal is up-mixed to generate the M-channel output audio signal.

7. The invention of claim 1, wherein:

when N>

M, the N-channel input audio signal is down-mixed to generate the M-channel output audio signal; and

when N<

M, the N-channel input audio signal is up-mixed to generate the M-channel output audio signal.

8. The invention of claim 1, wherein step (c) comprises the step of mixing based on weighted summation to generate the M frequency-domain mixed channel signals from the N frequency-domain input channel signals.

15. The invention of claim 1, wherein the frequency-dependent gain factor g_k(ω

) is given by;

$g_k<br><br>\left(\mathrm{\omega <br><br>}\right)={\left[\frac{{\mathrm{\Gamma <br><br>}}_k<br><br>\left(\mathrm{\omega <br><br>}\right)}{{\mathrm{\Lambda <br><br>}}_k<br><br>\left(\mathrm{\omega <br><br>}\right)+\mathrm{\varepsilon <br><br>}}\right]}^{\frac{1}{\mathrm{\beta <br><br>}}},$

where;

Γ

_k(ω

) is the first loudness measure;

Λ

_k(ω

) is the second loudness measure;

ε

is a positive constant; and

β

is a constant larger than 1.

9. An M-channel output audio signal generated by:

(a) transforming the N-channel input audio signal from a time domain into a frequency domain;

(b) generating a first spectral loudness measure for the N-channel input audio signal;

(c) mixing N frequency-domain input channel signals corresponding to the N-channel input audio signal to generate M frequency-domain mixed channel signals, where N and M are positive integers;

(d) generating a second spectral loudness measure for the M frequency-domain mixed channel signals;

(e) equalizing the M frequency-domain mixed channel signals using a frequency-dependent gain factor that is based on the first and second spectral loudness measures to generate M equalized mixed channel signals; and

(f) transforming the M equalized mixed channel signals from the frequency domain into the time domain to generate the M-channel output audio signal.

10. The invention of claim 9, wherein step (e) comprises the step of amplifying the M frequency-domain mixed channel signals using the frequency-dependent gain factor to maintain the overall spectral loudness of the M-channel output audio signal substantially equal to the overall spectral loudness of the N-channel input audio signal.

11. The invention of claim 9, wherein the first and second spectral loudness measures are smoothed over time.

16. The invention of claim 9, wherein the frequency-dependent gain factor g_k(ω

) is given by;

$g_k<br><br>\left(\mathrm{\omega <br><br>}\right)={\left[\frac{{\mathrm{\Gamma <br><br>}}_k<br><br>\left(\mathrm{\omega <br><br>}\right)}{{\mathrm{\Lambda <br><br>}}_k<br><br>\left(\mathrm{\omega <br><br>}\right)+\mathrm{\varepsilon <br><br>}}\right]}^{\frac{1}{\mathrm{\beta <br><br>}}},$

where;

Γ

_k(ω

) is the first loudness measure;

Λ

_k(ω

) is the second loudness measure;

ε

is a positive constant; and

β

is a constant larger than 1.

12. An apparatus for mixing an N-channel input audio signal to generate an M-channel output audio signal, where N and M are positive integers, the apparatus comprising:

a set of one or more transform blocks configured to transform the N-channel input audio signal from a time domain into a frequency domain;

a mixer configured to mix N frequency-domain input channel signals corresponding to the N-channel input audio signal to generate M frequency-domain mixed channel signals;

one or more loudness estimators configured to generate a first spectral loudness measure for the N-channel input audio signal and a second spectral loudness measure for the M frequency-domain mixed channel signals;

an equalizer configured to equalize the M frequency-domain mixed channel signals using a frequency-dependent gain factor that is based on the first and second spectral loudness measures to generate M equalized mixed channel signals; and

a set of one or more inverse transform blocks configured to transform the M equalized mixed channel signals from the frequency domain into the time domain to generate the M-channel output audio signal.

13. The invention of claim 12, wherein the equalizer is configured to amplify the M frequency-domain mixed channel signals using the frequency-dependent gain factor that is based on the first and second spectral loudness measures to maintain overall spectral loudness of the M-channel output audio signal substantially equal to the overall spectral loudness of the N-channel input audio signal.

14. The invention of claim 12, wherein the one or more loudness estimators are configured to smooth the first and second spectral loudness measures over time.

17. The invention of claim 12, wherein the frequency-dependent gain factor g_k(ω

) is given by;

$g_k<br><br>\left(\mathrm{\omega <br><br>}\right)={\left[\frac{{\mathrm{\Gamma <br><br>}}_k<br><br>\left(\mathrm{\omega <br><br>}\right)}{{\mathrm{\Lambda <br><br>}}_k<br><br>\left(\mathrm{\omega <br><br>}\right)+\mathrm{\varepsilon <br><br>}}\right]}^{\frac{1}{\mathrm{\beta <br><br>}}},$

where;

Γ

_k(ω

) is the first loudness measure;

Λ

_k(ω

) is the second loudness measure;

ε

is a positive constant; and

β

is a constant larger than 1.

18. The invention of claim 12, wherein:

the set of one or more transform blocks are adapted to apply a short-time Fourier transform (STFT) to each frame of each input channel signal; and

the set of one or more inverse transform blocks are adapted to apply an inverse STFT (ISTFT) to each frame of each equalized mixed channel signal.

19. The invention of claim 18, wherein consecutive frames of each input channel signal overlap in time.

20. The invention of claim 12, wherein the mixer is adapted to mix based on weighted summation to generate the M frequency-domain mixed channel signals from the N frequency-domain input channel signals.

21. A method for mixing an N-channel input audio signal to generate an M-channel output audio signal, where N and M are positive integers, the method comprising the steps of:

(a) generating a first loudness measure for the N-channel input audio signal;

(b) mixing N input channel signals corresponding to the N-channel input audio signal to generate M mixed channel signals;

(c) generating a second loudness measure for the M mixed channel signals; and

(d) equalizing the M mixed channel signals using a frequency-dependent gain factor that is based on the first and second loudness measures to generate the M-channel output audio signal, wherein the frequency-dependent gain factor g_k(ω

) is given by;

$g_k<br><br>\left(\mathrm{\omega <br><br>}\right)={\left[\frac{{\mathrm{\Gamma <br><br>}}_k<br><br>\left(\mathrm{\omega <br><br>}\right)}{{\mathrm{\Lambda <br><br>}}_k<br><br>\left(\mathrm{\omega <br><br>}\right)+\mathrm{\varepsilon <br><br>}}\right]}^{\frac{1}{\mathrm{\beta <br><br>}}},$

where;

Γ

_k(ω

) is the first loudness measure;

Λ

_k(ω

) is the second loudness measure;

ε

is a positive constant; and

β

is a constant larger than 1.

22. An M-channel output audio signal generated by:

(a) generating a first loudness measure for an N-channel input audio signal;

(b) mixing N input channel signals corresponding to the N-channel input audio signal to generate M mixed channel signals, where N and M are positive integers;

(c) generating a second loudness measure for the M mixed channel signals; and

(d) equalizing the M mixed channel signals using a frequency-dependent gain factor that is based on the first and second loudness measures to generate the M-channel output audio signal, wherein the frequency-dependent gain factor g_k(ω

) is given by;

$g_k<br><br>\left(\mathrm{\omega <br><br>}\right)={\left[\frac{{\mathrm{\Gamma <br><br>}}_k<br><br>\left(\mathrm{\omega <br><br>}\right)}{{\mathrm{\Lambda <br><br>}}_k<br><br>\left(\mathrm{\omega <br><br>}\right)+\mathrm{\varepsilon <br><br>}}\right]}^{\frac{1}{\mathrm{\beta <br><br>}}},$

where;

Γ

_k(ω

) is the first loudness measure;

Λ

_k(ω

) is the second loudness measure;

ε

is a positive constant; and

β

is a constant larger than 1.

23. An apparatus for mixing an N-channel input audio signal to generate an M-channel output audio signal, where N and M are positive integers, the apparatus comprising:

(a) a mixer configured to mix N input channel signals corresponding to the N-channel input audio signal to generate M mixed channel signals;

(b) one or more loudness estimators configured to generate a first loudness measure for the N-channel input audio signal and a second loudness measure for the M mixed channel signals; and

(c) an equalizer configured to equalize the M mixed channel signals using a frequency-dependent gain factor that is based on the first and second loudness measures to generate the M-channel output audio signal, wherein the frequency-dependent gain factor g_k(ω

) is given by;

$g_k<br><br>\left(\mathrm{\omega <br><br>}\right)={\left[\frac{{\mathrm{\Gamma <br><br>}}_k<br><br>\left(\mathrm{\omega <br><br>}\right)}{{\mathrm{\Lambda <br><br>}}_k<br><br>\left(\mathrm{\omega <br><br>}\right)+\mathrm{\varepsilon <br><br>}}\right]}^{\frac{1}{\mathrm{\beta <br><br>}}},$

where;

Γ

_k(ω

) is the first loudness measure;

Λ

_k(ω

) is the second loudness measure;

ε

is a positive constant; and

β

is a constant larger than 1.