Equalization for audio mixing

• US 20030235317A1
• Filed: 09/18/2002
• Published: 12/25/2003
• Est. Priority Date: 06/24/2002
• Status: Active Grant

• Current Assignee: Avago Technologies International Sales Pte Limited (Broadcom, Inc.)
• Original Assignee: Agere Systems Incorporated (Broadcom, Inc.)
• Inventors: Baumgarte, Frank
• Granted Patent: US 7,039,204 B2
• Time in Patent Office: Days
• US Class Current: 381/119
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 19/008 Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing
• CPC Class Codes: G10L 19/0212 using orthogonal transformation
• CPC Class Codes: H04S 1/007 in which the audio signals are in digital form data reduction aspects thereof based on psychoacoustics G10L19/02
• CPC Class Codes: H04S 2420/03 Application of parametric coding in stereophonic audio systems
• CPC Class Codes: H04S 3/00 Systems employing more than two channels, e.g. quadraphonic H04S5/00, H04S7/00 take precedence
• CPC Class Codes: H04S 5/005 of the pseudo five- or more-channel type, e.g. virtual surround

Abstract

During mixing of an N-channel input signal to generate an M-channel output signal, the mixed channel signals are equalized (e.g., amplified) to maintain the overall energy/loudness level of the output signal substantially equal to the overall energy/loudness level of the input signal. In one embodiment, the N input channel signals are converted to the frequency domain on a frame-by-frame basis, and the overall spectral loudness of the N-channel input signal is estimated. After mixing the spectral components for the N input channel signals (e.g., using weighted summation), the overall spectral loudness of the resulting M mixed channel signals is also estimated. A frequency-dependent gain factor, which is based on the two loudness estimates, is applied to the spectral components of the M mixed channel signals to generate M equalized mixed channel signals. The M-channel output signal is generated by converting the M equalized mixed channel signals to the time domain.

Claims

1. A method for mixing an N-channel input audio signal to generate an M-channel output audio signal, where N and M are positive integers, the method comprising the steps of:

(a) generating a first loudness measure for the N-channel input audio signal;

(b) mixing N input channel signals corresponding to the N-channel input audio signal to generate M mixed channel signals;

(c) generating a second loudness measure for the M mixed channel signals; and

(d) equalizing the M mixed channel signals based on the first and second loudness measures to generate the M-channel output audio signal.

2. The invention of claim 1, wherein step (d) comprises the step of amplifying the M mixed channel signals using a gain factor that is based on the first and second loudness measures to maintain the overall loudness of the M-channel output audio signal substantially equal to the overall loudness of the N-channel input audio signal.

3. The invention of claim 1, wherein:

step (a) comprises the steps of;

(1) transforming the N-channel input audio signal from a time domain into a frequency domain; and

(2) generating a first spectral loudness measure for the input audio signal;

step (c) comprises the step of generating a second spectral loudness measure for the M mixed channel signals;

the first and second loudness measures are the first and second spectral loudness measures, respectively; and

step (d) comprises the steps of;

(1) equalizing the M mixed channel signals based on the first and second spectral loudness measures to generate M equalized mixed channel signals; and

(2) transforming the M equalized mixed channel signals from the frequency domain into the time domain to generate the M-channel output audio signal.

4. The invention of claim 3, wherein step (d)(1) comprises the step of amplifying the M mixed channel signals using a frequency-dependent gain factor that is based on the first and second spectral loudness measures to maintain the overall spectral loudness of the M-channel output audio signal substantially equal to the overall spectral loudness of the N-channel input audio signal.

5. The invention of claim 3, wherein the first and second spectral loudness measures are smoothed over time.

6. The invention of claim 3, wherein:

step (a)(1) comprises the step of applying a short-time Fourier transform (STFT) to each frame of each input channel signal; and

step (d)(2) comprises the step of applying an inverse STFT (ISTFT) to each frame of each equalized mixed channel signal.

7. The invention of claim 6, wherein consecutive frames of each input channel signal overlap in time.

8. The invention of claim 7, wherein:

step (b) comprises the step of mixing based on weighted summation to generate the M mixed channel signals from the N input channel signals;

step (d)(1) comprises the step of amplifying the M mixed channel signals using a frequency-dependent gain factor that is based on the first and second spectral loudness measures to maintain the overall spectral loudness of the M-channel output audio signal substantially equal to the overall spectral loudness of the N-channel input audio signal;

the first and second spectral loudness measures are smoothed over time;

when N>

M, the N-channel input audio signal is down-mixed to generate the M-channel output audio signal; and

when N<

M, the N-channel input audio signal is up-mixed to generate the M-channel output audio signal.

9. The invention of claim 1, wherein:

when N>

M, the N-channel input audio signal is down-mixed to generate the M-channel output audio signal; and

when N<

M, the N-channel input audio signal is up-mixed to generate the M-channel output audio signal.

10. The invention of claim 1, wherein step (b) comprises the step of mixing based on weighted summation to generate the M mixed channel signals from the N input channel signals.

11. An M-channel output audio signal generated by:

(a) generating a first loudness measure for an N-channel input audio signal;

(b) mixing N input channel signals corresponding to the N-channel input audio signal to generate M mixed channel signals, where N and M are positive integers;

(c) generating a second loudness measure for the M mixed channel signals; and

(d) equalizing the M mixed channel signals based on the first and second loudness measures to generate the M-channel output audio signal.

12. The invention of claim 11. wherein step (d) comprises the step of amplifying the M mixed channel signals using a gain factor that is based on the first and second loudness measures to maintain the overall loudness of the M-channel output audio signal substantially equal to the overall loudness of the N-channel input audio signal.

13. The invention of claim 11. wherein:

step (a) comprises the steps of;

(1) transforming the N-channel input audio signal from a time domain into a frequency domain; and

(2) generating a first spectral loudness measure for the input audio signal;

step (c) comprises the step of generating a second spectral loudness measure for the M mixed channel signals;

the first and second loudness measures are the first and second spectral loudness measures, respectively; and

step (d) comprises the steps of;

(1) equalizing the M mixed channel signals based on the first and second spectral loudness measures to generate M equalized mixed channel signals; and

(2) transforming the M equalized mixed channel signals from the frequency domain into the time domain to generate the M-channel output audio signal.

14. The invention of claim 13, wherein the first and second spectral loudness measures are smoothed over time.

15. An apparatus for mixing an N-channel input audio signal to generate an M-channel output audio signal, where N and M are positive integers, the apparatus comprising:

(a) a mixer configured to mix N input channel signals corresponding to the N-channel input audio signal to generate M mixed channel signals;

(b) one or more loudness estimators configured to generate a first loudness measure for the N-channel input audio signal and a second loudness measure for the M mixed channel signals; and

(c) an equalizer configured to equalize the M mixed channel signals based on the first and second loudness measures to generate the M-channel output audio signal.

16. The invention of claim 15, wherein the equalizer is configured to amplify the M mixed channel signals using a gain factor that is based on the first and second loudness measures to maintain the overall loudness of the M-channel output audio signal substantially equal to the overall loudness of the N-channel input audio signal.

17. The invention of claim 15, further comprising:

(d) a set of one or more transform blocks configured to transform the N-channel input audio signal from a time domain into a frequency domain; and

(e) a set of one or more inverse transform blocks configured to transform the M equalized mixed channel signals from the frequency domain into the time domain to generate the M-channel output audio signal.

18. The invention of claim 17, wherein:

the one or more loudness estimators are configured to generate a first spectral loudness measure for the input audio signal and a second spectral loudness measure for the M mixed channel signals;

the first and second loudness measures are the first and second spectral loudness measures, respectively; and

the equalizer is configured to equalize the M mixed channel signals based on the first and second spectral loudness measures to generate the M equalized mixed channel signals.

19. The invention of claim 18, wherein the equalizer is configured to amplify the M mixed channel signals using a frequency-dependent gain factor that is based on the first and second spectral loudness measures to maintain overall spectral loudness of the M-channel output audio signal substantially equal to the overall spectral loudness of the N-channel input audio signal.

20. The invention of claim 18, wherein the one or more loudness estimators are configured to smooth the first and second spectral loudness measures over time.