Enhancement of multichannel audio

• US 8,271,276 B1
• Filed: 05/03/2012
• Issued: 09/18/2012
• Est. Priority Date: 02/26/2007
• Status: Active Grant

• Current Assignee: Dolby Laboratories Licensing Corporation (Dolby Laboratories Incorporated)
• Original Assignee: Dolby Laboratories Licensing Corporation (Dolby Laboratories Incorporated)
• Inventors: Muesch, Hannes
• Primary Examiner(s): Godbold, Douglas
• Application Number:
  US13/463,600
• Publication Number:
  US 20120221328A1
• Time in Patent Office: 138 Days
• Field of Search: 704224-230
• US Class Current: 704/225
• CPC Class Codes: G10L 19/012 Comfort noise or silence coding
• CPC Class Codes: G10L 19/018 Audio watermarking, i.e. embedding inaudible data in the audio signal
• CPC Class Codes: G10L 2025/932 Decision in previous or following frames
• CPC Class Codes: G10L 2025/937 Signal energy in various frequency bands
• CPC Class Codes: G10L 21/02 Speech enhancement, e.g. noise reduction or echo cancellation reducing echo effects in line transmission systems H04B3/20; echo suppression in hands-free telephones H04M9/08
• CPC Class Codes: G10L 21/0364 for improving intelligibility
• CPC Class Codes: G10L 25/78 Detection of presence or absence of voice signals switching of direction of transmission by voice frequency in two-way loud-speaking telephone systems H04M9/10
• CPC Class Codes: G10L 25/93 Discriminating between voiced and unvoiced parts of speech signals G10L25/90 takes precedence

Abstract

The invention relates to audio signal processing. More specifically, the invention relates to enhancing multichannel audio, such as television audio, by applying a gain to the audio that has been smoothed between segments of the audio. The invention relates to methods, apparatus for performing such methods, and to software stored on a computer-readable medium for causing a computer to perform such methods.

Claims

1. A method for enhancing an audio signal, wherein the audio signal comprises two or more channels of audio content, the method comprising:

dividing the audio signal into segments;

examining the segments to determine whether the segments contain one or more indicia of speech, and if the one or more indicia are present in a segment, classifying the segment as a speech segment;

estimating a loudness of a speech component associated with the speech segment;

calculating a gain for the speech segment based at least in part on the estimated loudness, a reference loudness level, and an estimated loudness associated with a previous segment;

smoothing the calculated gain to control the rate at which the calculated gain changes from the speech segment to a second segment of the audio signal; and

applying the smoothed gain to the audio signal.

2. The method of claim 1 wherein the estimating further comprises analyzing the outputs of a filter bank.

3. The method of claim 1 wherein the estimating further comprises analyzing the outputs of a time-to-frequency domain transformation.

4. The method of claim 1 wherein the one or more indicia of speech includes interchannel phase difference.

5. The method of claim 1 wherein the one or more indicia of speech includes interchannel correlation.

6. The method of claim 1 wherein the applying the smoothed gain creates a substantially uniform perceived loudness for a listener of the audio content.

7. A non-transitory computer-readable storage medium encoded with a computer program for causing a computer to perform the method of claim 1.

8. A system for enhancing an audio signal, wherein the audio signal comprises two or more channels of audio content, the system comprising:

a controller that receives the audio signal, wherein the controller comprises a 30 buffer that temporarily stores segments of the audio signal as the segments are received;

a detection module that determines whether one or more of the stored segments contains characteristics of dialog, and if a segment is determined to contain characteristics of dialog, identifies the segment as a dialog segment;

an analysis module that estimates a power level of a speech component associated with the dialog segment; and

an enhancement processor that calculates a gain for the dialog segment and smooths the calculated gain to control the rate at which the gain changes from the dialog segment to a second segment of the audio signal, the gain being calculated based at least in part on, the estimated power level of the speech component and an estimated loudness associated with a previous segment.

9. The system of claim 8 wherein the enhancement processor calculates a gain for segments of only one of the two or more channels of audio content.

10. The system of claim 8 wherein the enhancement processor calculates a first gain for one of the two or more channels and a second gain another one of the two or more channels, wherein the first gain and the second gain are calculated independently.

11. The system of claim 8 wherein the power includes a loudness based on a spectral energy of the audio signal.

12. The system of claim 8 wherein the enhancement processor operates in accordance with one or more processing parameters and adjustment of the parameters is operative to urge a metric of speech intelligibility of the audio content above a desired threshold level.

13. The system of claim 8 wherein the enhancement processor calculates the gain based in part on the level of noise in the dialog segment.

14. The system of claim 8 wherein the enhancement processor is operative to perform an enhancement operation selected from the group consisting of dynamic range control, dynamic equalization, dynamic gain modification, spectral sharpening, speech extraction, and noise reduction.

15. The system of claim 8 wherein the system is implemented in one of an audio decoder, an audio encoder, and a non-transitory computer-readable storage medium.

16. The system of claim 8 wherein each of the segments includes a fixed quantity of audio samples.

17. The system of claim 8 wherein each of the segments includes audio samples corresponding to a frame of a video signal.

18. The system of claim 8 wherein the system is operative to generate an output audio stream with a substantially constant perceived loudness despite loudness level changes in the audio signal.

19. A method for signal processing comprising:

receiving an audio signal, wherein the audio signal comprises two or more channels of audio content;

analyzing features of the audio signal;

classifying a segment of the audio signal as a speech segment if the segment contains one or more features of speech;

analyzing the speech segment to obtain an estimated loudness of a speech component of the speech segment;

calculating a gain for the speech segment based at least in part on the estimated loudness, a reference loudness, and an estimated loudness associated with a previous segment; and

smoothing the calculated gain to control the rate at which the calculated gain changes from the speech segment to a second segment of the audio signal.