Controlling loudness of speech in signals that contain speech and other types of audio material

• US 20040044525A1
• Filed: 08/30/2002
• Published: 03/04/2004
• Est. Priority Date: 08/30/2002
• Status: Active Grant

• Current Assignee: Cutanix Corporation, Dolby Laboratories Licensing Corporation (Dolby Laboratories Incorporated)
• Original Assignee: Dolby Laboratories Licensing Corporation (Dolby Laboratories Incorporated)
• Inventors: Gundry, Kenneth James, Riedmiller, Jeffrey Charles, Vinton, Mark Stuart, Robinson, Charles Quito, Venezia, Steven Joseph
• Granted Patent: US 7,454,331 B2
• Time in Patent Office: Days
• US Class Current: 704/224
• CPC Class Codes: H03G 5/165 Equalizers; Volume or gain control in limited frequency bands

Abstract

An indication of the loudness of an audio signal containing speech and other types of audio material is obtained by classifying segments of audio information as either speech or non-speech. The loudness of the speech segments is estimated and this estimate is used to derive the indication of loudness. The indication of loudness may be used to control audio signal levels so that variations in loudness of speech between different programs is reduced. A preferred method for classifying speech segments is described.

Claims

1. A method for signal processing that comprises:

receiving an input signal and obtaining audio information from the input signal, wherein the audio information represents an interval of an audio signal;

examining the audio information to classify segments of the audio information as being speech segments representing portions of the audio signal classified as speech or as being non-speech segments representing portions of the audio signal not classified as speech, wherein each portion of the audio signal represented by a segment has a respective loudness, and the loudness of the speech segments is less than the loudness of one or more loud non-speech segments;

examining the audio information to obtain an estimated loudness of the speech segments; and

providing an indication of the loudness of the interval of the audio signal by generating control information that is more responsive to the estimated loudness of the speech segments than to the loudness of the portions of the audio signal represented by the non-speech segments.

2. The method according to claim 1 that comprises:

controlling the loudness of the interval of the audio signal in response to the control information so as to reduce variations in the loudness of the speech segments, wherein the loudness of the portions of the audio signal represented by the one or more loud non-speech segments is increased when the loudness of the portions of the audio signal represented by the speech-segments is increased.

3. The method according to claim 1 that comprises:

assembling a representation of the audio information and the control information into an output signal and transmitting the output signal.

4. The method according to claim 1 or 2 that obtains the estimated loudness of the speech segments by calculating average power of a frequency-weighted version of the audio signal represented by the speech segments.

5. The method according to claim 1 or 2 that obtains the estimated loudness of the speech segments by applying a psychoacoustic model of loudness to the audio information.

6. The method according to claim 1 or 2 that classifies segments by deriving from the audio information a plurality of characteristics of the audio signal, weighting each characteristic by a respective measure of importance, and classifying the segments according to a combination of the weighted characteristics.

7. The method according to claim 1 or 2 that controls the loudness of the interval of the audio signal by adjusting the loudness only during intervals of the audio signal having a measure of audio energy less than a threshold.

8. The method according to claim 1 or 2 wherein the indication of the loudness of the interval of the audio signal is responsive only to the estimated loudness of the speech segments.

9. The method according to claim 1 or 2 that comprises estimating the loudness of one or more non-speech segments, wherein the indication of the loudness of the interval of the audio signal is more responsive to the estimated loudness of the speech segments than to the estimated loudness of the one or more non-speech segments.

10. The method according to claim 1 or 2 that comprises:

providing a speech measure that indicates a degree to which the audio signal represented by a respective segment has characteristics of speech; and

providing the indication of loudness such that it is responsive to the estimated loudness of respective segments according to the speech measures of the respective segments.

11. The method according to claim 1 or 2 that comprises providing the indication of loudness such that it is responsive to the estimated loudness of respective segments according to time order of the segments.

12. The method according to claim 1 or 2 that comprises adapting lengths of the segments of audio information in response to characteristics of the audio information.

13. A medium that is readable by a device and that conveys a program of instructions executable by the device to perform a method for signal processing that comprises steps performing the acts of:

receiving an input signal and obtaining audio information from the input signal, wherein the audio information represents an interval of an audio signal;

examining the audio information to classify segments of the audio information as being speech segments representing portions of the audio signal classified as speech or as being non-speech segments representing portions of the audio signal not classified as speech, wherein each portion of the audio signal represented by a segment has a respective loudness, and the loudness of the speech segments is less than the loudness of one or more loud non-speech segments;

examining the audio information to obtain an estimated loudness of the speech segments; and

providing an indication of the loudness of the interval of the audio signal by generating control information that is more responsive to the estimated loudness of the speech segments than to the loudness of the portions of the audio signal represented by the non-speech segments.

14. The medium of claim 13 wherein the method comprises:

controlling the loudness of the interval of the audio signal in response to the control information so as to reduce variations in the loudness of the speech segments, wherein the loudness of the portions of the audio signal represented by the one or more loud non-speech segments is increased when the loudness of the portions of the audio signal represented by the speech-segments is increased.

15. The medium of claim 13 wherein the method comprises:

assembling a representation of the audio information and the control information into an output signal and transmitting the output signal.

16. The medium according to claim 13 or 14 wherein the method obtains the estimated loudness of the speech segments by calculating average power of a frequency-weighted version of the audio signal represented by the speech segments.

17. The medium according to claim 13 or 14 wherein the method obtains the estimated loudness of the speech segments by applying a psychoacoustic model of loudness to the audio information.

18. The medium according to claim 13 or 14 wherein the method classifies segments by deriving from the audio information a plurality of characteristics of the audio signal, weighting each characteristic by a respective measure of importance, and classifying the segments according to a combination of the weighted characteristics.

19. The medium according to claim 13 or 14 wherein the method controls the loudness of the interval of the audio signal by adjusting the loudness only during intervals of the audio signal having a measure of audio energy less than a threshold.

20. The medium according to claim 13 or 14 wherein the indication of the loudness of the interval of the audio signal is responsive only to the estimated loudness of the speech segments.

21. The medium according to claim 13 or 14 wherein the method comprises estimating the loudness of one or more non-speech segments, wherein the indication of the loudness of the interval of the audio signal is more responsive to the estimated loudness of the speech segments than to the estimated loudness of the one or more non-speech segments.

22. The medium according to claim 13 or 14 wherein the method comprises:

providing a speech measure that indicates a degree to which the audio signal represented by a respective segment has characteristics of speech; and

providing the indication of loudness such that it is responsive to the estimated loudness of respective segments according to the speech measures of the respective segments.

23. The medium according to claim 13 or 14 wherein the method comprises providing the indication of loudness such that it is responsive to the estimated loudness of respective segments according to time order of the segments.

24. The medium according to claim 13 or 14 wherein the method comprises adapting lengths of the segments of audio information in response to characteristics of the audio information.

25. An apparatus for signal processing that comprises:

an input terminal that receives an input signal;

memory; and

processing circuitry coupled to the input terminal and the memory;

wherein the processing circuitry is adapted to;

receive an input signal and obtain audio information from the input signal, wherein the audio information represents an interval of an audio signal;

examine the audio information to classify segments of the audio information as being speech segments representing portions of the audio signal classified as speech or as being non-speech segments representing portions of the audio signal not classified as speech, wherein each portion of the audio signal represented by a segment has a respective loudness, and the loudness of the speech segments is less than the loudness of one or more loud non-speech segments;

examine the audio information to obtain an estimated loudness of the speech segments; and

provide an indication of the loudness of the interval of the audio signal by generating control information that is more responsive to the estimated loudness of the speech segments than to the loudness of the portions of the audio signal represented by the non-speech segments.

26. The apparatus according to claim 25 wherein the processing circuitry is adapted to control the loudness of the interval of the audio signal in response to the control information so as to reduce variations in the loudness of the speech segments, wherein the loudness of the portions of the audio signal represented by the one or more loud non-speech segments is increased when the loudness of the portions of the audio signal represented by the speech-segments is increased.

27. The apparatus according to claim 25 wherein the processing circuitry is adapted to assemble a representation of the audio information and the control information into an output signal and transmit the output signal.

28. The apparatus according to claim 25 or 26 wherein the processing circuitry is adapted to obtain the estimated loudness of the speech segments by calculating average power of a frequency-weighted version of the audio signal represented by the speech segments.

29. The apparatus according to claim 25 or 26 wherein the processing circuitry is adapted to obtain the estimated loudness of the speech segments by applying a psychoacoustic model of loudness to the audio information.

30. The apparatus according to claim 25 or 26 wherein the processing circuitry is adapted to classify segments by deriving from the audio information a plurality of characteristics of the audio signal, weight each characteristic by a respective measure of importance, and classify the segments according to a combination of the weighted characteristics.

31. The apparatus according to claim 25 or 26 wherein the processing circuitry is adapted to control the loudness of the interval of the audio signal by adjusting the loudness only during intervals of the audio signal having a measure of audio energy less than a threshold.

32. The apparatus according to claim 25 or 26 wherein the indication of the loudness of the interval of the audio signal is responsive only to the estimated loudness of the speech segments.

33. The apparatus according to claim 25 or 26 wherein the processing circuitry is adapted to estimate the loudness of one or more non-speech segments, wherein the indication of the loudness of the interval of the audio signal is more responsive to the estimated loudness of the speech segments than to the estimated loudness of the one or more non-speech segments.

34. The apparatus according to claim 25 or 26 wherein the processing circuitry is adapted to:

provide a speech measure that indicates a degree to which the audio signal represented by a respective segment has characteristics of speech; and

provide the indication of loudness such that it is responsive to the estimated loudness of respective segments according to the speech measures of the respective segments.

35. The apparatus according to claim 25 or 26 wherein the processing circuitry is adapted to provide the indication of loudness such that it is responsive to the estimated loudness of respective segments according to time order of the segments.

36. The apparatus according to claim 25 or 26 wherein the processing circuitry is adapted to detect characteristics of the audio information and adapt lengths of the segments of audio information in response to the detected characteristics.