Apparatus and a method for generating bandwidth extension output data

• US 8,612,214 B2
• Filed: 01/11/2011
• Issued: 12/17/2013
• Est. Priority Date: 07/11/2008
• Status: Active Grant

• Current Assignee: Fraunhofer Gesellschaft Zur Foerderung Der Angewandten Forsching E.V.
• Original Assignee: Fraunhofer Gesellschaft Zur Foerderung Der Angewandten Forsching E.V.
• Inventors: Neuendorf, Max, Grill, Bernhard, Kraemer, Ulrich, Multrus, Markus, Popp, Harald, Rettelbach, Nikolaus, Nagel, Frederik, Lohwasser, Markus, Gayer, Marc, Jander, Manuel, Bacigalupo, Virgilio
• Primary Examiner(s): Godbold, Douglas
• Application Number:
  US13/004,264
• Publication Number:
  US 20110202352A1
• Time in Patent Office: 1,071 Days
• Field of Search: 704200-201, 704500-502
• US Class Current: 704/201
• CPC Class Codes: G10L 19/0208 Subband vocoders
• CPC Class Codes: G10L 19/025 Detection of transients or attacks for time/frequency resolution switching
• CPC Class Codes: G10L 19/20 using sound class specific coding, hybrid encoders or object based coding
• CPC Class Codes: G10L 21/038 using band spreading techniques

Abstract

An apparatus for generating bandwidth extension output data for an audio signal has a noise floor measurer, a signal energy characterizer and a processor. The audio signal has components in a first frequency band and components in a second frequency band, the bandwidth extension output data are adapted to control a synthesis of the components in the second frequency band. The noise floor measurer measures noise floor data of the second frequency band for a time portion of the audio signal. The signal energy characterizer derives energy distribution data, the energy distribution data characterizing an energy distribution in a spectrum of the time portion of the audio signal. The processor combines the noise floor data and the energy distribution data to obtain the bandwidth extension output data.

Claims

1. An encoder for encoding an audio signal, the audio signal comprising components in a first frequency band and components in a second frequency band, the encoder comprising:

a core coder for encoding the components in the first frequency band to acquire an encoded audio signal;

an envelope data calculator for calculating bandwidth extension (BWE) data based on the components in the second frequency band, the envelope data calculator comprising an apparatus for generating bandwidth extension output data for the audio signal, the bandwidth extension output data being adapted to control a synthesis of the components in the second frequency band, the apparatus comprising;

a spectral envelope data calculator for calculating spectral envelope data of the second frequency band;

a noise floor measurer for measuring noise floor data of the second frequency band for a time portion of the audio signal;

a signal energy characterizer for deriving energy distribution data, in addition to the spectral envelope data, the energy distribution data characterizing an energy distribution in a spectrum of the time portion of the audio signal; and

a processor for combining the noise floor data, the energy distribution data and the spectral envelope data to obtain the bandwidth extension output data, wherein the bandwidth extension output data comprise the noise floor data the energy distribution data and the spectral envelope data; and

a bitstream payload formatter adapted for outputting a coded audio stream by combining the bandwidth extension output data with the encoded audio signal, wherein the processor is part of the bitstream payload formatter.

2. The encoder of claim 1, wherein the signal energy characterizer is configured to use, as energy distribution data, a sibilance parameter or a spectral tilt parameter, the sibilance parameter or spectral tilt parameter identifying an increasing or decreasing level of the audio signal with frequency.

3. The encoder of claim 2, wherein the signal energy characterizer is configured to use a first linear predictive coding coefficient as the sibilance parameter.

4. The encoder of claim 1, wherein the processor is configured to add the noise floor data and the spectral energy distribution data to a bitstream as the BWE output data.

5. The encoder of claim 1, wherein the processor is configured to change the noise floor data in accordance to the energy distribution data to acquire modified noise floor data, and wherein the processor is configured to add the modified noise floor data to a bitstream as the BWE output data.

6. The encoder of claim 5, wherein the change of the noise floor data is such that the modified noise floor is increased for an audio signal comprising more sibilance compared to an audio signal comprising less sibilance.

7. The encoder of claim 1, wherein the time portion covers an SBR frame, the SBR frame comprising a plurality of noise envelopes, and wherein the noise envelope data calculator is configured to calculate different BWE data for different noise envelopes of the plurality of noise envelopes.

8. The encoder of claim 1, wherein the envelope data calculator is configured to change a number of envelopes depending on a change of the measured noise floor data.

9. A method of encoding an audio signal, the audio signal comprising components in a first frequency band and components in a second frequency band, the method comprising:

encoding the components in the first frequency band to acquire an encoded audio signal;

calculating bandwidth extension data by an envelope data calculator based on the components in the second frequency band, calculating comprising generating bandwidth extension output data for the audio signal, the bandwidth extension output data being adapted to control a synthesis of the components in the second frequency band, generating bandwidth extension output data comprising;

calculating spectral envelope data of the second frequency band;

measuring noise floor data of the second frequency band for a time portion of the audio signal;

deriving energy distribution data, in addition to the spectral envelope data, the energy distribution data characterizing an energy distribution in a spectrum of the time portion of the audio signal; and

combining the spectral envelope data, the noise floor data and the energy distribution data to obtain the bandwidth extension output data;

wherein the bandwidth extension output data comprise the noise floor data, the energy distribution data, and the spectral envelope data, and

bitstream payload formatting and outputting a coded audio stream by combining the bandwidth extension output data with the encoded audio signal, wherein combining is part of bitstream payload formatting.

10. A bandwidth extension tool for generating components in a second frequency band of an audio signal based on bandwidth extension output data and based on a raw signal spectral representation for the components in the second frequency band, wherein the bandwidth extension output data comprise energy distribution data, the bandwidth extension tool comprising:

a noise floor modifier tool, which is configured to modify a transmitted noise floor in accordance to the energy distribution data, the energy distribution data characterizing an energy distribution in a spectrum of a time portion of the audio signal, to obtain a modified transmitted noise floor;

a noise floor calculation tool for calculating modified noise spectral lines and raw signal lines based on spectral envelope data, the modified transmitted noise floor and the raw signal spectral representation; and

a combiner for combining the raw spectral lines with the modified noise spectral lines to generate the components in the second frequency band with the modified noise floor,

wherein the noise floor modifier tool or the combiner comprises a hardware implementation.

11. The bandwidth extension tool of claim 10, wherein the audio signal comprises components in a first frequency band and the bandwidth extension output data further comprise transmitted noise floor data indicating a noise level for the noise floor, and

wherein the noise floor modifier tool is adapted

to increase the noise level in case the energy distribution data indicates an audio signal comprising more energy in the components of the second frequency band than in first frequency band, or

to decrease the noise level in case the energy distribution data indicates an audio signal comprising more energy in the components of the first frequency band than in the second frequency band.

12. A decoder for decoding a coded audio stream to acquire an audio signal comprising:

a bitstream deformatter separating an encoded signal and the BWE output data;

a bandwidth extension tool for generating components in a second frequency band of an audio signal based on bandwidth extension output data and based on a raw signal spectral representation for the components in the second frequency band, wherein the bandwidth extension output data comprise energy distribution data, the energy distribution data characterizing an energy distribution in a spectrum of a time portion of the audio signal, the bandwidth extension tool comprising;

a noise floor modifier tool, which is configured to modify a transmitted noise floor in accordance to the energy distribution data to obtain a modified transmitted noise floor;

a noise floor calculation tool for calculating modified noise spectral lines and raw spectral lines based on spectral envelope data, the modified transmitted noise floor and the raw signal spectral representation; and

a combiner for combining the raw signal lines with the modified noise spectral lines to generate the components in the second frequency band with the modified noise floor;

a core decoder for decoding components in a first frequency band from the encoded audio signal; and

a synthesis unit for synthesizing the audio signal by combining the components of the first and second frequency band,

wherein the bitstream deformatter, the bandwidth extension tool, the core decoder or the synthesis unit comprises a hardware implementation.

13. A method for decoding a coded audio stream to acquire an audio signal, the audio signal comprising components in a first frequency band and bandwidth extension output data, wherein the bandwidth extension output data comprise energy distribution data and noise floor data, the energy distribution data characterizing an energy distribution in a spectrum of a time portion of the audio signal, the method comprising:

separating from the coded audio stream an encoded audio signal and the BWE output data;

decoding components in a first frequency band from the encoded audio signal;

generating a raw signal spectral representation for components in a second frequency band from the components in the first frequency band;

modifying a noise floor in accordance to the energy distribution data and in accordance to the transmitted noise floor data to obtain a modified transmitted noise floor;

calculating modified noise spectral lines and raw spectral lines based on spectral envelope data, the modified transmitted noise floor and the raw spectral representation;

combining the raw spectral lines with the modified noise spectral lines to generate the components in the second frequency band with the modified noise floor; and

synthesizing the audio signal by combining the components of the first and second frequency band.

14. Non-transitory storage medium having stored thereon a computer program for performing, when running on a computer, a method of encoding an audio signal, the audio signal comprising components in a first frequency band and components in a second frequency band, the method comprising:

encoding the components in the first frequency band to acquire an encoded audio signal;

calculating bandwidth extension data by an envelope data calculator based on the components in the second frequency band, calculating comprising generating bandwidth extension output data for the audio signal, the bandwidth extension output data being adapted to control a synthesis of the components in the second frequency band, generating bandwidth extension output data comprising;

calculating spectral envelope data of the second frequency band;

measuring noise floor data of the second frequency band for a time portion of the audio signal;

deriving energy distribution data, in addition to the spectral envelope data, the energy distribution data characterizing an energy distribution in a spectrum of the time portion of the audio signal; and

combining the spectral envelope data, the noise floor data and the energy distribution data to obtain the bandwidth extension output data; and

wherein the bandwidth extension output data comprise the spectral envelope data, the noise floor data and the energy distribution data; and

bitstream payload formatting and outputting a coded audio stream by combining the bandwidth extension data with the encoded audio signal, wherein combining is part of bitstream payload formatting.

15. Non-transitory storage medium having stored thereon a computer program for performing, when running on a computer, a method for decoding a coded audio stream to acquire an audio signal, the audio signal comprising components in a first frequency band and bandwidth extension output data, wherein the bandwidth extension output data comprise energy distribution data and noise floor data, the energy distribution data characterizing an energy distribution in a spectrum of a time portion of the audio signal, the method comprising:

separating from the coded audio stream an encoded audio signal and the BWE output data;

decoding components in a first frequency band from the encoded audio signal;

generating a raw signal spectral representation for components in a second frequency band from the components in the first frequency band;

modifying a noise floor in accordance to the energy distribution data and in accordance to the transmitted noise floor data;

combining the raw signal spectral representation with the modified noise floor to generate the components in the second frequency band with the calculated noise floor; and

synthesizing the audio signal by combining the components of the first and second frequency band.

16. Non-transitory storage medium having stored thereon an encoded audio stream, comprising:

an encoded audio signal for components in a first frequency band of an audio signal;

noise floor data adapted to control a synthesis of a noise floor for components in a second frequency band of the audio signal;

energy distribution data adapted to control a modification of the noise floor; and

spectral envelope data describing a spectral envelope of the components in the second frequency band.

17. A method for generating components in a second frequency band of an audio signal based on bandwidth extension output data and based on a raw signal spectral representation for the components in the second frequency band, wherein the bandwidth extension output data comprise energy distribution data, the method comprising:

modifying a transmitted noise floor in accordance to the energy distribution data, the energy distribution data characterizing an energy distribution in a spectrum of a time portion of the audio signal, to obtain a modified transmitted noise floor;

calculating modified noise spectral lines and raw spectral lines based on spectral envelope data, the modified transmitted noise floor and the raw signal spectral representation; and

combining the raw spectral lines with the modified noise spectral lines to generate the components in the second frequency band with the modified noise floor.

18. Non-transitory storage medium having stored thereon a computer program for performing, when running on a computer, a method for generating components in a second frequency band of an audio signal based on bandwidth extension output data and based on a raw signal spectral representation for the components in the second frequency band, wherein the bandwidth extension output data comprise energy distribution data, the method comprising:

modifying a transmitted noise floor in accordance to the energy distribution data, the energy distribution data characterizing an energy distribution in a spectrum of a time portion of the audio signal, to obtain a modified transmitted noise floor;

calculating modified noise spectral lines and raw spectral lines based on spectral envelope data, the modified transmitted noise floor and the raw signal spectral representation; and

combining the raw spectral lines with the modified noise spectral lines to generate the components in the second frequency band with the modified noise floor.

19. An encoder for encoding an audio signal, the audio signal comprising components in a first frequency band and components in a second frequency band, the encoder comprising:

a core coder for encoding the components in the first frequency band to acquire an encoded audio signal;

an bandwidth extension output data calculator for calculating bandwidth extension output data based on the components in the second frequency band, the bandwidth extension output data calculator comprising;

a noise floor measurer for measuring noise floor data of the second frequency band for a time portion of the audio signal;

a signal energy characterizer for deriving energy distribution data, the energy distribution data characterizing an energy distribution in a spectrum of the time portion of the audio signal; and

a processor for combining the noise floor data and the energy distribution data to acquire the bandwidth extension output data, wherein the processor is configured to change the noise floor data in accordance to the energy distribution data to obtain modified noise floor data, wherein the processor is configured to change the noise floor data such that the modified noise floor is increased for an audio signal comprising more sibilance compared to an audio signal comprising less sibilance; and

a bitstream payload formatter adapted for outputting a coded audio stream by combining the bandwidth extension output data comprising the modified noise floor data with the encoded audio signal.

20. An method of encoding an audio signal, the audio signal comprising components in a first frequency band and components in a second frequency band, the method comprising:

encoding the components in the first frequency band to acquire an encoded audio signal;

calculating bandwidth extension output data based on the components in the second frequency band, the bandwidth extension output data being adapted to control a synthesis of the components in the second frequency band, the calculating bandwidth extension output data comprising;

measuring noise floor data of the second frequency band for a time portion of the audio signal;

deriving energy distribution data, the energy distribution data characterizing an energy distribution in a spectrum of the time portion of the audio signal; and

combining the noise floor data and the energy distribution data to acquire the bandwidth extension output data comprising changing the noise floor data in accordance to the energy distribution data to obtain modified noise floor data,

wherein the modified noise floor is increased for an audio signal comprising more sibilance compared to an audio signal comprising less sibilance; and

outputting a coded audio stream by combining the bandwidth extension output data comprising the modified noise floor data with the encoded audio signal.

21. Non-transitory storage medium having stored thereon a computer program for performing, when running on a computer, a method of encoding an audio signal, the audio signal comprising components in a first frequency band and components in a second frequency band, the method comprising:

encoding the components in the first frequency band to acquire an encoded audio signal;

calculating bandwidth extension output data based on the components in the second frequency band, the bandwidth extension output data being adapted to control a synthesis of the components in the second frequency band, the calculating bandwidth extension output data comprising;

measuring noise floor data of the second frequency band for a time portion of the audio signal;

deriving energy distribution data, the energy distribution data characterizing an energy distribution in a spectrum of the time portion of the audio signal; and

combining the noise floor data and the energy distribution data to acquire the bandwidth extension output data comprising changing the noise floor data in accordance to the energy distribution data to obtain modified noise floor data, wherein the modified noise floor is increased for an audio signal comprising more sibilance compared to an audio signal comprising less sibilance; and

outputting a coded audio stream by combining the bandwidth extension output data comprising the modified noise floor data with the encoded audio signal.