AUDIO ENCODER AND DECODER USING A FREQUENCY DOMAIN PROCESSOR WITH FULL-BAND GAP FILLING AND A TIME DOMAIN PROCESSOR

• US 20170256267A1
• Filed: 01/24/2017
• Published: 09/07/2017
• Est. Priority Date: 07/28/2014
• Status: Active Grant

• Current Assignee: Fraunhofer Gesellschaft Zur Foerderung Der Angewandten Forsching E.V.
• Original Assignee: Fraunhofer Gesellschaft Zur Foerderung Der Angewandten Forsching E.V.
• Inventors: DISCH, Sascha, DIETZ, Martin, MULTRUS, Markus, FUCHS, Guillaume, RAVELLI, Emmanuel, NEUSINGER, Matthias, SCHNELL, Markus, SCHUBERT, Benjamin, GRILL, Bernhard
• Granted Patent: US 10,332,535 B2
• Time in Patent Office: Days
• CPC Class Codes: G10L 19/02 using spectral analysis, e.g. transform vocoders or subband vocoders
• CPC Class Codes: G10L 19/028 Noise substitution, i.e. substituting non-tonal spectral components by noisy source comfort noise for discontinuous speech transmission G10L19/012
• CPC Class Codes: G10L 19/032 Quantisation or dequantisation of spectral components
• CPC Class Codes: G10L 19/04 using predictive techniques
• CPC Class Codes: G10L 19/06 Determination or coding of the spectral characteristics, e.g. of the short-term prediction coefficients
• CPC Class Codes: G10L 19/18 Vocoders using multiple modes
• CPC Class Codes: G10L 19/20 using sound class specific coding, hybrid encoders or object based coding
• CPC Class Codes: G10L 19/24 Variable rate codecs, e.g. for generating different qualities using a scalable representation such as hierarchical encoding or layered encoding
• CPC Class Codes: G10L 19/265 Pre-filtering, e.g. high frequency emphasis prior to encoding
• CPC Class Codes: G10L 21/038 using band spreading techniques

Abstract

An audio encoder for encoding an audio signal has: a first encoding processor for encoding a first audio signal portion in a frequency domain, having: a time frequency converter for converting the first audio signal portion into a frequency domain representation; an analyzer for analyzing the frequency domain representation to determine first spectral portions to be encoded with a first spectral resolution and second regions to be encoded with a second resolution; and a spectral encoder for encoding the first spectral portions with the first spectral resolution and encoding the second portions with the second resolution; a second encoding processor for encoding a second different audio signal portion in the time domain; a controller for analyzing and determining, which portion of the audio signal is the first audio signal portion encoded in the frequency domain and which portion is the second audio signal portion encoded in the time domain; and an encoded signal former for forming an encoded audio signal having a first encoded signal portion for the first audio signal portion and a second encoded signal portion for the second portion.

Claims

1. An audio encoder for encoding an audio signal, comprising:

a first encoding processor for encoding a first audio signal portion in a frequency domain, wherein the first encoding processor comprises;

a time frequency converter for converting the first audio signal portion into a frequency domain representation comprising spectral lines up to a maximum frequency of the first audio signal portion;

an analyzer for analyzing the frequency domain representation up to the maximum frequency to determine first spectral portions to be encoded with a first spectral resolution and second spectral portions to be encoded with a second spectral resolution, the second spectral resolution being lower than the first spectral resolution, wherein the analyzer is configured to determine a first spectral portion from the first spectral portions, the first spectral portion being placed, with respect to frequency, between two second spectral portions from the second spectral portions;

a spectral encoder for encoding the first spectral portions with the first spectral resolution and for encoding the second spectral portions with the second spectral resolution, wherein the spectral encoder comprises a parametric coder for calculating spectral envelope information comprising the second spectral resolution from the second spectral portions;

a second encoding processor for encoding a second different audio signal portion in the time domain, wherein the second encoding processor comprises;

a sampling rate converter for converting the second audio signal portion to a lower sampling rate representation, the lower sampling rate being lower than a sampling rate of the audio signal, wherein the lower sampling rate representation does not comprise the high band of the input signal;

a time domain low band encoder for time domain encoding the lower sampling rate representation; and

a time domain bandwidth extension encoder for parametrically encoding the high band;

a controller configured for analyzing the audio signal and for determining, which portion of the audio signal is the first audio signal portion encoded in the frequency domain and which portion of the audio signal is the second audio signal portion encoded in the time domain; and

an encoded signal former for forming an encoded audio signal comprising a first encoded signal portion for the first audio signal portion and a second encoded signal portion for the second audio signal portion.

2. The audio encoder of claim 1, further comprising:

a preprocessor configured for preprocessing the first audio signal portion and the second audio signal portion,

wherein the preprocessor comprises;

a prediction analyzer for determining prediction coefficients; and

wherein the second encoding processor comprises;

a prediction coefficient quantizer for generating a quantized version of the prediction coefficients; and

an entropy coder for generating an encoded version of the quantized prediction coefficients,

wherein the encoded signal former is configured for introducing the encoded version into the encoded audio signal.

3. The audio encoder of claim 1,

wherein a preprocessor comprises a resampler for resampling the audio signal to a sampling rate of the second encoding processor;

and

wherein a prediction analyzer is configured to determine the prediction coefficients using a resampled audio signal, or

wherein the preprocessor further comprises a long term prediction analysis stage for determining one or more long term prediction parameters for the first audio signal portion.

4. The audio encoder of claim 1, further comprising a cross-processor for calculating, from the encoded spectral representation of the first audio signal portion, initialization data of the second encoding processor, so that the second encoding processing is initialized to encode the second audio signal portion immediately following the first audio signal portion in time in the audio signal.

5. The audio encoder of claim 4, wherein the cross-processor comprises:

a spectral decoder for calculating a decoded version of the first encoded signal portion;

a delay stage for feeding a delayed version of the decoded version into a de-emphasis stage of the second encoding processor for initialization;

a weighted prediction coefficient analysis filtering block for feeding a filter output into a codebook determinator of the second encoding processor for initialization;

an analysis filtering stage for filtering the decoded version or a pre-emphasized version and for feeding a filter residual into an adaptive codebook determinator of the second encoding processor for initialization;

or

a pre-emphasis filter for filtering the decoded version and for feeding a delayed or pre-emphasized version to a synthesis filtering stage of the second encoding processor for initialization.

6. The audio encoder of claim 1,

wherein the analyzer is configured to perform a temporal tile shaping or temporal noise shaping analysis or an operation of setting to zero spectral values in the second spectral portions,

wherein the first encoding processor is configured to perform a shaping of spectral values of the first spectral portions using prediction coefficients derived from the first audio signal portion, and wherein the first encoding processor is furthermore configured to perform a quantization and entropy coding operation of shaped spectral values of the first spectral portions, and

wherein spectral values of the second spectral portions are set to zero.

7. The audio encoder of claim 6, further comprising a cross-processor, wherein the cross-processor comprises:

a noise shaper for shaping quantized spectral values of the first spectral portions using LPC coefficients derived from the first audio signal portion;

a spectral decoder for decoding the spectrally shaped spectral portions of the first spectral portion with a high spectral resolution and for synthesizing second spectral portions using a parametric representation of the second spectral portions and at least a decoded first spectral portion to acquire a decoded spectral representation;

a frequency-time converter for converting the spectral representation into a time domain to acquire a decoded first audio signal portion, wherein a sampling rate associated with the decoded first audio signal portion is different than a sampling rate of the audio signal, and a sampling rate associated with an output signal of the frequency-time converter is different from a sampling rate of the audio signal input into the frequency-time converter.

8. The audio encoder of claim 1,

wherein the second encoding processor comprises at least one block of the following group of blocks:

a prediction analysis filter;

an adaptive codebook stage;

an innovative codebook stage;

an estimator for estimating an innovative codebook entry;

an ACELP/gain coding stage;

a prediction synthesis filtering stage;

a de-emphasis stage; and

a bass post-filter analysis stage.

9. The audio encoder of claim 1,

wherein the time domain encoding processor comprises an associated second sampling rate,

wherein the frequency domain encoding processor has associated therewith a first sampling rate being higher than the second sampling rate, wherein the audio encoder further comprises a cross-processor for calculating, from the encoded spectral representation of the first audio signal portion, initialization data of the second encoding processor,

wherein the cross-processor comprises a frequency-time converter for generating a time domain signal at the second sampling rate,

wherein the frequency time converter comprises:

a selector for selecting a low portion of a spectrum input into the frequency time converter in accordance with a ratio of the first sampling rate and the second sampling rate, the ratio being smaller than 1,

a transform processor comprising a transform length being smaller than a transform length of the time-frequency converter; and

a synthesis windower for windowing using a window comprising a smaller number of window coefficients compared to a window used by the time frequency converter.

10. An audio decoder for decoding an encoded audio signal, comprising:

a first decoding processor for decoding a first encoded audio signal portion in a frequency domain, the first decoding processor comprising;

a spectral decoder for decoding first spectral portions with a high spectral resolution and for synthesizing second spectral portions using a parametric representation of the second spectral portions and at least a decoded first spectral portion to acquire a decoded spectral representation, wherein the spectral decoder is configured to generate the first decoded representation so that a first spectral portion is placed with respect to frequency between two second spectral portions; and

a frequency-time converter for converting the decoded spectral representation into a time domain to acquire a decoded first audio signal portion;

a second decoding processor for decoding a second encoded audio signal portion in the time domain to acquire a decoded second audio signal portion, wherein the second decoding processor comprises;

a time domain low band decoder for decoding a low band time domain signal;

an upsampler for upsampling the low band time domain signal;

a time domain bandwidth extension decoder for synthesizing a high band of a time domain output signal; and

a mixer for mixing a synthesized high band of the time domain signal and an upsampled low band time domain signal; and

a combiner for combining the decoded first spectral portion and the decoded second spectral portion to acquire a decoded audio signal.

11. The audio decoder of claim 10,

wherein the upsampler comprises an analysis filterbank operating at a first time domain low band decoder sampling rate and a synthesis filterbank operating at a second output sampling rate being higher than the first time domain low band sampling rate.

12. The audio decoder of claim 10,

wherein the time domain low band decoder comprises a residual signal, a decoder and a synthesis filter for filtering a residual signal using synthesis filter coefficients,

wherein the time domain bandwidth extension decoder is configured to upsample the residual signal and to process an upsampled residual signal using a non-linear operation to acquire a high band residual signal, and to spectrally shape the high band residual signal to acquire the synthesized high band.

13. The audio decoder of claim 10,

wherein the first decoding processor comprises an adaptive long term prediction post-filter for post-filtering the first decoded first signal portion, wherein the filter is controlled by one or more long term prediction parameters comprised in the encoded audio signal.

14. The audio decoder of claim 10, further comprising:

a cross-processor for calculating, from the decoded spectral representation of the first encoded audio signal portion, initialization data of the second decoding processor, so that the second decoding processor is initialized to decode the encoded second audio signal portion following in time the first audio signal portion in the encoded audio signal.

15. The audio decoder of claim 14, wherein the cross-processor further comprises:

a frequency-time converter operating at a lower sampling rate than the frequency-time converter of the first decoding processor to acquire a further decoded first signal portion in the time domain,

wherein the signal output by the frequency-time converter comprises a second sampling rate being lower than the first sampling rate associated with an output of the frequency-time converter of the second decoding processor,

wherein the additional frequency-time converter comprises a selector for selecting a low portion of a spectrum input into the additionally frequency-time converter in accordance with a ratio of the first sampling rate and the second sampling rate, the ratio being smaller than 1;

a transform processor comprising a transform length being smaller than a transform length of the time-frequency converter; and

a synthesis windower using a window comprising a smaller number of coefficients compared to a window used by the frequency-time converter.

16. The audio decoder of claim 14,

wherein the cross-processor comprises:

a delay stage for delaying the further decoded first signal portion and for feeding a delayed version of the decoded first signal portion into a de-emphasis stage of the second decoding processor for initialization;

a pre-emphasis filter and a delay stage for filtering and delaying the further decoded first signal portion and for feeding a delay stage output into a prediction synthesis filter of the second decoding processor for initialization;

a prediction analysis filter for generating a prediction residual signal from the further decoded first spectral portion or a pre-emphasized further decoded first signal portion and for feeding a prediction residual signal into a codebook synthesizer of the second decoding processor;

or

a switch for feeding the further decoded first signal portion into an analysis stage of a resampler of the second decoding processor for initialization.

17. The audio decoder of claim 10,

wherein the second decoding processor comprises at least one block of the group of blocks comprising:

an ACELP for decoding gains and an innovative codebook;

an adaptive codebook synthesis stage;

an ACELP post-processor;

a prediction synthesis filter; and

a de-emphasis stage.

18. A method of encoding an audio signal, comprising:

first encoding a first audio signal portion in a frequency domain, wherein the first encoding comprises;

converting the first audio signal portion into a frequency domain representation comprising spectral lines up to a maximum frequency of the first audio signal portion;

analyzing the frequency domain representation up to the maximum frequency to determine first spectral portions to be encoded with a first spectral resolution and second spectral portions to be encoded with a second spectral resolution, the second spectral resolution being lower than the first spectral resolution, wherein the analyzing determines a first spectral portion from the first spectral portions, the first spectral portion being placed, with respect to frequency, between two second spectral portions from the second spectral portions;

encoding the first spectral portions with the first spectral resolution and for encoding the second spectral portions with the second spectral resolution, wherein the encoding the second spectral portion comprises calculating, from the second spectral portions, spectral envelope information comprising the second spectral resolution;

second encoding a second different audio signal portion in the time domain wherein the second encoding comprises;

converting the second audio signal portion to a lower sampling rate representation, the lower sampling rate being lower than a sampling rate of the audio signal, wherein the lower sampling rate representation does not comprise the high band of the input signal;

time domain encoding the lower sampling rate representation; and

parametrically encoding the high band;

analyzing the audio signal and determining, which portion of the audio signal is the first audio signal portion encoded in the frequency domain and which portion of the audio signal is the second audio signal portion encoded in the time domain; and

forming an encoded audio signal comprising a first encoded signal portion for the first audio signal portion and a second encoded signal portion for the second audio signal portion.

19. A method of decoding an encoded audio signal, comprising:

first decoding a first encoded audio signal portion in a frequency domain, the first decoding comprising;

decoding first spectral portions with a high spectral resolution and synthesizing second spectral portions using a parametric representation of the second spectral portions and at least a decoded first spectral portion to acquire a decoded spectral representation, wherein decoding comprises generating the first decoded representation so that a first spectral portion is placed with respect to frequency between two second spectral portions; and

converting the decoded spectral representation into a time domain to acquire a decoded first audio signal portion;

second decoding a second encoded audio signal portion in the time domain to acquire a decoded second audio signal portion, wherein the second decoding comprises;

decoding a low band time domain signal;

upsampling the low band time domain signal;

synthesizing a high band of a time domain output signal; and

mixing a synthesized high band of the time domain signal and an upsampled low band time domain signal; and

combining the decoded first spectral portion and the decoded second spectral portion to acquire a decoded audio signal.

20. A non-transitory digital storage medium having stored thereon a computer program for performing a method of encoding an audio signal, comprising:

first encoding a first audio signal portion in a frequency domain, wherein the first encoding comprises;

converting the first audio signal portion into a frequency domain representation comprising spectral lines up to a maximum frequency of the first audio signal portion;

analyzing the frequency domain representation up to the maximum frequency to determine first spectral portions to be encoded with a first spectral resolution and second spectral portions to be encoded with a second spectral resolution, the second spectral resolution being lower than the first spectral resolution, wherein the analyzing determines a first spectral portion from the first spectral portions, the first spectral portion being placed, with respect to frequency, between two second spectral portions from the second spectral portions;

encoding the first spectral portions with the first spectral resolution and for encoding the second spectral portions with the second spectral resolution, wherein the encoding the second spectral portion comprises calculating, from the second spectral portions, spectral envelope information comprising the second spectral resolution;

second encoding a second different audio signal portion in the time domain wherein the second encoding comprises;

converting the second audio signal portion to a lower sampling rate representation, the lower sampling rate being lower than a sampling rate of the audio signal, wherein the lower sampling rate representation does not comprise the high band of the input signal;

time domain encoding the lower sampling rate representation; and

parametrically encoding the high band;

analyzing the audio signal and determining, which portion of the audio signal is the first audio signal portion encoded in the frequency domain and which portion of the audio signal is the second audio signal portion encoded in the time domain; and

forming an encoded audio signal comprising a first encoded signal portion for the first audio signal portion and a second encoded signal portion for the second audio signal portion,

when said computer program is run by a computer.

21. A non-transitory digital storage medium having stored thereon a computer program for performing a method of decoding an encoded audio signal, comprising:

first decoding a first encoded audio signal portion in a frequency domain, the first decoding comprising;

decoding first spectral portions with a high spectral resolution and synthesizing second spectral portions using a parametric representation of the second spectral portions and at least a decoded first spectral portion to acquire a decoded spectral representation, wherein decoding comprises generating the first decoded representation so that a first spectral portion is placed with respect to frequency between two second spectral portions; and

converting the decoded spectral representation into a time domain to acquire a decoded first audio signal portion;

second decoding a second encoded audio signal portion in the time domain to acquire a decoded second audio signal portion, wherein the second decoding comprises;

decoding a low band time domain signal;

upsampling the low band time domain signal;

synthesizing a high band of a time domain output signal; and

mixing a synthesized high band of the time domain signal and an upsampled low band time domain signal; and

combining the decoded first spectral portion and the decoded second spectral portion to acquire a decoded audio signal,

when said computer program is run by a computer.