Audio encoder and decoder using a frequency domain processor , a time domain processor, and a cross processing for continuous initialization

• US 10,236,007 B2
• Filed: 01/24/2017
• Issued: 03/19/2019
• Est. Priority Date: 07/28/2014
• Status: Active Grant

• Current Assignee: Fraunhofer Gesellschaft Zur Foerderung Der Angewandten Forsching E.V.
• Original Assignee: Fraunhofer Gesellschaft Zur Foerderung Der Angewandten Forsching E.V.
• Inventors: Disch, Sascha, Dietz, Martin, Multrus, Markus, Fuchs, Guillaume, Ravelli, Emmanuel, Neusinger, Matthias, Schnell, Markus, Schubert, Benjamin, Grill, Bernhard
• Primary Examiner(s): Washburn, Daniel C
• Assistant Examiner(s): Ogunbiyi, Oluwadamilola M
• Application Number:
  US15/414,289
• Publication Number:
  US 20170133023A1
• Time in Patent Office: 784 Days
• Field of Search: 704E19042, 704500
• CPC Class Codes: G10L 19/02 using spectral analysis, e.g. transform vocoders or subband vocoders
• CPC Class Codes: G10L 19/0208 Subband vocoders
• CPC Class Codes: G10L 19/022 Blocking, i.e. grouping of samples in time; Choice of analysis windows; Overlap factoring
• CPC Class Codes: G10L 19/028 Noise substitution, i.e. substituting non-tonal spectral components by noisy source comfort noise for discontinuous speech transmission G10L19/012
• CPC Class Codes: G10L 19/04 using predictive techniques
• CPC Class Codes: G10L 19/083 the excitation function being an excitation gain G10L25/90 takes precedence
• CPC Class Codes: G10L 19/18 Vocoders using multiple modes
• CPC Class Codes: G10L 19/24 Variable rate codecs, e.g. for generating different qualities using a scalable representation such as hierarchical encoding or layered encoding
• CPC Class Codes: G10L 19/26 Pre-filtering or post-filtering
• CPC Class Codes: G10L 2019/0001 Codebooks
• CPC Class Codes: G10L 21/038 using band spreading techniques

Abstract

An audio encoder for encoding an audio signal, includes: a first encoding processor for encoding a first audio signal portion in a frequency domain, wherein the first encoding processor includes: a time frequency converter for converting the first audio signal portion into a frequency domain representation having spectral lines up to a maximum frequency of the first audio signal portion; a spectral encoder for encoding the frequency domain representation; a second encoding processor for encoding a second different audio signal portion in the time domain; a cross-processor for calculating, from the encoded spectral representation of the first audio signal portion, initialization data of the second encoding processor, so that the second encoding processing is initialized to encode the second audio signal portion immediately following the first audio signal portion in time in the audio signal.

Claims

1. An audio encoder for encoding an audio signal, comprising:

a first encoding processor configured for encoding a first audio signal portion in a frequency domain, wherein the first encoding processor comprises;

a time-frequency converter configured for converting the first audio signal portion into a frequency domain representation comprising spectral lines up to a maximum frequency of the first audio signal portion;

a spectral encoder configured for encoding the frequency domain representation;

a second encoding processor configured for encoding a second different audio signal portion in the time domain,

wherein the second encoding processor comprises an associated second sampling rate,

wherein the first encoding processor has associated therewith a first sampling rate being different from the second sampling rate;

a cross-processor configured for calculating, from the encoded spectral representation of the first audio signal portion, initialization data of the second encoding processor, so that the second encoding processing is initialized to encode the second audio signal portion immediately following the first audio signal portion in time in the audio signal;

wherein the cross-processor comprises a frequency-time converter configured for generating a time domain signal at the second sampling rate, wherein the frequency-time converter comprises;

a selector configured for selecting a portion of a spectrum input into the frequency-time converter in accordance with a ratio of the first sampling rate and the second sampling rate,

a transform processor comprising a transform length being different from a transform length of the time-frequency converter; and

a synthesis windower configured for windowing using a window comprising a different number of window coefficients compared to a window used by the time-frequency converter;

a controller configured for analyzing the audio signal and for determining, which portion of the audio signal is the first audio signal portion encoded in the frequency domain and which portion of the audio signal is the second audio signal portion encoded in the time domain; and

an encoded signal former configured for forming an encoded audio signal comprising a first encoded signal portion for the first audio signal portion and a second encoded signal portion for the second audio signal portion;

wherein at least one of the first encoding processor, the time-frequency converter, the spectral encoder, the second encoding processor, the cross-processor, the frequency-time converter, the selector, the transform processor, the synthesis windower, the controller and the encoded signal former are implemented, at least in part, by a hardware element of the audio encoder.

2. The audio encoder of claim 1, wherein the input signal comprises a high band and a low band,

wherein the second encoding processor comprises a sampling rate converter configured for converting the second audio signal portion to a lower sampling rate representation, the lower sampling rate being lower than a sampling rate of the audio signal, wherein the lower sampling rate representation does not comprise the high band of the input signal;

a time domain low band encoder configured for time domain encoding the lower sampling rate representation; and

a time domain bandwidth extension encoder configured for parametrically encoding the high band.

3. The audio encoder of claim 1, further comprising:

a preprocessor configured for preprocessing the first audio signal portion and the second audio signal portion,

wherein the preprocessor comprises a prediction analyzer configured for determining prediction coefficients;

wherein the encoded signal former is configured for introducing an encoded version of the prediction coefficients into the encoded audio signal.

4. The audio encoder of claim 1,

wherein a preprocessor comprises a resampler configured for resampling the audio signal to a sampling rate of the second encoding processor;

and

wherein a prediction analyzer is configured to determine the prediction coefficients using a resampled audio signal, or

wherein the preprocessor further comprises a long term prediction analysis stage configured for determining one or more long term prediction parameters for the first audio signal portion.

5. The audio encoder of claim 1, wherein the cross-processor comprises:

a spectral decoder configured for calculating a decoded version of the first encoded signal portion;

a delay stage configured for feeding a delayed version of the decoded version into a de-emphasis stage of the second encoding processor for initialization;

a weighted prediction coefficient analysis filtering block configured for feeding a filter output into a codebook determinator of the second encoding processor for initialization;

an analysis filtering stage configured for filtering the decoded version or a pre-emphasized version and for feeding a filter residual into an adaptive codebook determinator of the second encoding processor for initialization;

or

a pre-emphasis filter configured for filtering the decoded version and for feeding a delayed or pre-emphasized version to a synthesis filtering stage of the second encoding processor for initialization.

6. The audio encoder of claim 1,

wherein the first encoding processor is configured to perform a shaping of spectral values of the frequency domain representation using prediction coefficients derived from the first audio signal portion, and wherein the first encoding processor is furthermore configured to perform a quantization and entropy coding operation of shaped spectral values of the first spectral regions.

7. The audio encoder of claim 1, wherein the cross-processor comprises:

a noise shaper configured for shaping quantized spectral values of the frequency domain representation using linear prediction coding (LPC) coefficients derived from the first audio signal portion;

a spectral decoder configured for decoding spectrally shaped spectral portions of the frequency domain representation with a high spectral resolution to acquire a decoded spectral representation;

the frequency-time converter configured for converting the spectral representation into a time domain to acquire a decoded first audio signal portion, wherein a sampling rate associated with the decoded first audio signal portion is different from a sampling rate of the audio signal, and a sampling rate associated with an output signal of the frequency-time converter is different from a sampling rate associated with the audio signal input into the frequency-time converter.

8. The audio encoder of claim 1,

wherein the second encoding processor comprises at least one block of the following group of blocks:

a prediction analysis filter;

an adaptive codebook stage;

an innovative codebook stage;

an estimator configured for estimating an innovative codebook entry;

an ACELP/gain coding stage;

a prediction synthesis filtering stage;

a de-emphasis stage; and

a bass post-filter analysis stage.

9. An audio decoder for decoding an encoded audio signal, comprising:

a first decoding processor configured for decoding a first encoded audio signal portion in a frequency domain, the first decoding processor comprising a frequency-time converter configured for converting a decoded spectral representation into a time domain to acquire a decoded first audio signal portion;

a second decoding processor configured for decoding a second encoded audio signal portion in the time domain to acquire a decoded second audio signal portion;

a cross-processor configured for calculating, from the decoded spectral representation of the first encoded audio signal portion, initialization data of the second decoding processor, so that the second decoding processor is initialized to decode the encoded second audio signal portion following in time the first audio signal portion in the encoded audio signal; and

a combiner configured for combining the decoded first spectral portion and the decoded second spectral portion to acquire a decoded audio signal,

wherein the cross-processor further comprises

a further frequency-time converter operating at a first effective sampling rate being different from a second effective sampling rate associated with the frequency-time converter of the first decoding processor to acquire a further decoded first signal portion in the time domain,

wherein the signal output by the further frequency-time converter has the second sampling rate being different from the first sampling rate associated with an output of the frequency-time converter of the first decoding processor,

wherein the further frequency-time converter comprises a selector configured for selecting a portion of a spectrum input into the further frequency-time converter in accordance with a ratio of the first sampling rate and the second sampling rate;

a transform processor comprising a transform length being different from a transform length of the time-frequency converter of the first decoding processor; and

a synthesis windower using a window comprising a different number of coefficients compared to a window used by the frequency-time converter of the first decoding processors;

wherein at least one of the first decoding processor, the fret frequency-time converter, the second decoding processor, the cross-processor, the combiner, the further frequency-time converter, the selector, the transform processor, and the synthesis windower are implemented, at least in part, by a hardware element of the audio decoder.

10. The audio decoder of claim 9, wherein the second decoding processor comprises:

a time domain low band decoder configured for decoding a low band time domain signal;

a resampler configured for resampling the low band time domain signal;

a time domain bandwidth extension decoder configured for synthesizing a high band of a time domain output signal; and

a mixer configured for mixing a synthesized high band of the time domain signal and a resampled low band time domain signal.

11. The audio decoder of claim 9,

wherein the first decoding processor comprises an adaptive long term prediction post-filter configured for post-filtering the first decoded first signal portion, wherein the filter is controlled by one or more long term prediction parameters comprised in the encoded audio signal.

12. The audio decoder of claim 9, wherein the cross-processor comprises:

a delay stage configured for delaying the further decoded first signal portion and configured for feeding a delayed version of the decoded first signal portion into a de-emphasis stage of the second decoding processor for initialization;

a pre-emphasis filter and a delay stage configured for filtering and configured for delaying the further decoded first signal portion and configured for feeding a delay stage output into a prediction synthesis filter of the second decoding processor for initialization;

a prediction analysis filter configured for generating a prediction residual signal from the further decoded first spectral portion or a pre-emphasized further decoded first signal portion and configured for feeding a prediction residual signal into a codebook synthesizer of the second decoding processor;

or

a switch configured for feeding the further decoded first signal portion into an analysis stage of a resampler of the second decoding processor for initialization.

13. The audio decoder of claim 9,

wherein the second decoding processor comprises at least one block of the group of blocks comprising:

a stage configured for decoding ACELP gains and an innovative codebook;

an adaptive codebook synthesis stage;

an ACELP post-processor;

a prediction synthesis filter; and

a de-emphasis stage.

14. A method of encoding an audio signal, comprising:

encoding a first audio signal portion in a frequency domain, comprising;

converting the first audio signal portion into a frequency domain representation comprising spectral lines up to a maximum frequency of the first audio signal portion;

encoding the frequency domain representation;

encoding a second different audio signal portion in the time domain;

wherein the encoding the second audio signal portion comprises an associated second sampling rate,

wherein the encoding the first audio signal portion has associated therewith a first sampling rate being different from the second sampling rate

calculating, from the encoded spectral representation of the first audio signal portion, initialization data for the encoding of the second different audio signal portion, so that the encoding of the second different audio signal portion is initialized to encode the second audio signal portion immediately following the first audio signal portion in time in the audio signal wherein the calculating comprises generating, by a frequency-time converter, a time domain signal at the second sampling rate, wherein the generating comprises;

selecting a portion of a spectrum input into the frequency-time converter in accordance with a ratio of the first sampling rate and the second sampling rate,

processing using a transform processor comprising a transform length being different from a transform length of a time-frequency converter used in the converting the first audio signal portion; and

synthesis windowing using a window comprising a different number of window coefficients compared to a window used by the time frequency converter used in the converting the first audio signal portion;

analyzing the audio signal and determining, which portion of the audio signal is the first audio signal portion encoded in the frequency domain and which portion of the audio signal is the second audio signal portion encoded in the time domain; and

forming an encoded audio signal comprising a first encoded signal portion for the first audio signal portion and a second encoded signal portion for the second audio signal portion.

15. A method of decoding an encoded audio signal, comprising:

decoding, by a first decoding processor, a first encoded audio signal portion in a frequency domain, the decoding comprising;

converting, by a frequency-time converter, a decoded spectral representation into a time domain to acquire a decoded first audio signal portion;

decoding a second encoded audio signal portion in the time domain to acquire a decoded second audio signal portion;

calculating, from the decoded spectral representation of the first encoded audio signal portion, initialization data of the decoding of the second encoded audio signal portion, so that the decoding of the second encoded audio signal portion is initialized to decode the encoded second audio signal portion following in time the first audio signal portion in the encoded audio signal; and

combining the decoded first spectral portion and the decoded second spectral portion to acquire a decoded audio signal,

wherein the calculating further comprises

using a further frequency-time converter operating at a first effective sampling rate being different from a second effective sampling rate associated with the frequency-time converter of the first decoding processor to acquire a further decoded first signal portion in the time domain,

wherein the signal output by the further frequency-time converter has the second sampling rate being different from the first sampling rate associated with an output of the frequency-time converter of the first decoding processor,

wherein the using the further frequency-time converter comprises;

selecting a portion of a spectrum input into the further frequency-time converter in accordance with a ratio of the first sampling rate and the second sampling rate;

using a transform processor comprising a transform length being different from a transform length of the time-frequency converter of the first decoding processor; and

using a synthesis windower using a window comprising a different number of coefficients compared to a window used by the frequency-time converter of the first decoding processor.

16. A non-transitory digital storage medium having a computer program stored thereon to perform the method of encoding an audio signal, comprising:

encoding a first audio signal portion in a frequency domain, comprising;

converting the first audio signal portion into a frequency domain representation comprising spectral lines up to a maximum frequency of the first audio signal portion;

encoding the frequency domain representation;

encoding a second different audio signal portion in the time domain;

wherein the encoding the second audio signal portion comprises an associated second sampling rate,

wherein the encoding the first audio signal portion has associated therewith a first sampling rate being different from the second sampling rate

calculating, from the encoded spectral representation of the first audio signal portion, initialization data for the encoding of the second different audio signal portion, so that the encoding of the second different audio signal portion is initialized to encode the second audio signal portion immediately following the first audio signal portion in time in the audio signal wherein the calculating comprises generating, by a frequency-time converter, a time domain signal at the second sampling rate, wherein the generating comprises;

selecting a portion of a spectrum input into the frequency-time converter in accordance with a ratio of the first sampling rate and the second sampling rate,

processing using a transform processor comprising a transform length being different from a transform length of a time-frequency converter used in the converting the first audio signal portion; and

synthesis windowing using a window comprising a different number of window coefficients compared to a window used by the time frequency converter used in the converting the first audio signal portion;

analyzing the audio signal and determining, which portion of the audio signal is the first audio signal portion encoded in the frequency domain and which portion of the audio signal is the second audio signal portion encoded in the time domain; and

forming an encoded audio signal comprising a first encoded signal portion for the first audio signal portion and a second encoded signal portion for the second audio signal portion,

when said computer program is run by a computer.

17. A non-transitory digital storage medium comprising a computer program stored thereon to perform the method of decoding an encoded audio signal, comprising:

decoding, by a first decoding processor, a first encoded audio signal portion in a frequency domain, the decoding comprising;

converting, by a frequency-time converter, a decoded spectral representation into a time domain to acquire a decoded first audio signal portion;

decoding a second encoded audio signal portion in the time domain to acquire a decoded second audio signal portion;

calculating, from the decoded spectral representation of the first encoded audio signal portion, initialization data of the decoding of the second encoded audio signal portion, so that the decoding of the second encoded audio signal portion is initialized to decode the encoded second audio signal portion following in time the first audio signal portion in the encoded audio signal; and

combining the decoded first spectral portion and the decoded second spectral portion to acquire a decoded audio signal,

wherein the calculating further comprises

using a further frequency-time converter operating at a first effective sampling rate being different from a second effective sampling rate associated with the frequency-time converter of the first decoding processor to acquire a further decoded first signal portion in the time domain,

wherein the signal output by the further frequency-time converter has the second sampling rate being different from the first sampling rate associated with an output of the frequency-time converter of the first decoding processor,

wherein the using the further frequency-time converter comprises;

selecting a portion of a spectrum input into the further frequency-time converter in accordance with a ratio of the first sampling rate and the second sampling rate;

using a transform processor comprising a transform length being different from a transform length of the time-frequency converter of the first decoding processor; and

using a synthesis windower using a window comprising a different number of coefficients compared to a window used by the frequency-time converter of the first decoding processor,

when said computer program is run by a computer.