APPARATUS AND METHOD FOR ENCODING OR DECODING AN AUDIO SIGNAL WITH INTELLIGENT GAP FILLING IN THE SPECTRAL DOMAIN

• US 20160133265A1
• Filed: 01/20/2016
• Published: 05/12/2016
• Est. Priority Date: 07/22/2013
• Status: Active Grant

• Current Assignee: Fraunhofer Gesellschaft Zur Foerderung Der Angewandten Forsching E.V.
• Original Assignee: Fraunhofer Gesellschaft Zur Foerderung Der Angewandten Forsching E.V.
• Inventors: NAGEL, Frederik, Erlangen, Ralf, THOSHKAHNA, Balaji, DISCH, Sascha, SCHMIDT, Konstantin, BAYER, Stefan, NEUKAM, Christian, EDLER, Bernd, HELMRICH, Christian
• Granted Patent: US 10,573,334 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G10L 19/008 Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing
• CPC Class Codes: G10L 19/02 using spectral analysis, e.g. transform vocoders or subband vocoders
• CPC Class Codes: G10L 19/0204 using subband decomposition
• CPC Class Codes: G10L 19/0208 Subband vocoders
• CPC Class Codes: G10L 19/0212 using orthogonal transformation
• CPC Class Codes: G10L 19/022 Blocking, i.e. grouping of samples in time; Choice of analysis windows; Overlap factoring
• CPC Class Codes: G10L 19/025 Detection of transients or attacks for time/frequency resolution switching
• CPC Class Codes: G10L 19/03 Spectral prediction for preventing pre-echo; Temporary noise shaping [TNS], e.g. in MPEG2 or MPEG4
• CPC Class Codes: G10L 19/032 Quantisation or dequantisation of spectral components
• CPC Class Codes: G10L 19/06 Determination or coding of the spectral characteristics, e.g. of the short-term prediction coefficients
• CPC Class Codes: G10L 21/0388 Details of processing therefor
• CPC Class Codes: G10L 25/06 the extracted parameters being correlation coefficients
• CPC Class Codes: G10L 25/18 the extracted parameters being spectral information of each sub-band
• CPC Class Codes: G10L 25/21 the extracted parameters being power information
• CPC Class Codes: H04S 1/007 in which the audio signals are in digital form data reduction aspects thereof based on psychoacoustics G10L19/02

Abstract

An apparatus for decoding an encoded audio signal, includes a spectral domain audio decoder for generating a first decoded representation of a first set of first spectral portions, the decoded representation having a first spectral resolution; a parametric decoder for generating a second decoded representation of a second set of second spectral portions having a second spectral resolution being lower than the first spectral resolution; a frequency regenerator for regenerating every constructed second spectral portion having the first spectral resolution using a first spectral portion and spectral envelope information for the second spectral portion; and a spectrum time converter for converting the first decoded representation and the reconstructed second spectral portion into a time representation.

Claims

1. Apparatus for decoding an encoded audio signal, comprising:

a spectral domain audio decoder configured for generating a first decoded representation of a first set of first spectral portions, the decoded representation comprising a first spectral resolution;

a parametric decoder configured for generating a second decoded representation of a second set of second spectral portions, the second decoded representation comprising spectral envelope information comprising a second spectral resolution being lower than the first spectral resolution;

a frequency regenerator configured for regenerating a reconstructed second spectral portion comprising the first spectral resolution using a first spectral portion and the spectral envelope information for a second spectral portion from the second set of second spectral portions; and

a spectrum time converter configured for converting the first decoded representation and the reconstructed second spectral portion into a time representation,

wherein the spectral domain audio decoder is configured to generate the first decoded representation so that the first decoded representation comprises a Nyquist frequency defining a sampling rate being equal to a sampling rate of the time representation generated by the spectrum-time converter, or

wherein the spectral domain audio decoder is configured to generate the first decoded representation so that a first spectral portion is placed, with respect to frequency, between two second spectral portions.

2. Apparatus of claim 1, wherein the parametric decoder is configured for generating the second decoded representation comprising matching information on a first spectral portion indicating that the first spectral portion matches with a second spectral portion, and

wherein the frequency regenerator is configured for regenerating the reconstructed second spectral portion using the first spectral portion identified by the matching information.

3. Apparatus of claim 1, wherein the spectral domain audio decoder is configured to output a sequence of decoded frames of spectral values, a decoded frame being the first decoded representation, wherein the frame comprises spectral values for the first set of spectral portions and zero indications for the second set of second spectral portions,

wherein the apparatus for decoding further comprises a combiner for combining spectral values generated by the frequency regenerator for the second set of second spectral portions and spectral values of the first set of first spectral portions in a reconstruction band to acquire a reconstructed spectral frame comprising spectral values for the first set of the first spectral portions and the second set of second spectral portion, and

wherein the spectrum-time converter is configured to convert the reconstructed spectral frame into the time representation.

4. Apparatus of claim 1, wherein the spectrum-time converter is configured to perform an inverse modified discrete cosine transform, and further comprises an overlap-add stage for overlapping and adding subsequent time domain frames, each subsequent time domain frame originating from a spectrum representation comprising the first decoded representation and the reconstructed second spectral portion.

5. Apparatus of claim 1, wherein a maximum frequency represented by a spectral value for the maximum frequency in the first decoded representation is equal to a maximum frequency comprised in the time representation generated by the spectrum-time converter, wherein the spectral value for the maximum frequency in the first representation is zero or different from zero.

6. Apparatus of claim 1,

wherein the encoded audio signal comprises the first encoded representation being a frequency domain encoded version of the first set of first spectral portions and an encoded version of the second set of second spectral portions,

wherein the apparatus further comprises a data stream parser for extracting the first encoded representation and for forwarding the encoded representation to the spectral domain audio decoder and for extracting the second encoded representation and for forwarding the second encoded representation to the parametric decoder.

7. The apparatus in accordance with claim 1, wherein the encoded audio signal further comprises an encoded representation of the third set of third spectral portions to be reconstructed by noise filling, further comprising:

a noise filler for extracting noise filling information from the encoded representation of the third set of third spectral portions and for applying a noise filling operation in the third set of third spectral portions without using a first spectral portion in a different frequency range to generate a reconstructed third spectral portion,

wherein the spectrum-time converter is configured for additionally converting the third spectral portion into the time representation.

8. Apparatus in accordance with claim 1,

wherein the spectral domain audio decoder is configured to generate the first decoded representation comprising the first spectral portions with frequency values being greater than a frequency being equal to a frequency in a middle of a frequency range covered by the time representation output by the spectrum-time converter.

9. Apparatus of claim 1,

wherein the frequency regenerator is configured to generate a reconstruction band comprising a spectral portion of the first set of spectral portions at a frequency in the reconstruction band being different from a center frequency of the reconstruction band, wherein the reconstruction band is a scale factor band, for which an energy value indicating the spectral envelope information is comprised in the second set of second spectral portions comprising the second spectral resolution.

10. Apparatus for encoding an audio signal, comprising:

a time-spectrum converter configured for converting an audio signal comprising a sampling rate into a spectral representation;

a spectral analyzer configured for analyzing the spectral representation for determining a first set of first spectral portions to be encoded with a first spectral resolution and a different second set of second spectral portions to be encoded with a second spectral resolution, the second spectral resolution being smaller than the first spectral resolution, wherein a first spectral portion is placed, with respect to frequency, between two second spectral portions;

a spectral domain audio encoder configured for generating a first encoded representation of the first set of spectral portions comprising the first spectral resolution; and

a parametric coder configured for calculating spectral envelope information for the second set of second spectral portions, the spectral envelope information comprising the second spectral resolution.

11. Apparatus of claim 10, wherein the parameter coder is configured for calculating similarities between source ranges comprising first spectral portions and target ranges comprising second spectral portions and for determining, based on calculated similarities, for a second spectral portion, a first spectral portion matching with the second spectral portion and for providing the matching information on the first spectral portion matching with the second spectral portion into an encoded representation.

12. Apparatus of claim 10,

wherein the spectral analyzer is configured for analyzing the spectral representation up to a maximum analysis frequency being at least one quarter of a sampling frequency of the audio signal.

13. Apparatus of claim 10,

wherein the time-spectrum converter is configured for windowing the audio signal with overlapping windows to acquire a sequence of windowed frames, a windowed frame comprising a first number of samples, and for converting the sequence of frames into a spectral representation to acquire spectral frames, a spectral frame comprising a second number of spectral samples, the second number being smaller than the first number.

14. Apparatus of claim 10,

wherein the spectral domain audio encoder is configured to process a sequence of frames of spectral values for quantization and entropy coding, wherein, in a frame, spectral values of the second set of second portions are set to zero or wherein, in a frame, spectral values of the first set of first spectral portions and the second set of the second spectral portions are present and wherein, during processing or subsequent to processing, spectral values in the second set of spectral values are set to zero.

15. Apparatus of claim 10,

wherein the spectral domain audio encoder is configured to generate a spectral representation comprising a Nyquist frequency defined by the sampling rate of the audio signal.

16. Apparatus of claim 10,

wherein the spectral domain audio encoder is configured to provide the first encoded representation so that, for a frame of a sampled audio signal, the encoded representation comprises the first set of first spectral portions and the second set of second spectral portions, wherein the spectral values in the second set of spectral portions are encoded as zero or noise values.

17. Apparatus of claim 10,

wherein the spectral analyzer is configured to analyze the spectral representation starting with a frequency gap filling start frequency, and ending with a maximum frequency represented by a maximum frequency comprised in the spectral representation, and

wherein a spectral portion extending from a minimum frequency up to the frequency gap filling start frequency belongs to the first set of first spectral portions.

18. Apparatus of claim 10,

wherein the spectral analyzer is configured to apply a tonal mask processing to at least a portion of the spectral representation so that tonal components and non-tonal components are separated from each other, wherein the first set of the first spectral portions comprises tonal components and wherein the second set of the second spectral portions comprises the non-tonal components.

19. Apparatus of claim 10,

wherein the spectral domain audio encoder comprises a psycho-acoustic module for quantizing the first set of first spectral portions under consideration of a masking threshold determined in the psycho-acoustic module.

20. Apparatus of claim 10,

wherein the time-spectrum converter is configured to apply a Modified Discrete Cosine Transform.

21. Apparatus of claim 10,

wherein the spectral analyzer is configured to separate tonal components from non-tonal components in the spectral representation,

wherein the spectral analyzer is configured to further analyze the non-tonal components to be reconstructed by using a spectral portion from the first set of spectral portions, wherein the spectral analyzer is configured to determine noise-like spectral portions in the non-tonal components to be reconstructed by noise filling,

wherein the first set of first spectral portions comprises the tonal components, wherein the second set of second spectral portions comprises the spectral portions in the non-tonal components, and wherein a third set of third spectral portions comprises the noise-like spectral portions to be reconstructed by noise filling,

wherein the parametric coder is configured for introducing an energy information for the noise-like spectral portions into the second encoded representation.

22. Apparatus of claim 10,

wherein the spectral analyzer is configured to analyze the spectral representation in a frequency range starting from a frequency gap filling start frequency and extending to frequencies higher than the frequency gap filling frequency.

wherein the spectral domain audio encoder is configured to encode at least a third spectral portion in the spectral representation comprising frequencies below the frequency gap filling start frequency with a spectral resolution lower than the first spectral resolution by setting spectral values in the third spectral portion to zero and by calculating and encoding the spectral envelope information indicating an energy in the at least third spectral portion.

23. Method of decoding an encoded audio signal, comprising:

generating a first decoded representation of a first set of first spectral portions, the decoded representation comprising a first spectral resolution;

generating a second decoded representation of a second set of second spectral portions, the second decoded representation comprising spectral envelope information comprising a second spectral resolution being lower than the first spectral resolution;

regenerating a reconstructed second spectral portion comprising the first spectral resolution using a first spectral portion and the spectral envelope information for a second spectral portion from the second set of second spectral portions; and

converting the first decoded representation and the reconstructed second spectral portion into a time representation,

wherein the generating the first decoded representation generates the first decoded representation so that the first decoded representation comprises a Nyquist frequency defining a sampling rate being equal to a sampling rate of the time representation generated by the converting, or

wherein the generating the first decoded representation generates the first decoded representation so that a first spectral portion is placed, with respect to frequency, between two second spectral portions.

25. Non-transitory digital storage medium having computer-readable code stored thereon to perform, when running on a computer or processor, the method of claim 23.

24. Method for encoding an audio signal, comprising:

converting an audio signal comprising a sampling rate into a spectral representation;

analyzing the spectral representation for determining a first set of first spectral portions to be encoded with a first spectral resolution and a different second set of second spectral portions to be encoded with a second spectral resolution, the second spectral resolution being smaller than the first spectral resolution, wherein a first spectral portion is placed, with respect to frequency, between two second spectral portions;

generating a first encoded representation of the first set of spectral portions comprising the first spectral resolution; and

calculating spectral envelope information for the second set of second spectral portions, the spectral envelope information comprising the second spectral resolution.

26. Non-transitory digital storage medium having computer-readable code stored thereon to perform, when running on a computer or processor, the method of claim 24.