Apparatus and method for encoding or decoding an audio signal with intelligent gap filling in the spectral domain

• US 10,573,334 B2
• Filed: 01/20/2016
• Issued: 02/25/2020
• Est. Priority Date: 07/22/2013
• Status: Active Grant

• Current Assignee: Fraunhofer Gesellschaft Zur Foerderung Der Angewandten Forsching E.V.
• Original Assignee: Fraunhofer Gesellschaft Zur Foerderung Der Angewandten Forsching E.V.
• Inventors: Disch, Sascha, Nagel, Frederik, Geiger, Ralf, Thoshkahna, Balaji Nagendran, Schmidt, Konstantin, Bayer, Stefan, Neukam, Christian, Edler, Bernd, Helmrich, Christian
• Primary Examiner(s): Serrou, Abdelali
• Application Number:
  US15/002,370
• Publication Number:
  US 20160133265A1
• Time in Patent Office: 1,497 Days
• CPC Class Codes: G10L 19/008 Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing
• CPC Class Codes: G10L 19/02 using spectral analysis, e.g. transform vocoders or subband vocoders
• CPC Class Codes: G10L 19/0204 using subband decomposition
• CPC Class Codes: G10L 19/0208 Subband vocoders
• CPC Class Codes: G10L 19/0212 using orthogonal transformation
• CPC Class Codes: G10L 19/022 Blocking, i.e. grouping of samples in time; Choice of analysis windows; Overlap factoring
• CPC Class Codes: G10L 19/025 Detection of transients or attacks for time/frequency resolution switching
• CPC Class Codes: G10L 19/03 Spectral prediction for preventing pre-echo; Temporary noise shaping [TNS], e.g. in MPEG2 or MPEG4
• CPC Class Codes: G10L 19/032 Quantisation or dequantisation of spectral components
• CPC Class Codes: G10L 19/06 Determination or coding of the spectral characteristics, e.g. of the short-term prediction coefficients
• CPC Class Codes: G10L 21/0388 Details of processing therefor
• CPC Class Codes: G10L 25/06 the extracted parameters being correlation coefficients
• CPC Class Codes: G10L 25/18 the extracted parameters being spectral information of each sub-band
• CPC Class Codes: G10L 25/21 the extracted parameters being power information
• CPC Class Codes: H04S 1/007 in which the audio signals are in digital form data reduction aspects thereof based on psychoacoustics G10L19/02

Abstract

An apparatus for decoding an encoded audio signal, includes a spectral domain audio decoder for generating a first decoded representation of a first set of first spectral portions, the decoded representation having a first spectral resolution; a parametric decoder for generating a second decoded representation of a second set of second spectral portions having a second spectral resolution being lower than the first spectral resolution; a frequency regenerator for regenerating every constructed second spectral portion having the first spectral resolution using a first spectral portion and spectral envelope information for the second spectral portion; and a spectrum time converter for converting the first decoded representation and the reconstructed second spectral portion into a time representation.

Claims

1. Audio decoder for decoding an encoded audio signal, comprising:

a spectral domain audio decoder configured for generating a first decoded representation of a first set of first spectral portions, the first decoded representation comprising a first spectral resolution;

a parametric decoder configured for generating a second decoded representation of a second set of second spectral portions, the second decoded representation comprising spectral envelope information comprising a second spectral resolution being lower than the first spectral resolution;

a frequency regenerator configured for regenerating a reconstructed second spectral portion comprising the first spectral resolution using a first spectral portion of the first set of first spectral portions and the spectral envelope information for a second spectral portion of the second set of second spectral portions; and

a spectrum time converter configured for converting the first decoded representation and the reconstructed second spectral portion into a time representation to obtain a decoded audio signal,

wherein the spectral domain audio decoder is configured to generate the first decoded representation so that the first spectral portion of the first set of first spectral portions is placed, with respect to frequency, between two second spectral portions of the second set of second spectral portions,

wherein one or more of the spectral domain audio decoder, the parametric decoder, the frequency regenerator, and the spectrum time converter is implemented, at least in part, by one or more hardware elements of the audio decoder.

2. The audio decoder of claim 1, wherein the encoded audio signal comprises, as side information, matching information on a first spectral portion indicating that the first spectral portion matches with a second spectral portion, wherein the parametric decoder is configured for generating the second decoded representation comprising the matching information on the first spectral portion of the first set of first spectral portions indicating that the first spectral portion matches with the second spectral portion of the second set of second spectral portions, and

wherein the frequency regenerator is configured for regenerating the reconstructed second spectral portion using the first spectral portion identified by the matching information.

3. The audio decoder of claim 1, wherein the spectral domain audio decoder is configured to output a sequence of decoded frames of spectral values, a decoded frame being the first decoded representation, wherein the decoded frame comprises spectral values for the first set of first spectral portions and zero indications for the second set of second spectral portions,

wherein the apparatus for decoding further comprises a combiner configured for combining spectral values generated by the frequency regenerator for the second set of second spectral portions and spectral values of the first set of first spectral portions in a reconstruction band to acquire a reconstructed spectral frame comprising spectral values for the first set of the first spectral portions and the second set of second spectral portion, and

wherein the spectrum-time converter is configured to convert the reconstructed spectral frame into the time representation.

4. The audio decoder of claim 1, wherein the spectrum-time converter is configured to perform an inverse modified discrete cosine transform, and further comprises an overlap-add stage configured for overlapping and adding subsequent time domain frames, each subsequent time domain frame originating from a spectrum representation comprising the first decoded representation and the reconstructed second spectral portion.

5. The audio decoder of claim 1, wherein a maximum frequency represented by a spectral value for the maximum frequency in the first decoded representation is equal to a maximum frequency comprised in the time representation generated by the spectrum-time converter, wherein the spectral value for the maximum frequency in the first representation is zero or different from zero.

6. The audio decoder of claim 1,

wherein the encoded audio signal comprises a first encoded representation being a frequency domain encoded version of the first set of first spectral portions and second encoded representation of the second set of second spectral portions,

wherein the apparatus further comprises a data stream parser configured for extracting the first encoded representation and configured for forwarding the first encoded representation to the spectral domain audio decoder and configured for extracting the second encoded representation and configured for forwarding the second encoded representation to the parametric decoder.

7. The audio decoder of claim 1, wherein the encoded audio signal further comprises an encoded representation of a third set of third spectral portions to be reconstructed by noise filling, further comprising:

a noise filler configured for extracting noise filling information from the encoded representation of the third set of third spectral portions and configured for applying a noise filling operation in the third set of third spectral portions without using the first spectral portion of the first set of first spectral portions in a different frequency range to generate a reconstructed third spectral portion,

wherein the spectrum-time converter is configured for additionally converting the third set of third spectral portions into the time representation.

8. The audio decoder of claim 1,

wherein the spectral domain audio decoder is configured to generate the first decoded representation comprising the first spectral portions with frequency values being greater than a frequency being equal to a frequency in a middle of a frequency range covered by the time representation output by the spectrum-time converter.

9. The audio decoder of claim 1,

wherein the frequency regenerator is configured to generate a reconstruction band comprising a spectral portion of the first set of first spectral portions at a frequency in the reconstruction band being different from a center frequency of the reconstruction band, wherein the reconstruction band is a scale factor band, for which an energy value indicating the spectral envelope information is comprised in the second set of second spectral portions comprising the second spectral resolution.

10. Method of decoding an encoded audio signal, comprising:

generating a first decoded representation of a first set of first spectral portions, the first decoded representation comprising a first spectral resolution;

generating a second decoded representation of a second set of second spectral portions, the second decoded representation comprising spectral envelope information comprising a second spectral resolution being lower than the first spectral resolution;

regenerating a reconstructed second spectral portion comprising the first spectral resolution using a first spectral portion of the first set of first spectral portions and the spectral envelope information for a second spectral portion of the second set of second spectral portions; and

converting the first decoded representation and the reconstructed second spectral portion into a time representation to obtain a decoded audio signal,

wherein the generating the first decoded representation generates the first decoded representation so that the first spectral portion of the first set of first spectral portions is placed, with respect to frequency, between two second spectral portions of the second set of second spectral portions,

wherein one or more of the generating the first decoded representation, the generating the second decoded representation, the regenerating the reconstructed second spectral portion, and the converting the first decoded representation and the reconstructed second spectral portion is implemented, at least in part, by one or more hardware elements of an audio signal processing device.

11. Non-transitory digital storage medium having computer-readable code stored thereon to perform, when running on a computer or processor, a method of decoding an encoded audio signal, the method comprising:

generating a first decoded representation of a first set of first spectral portions, the first decoded representation comprising a first spectral resolution;

generating a second decoded representation of a second set of second spectral portions, the second decoded representation comprising spectral envelope information comprising a second spectral resolution being lower than the first spectral resolution;

regenerating a reconstructed second spectral portion comprising the first spectral resolution using a first spectral portion of the first set of first spectral portions and the spectral envelope information for a second spectral portion of the second set of second spectral portions; and

converting the first decoded representation and the reconstructed second spectral portion into a time representation to obtain a decoded audio signal,

wherein the generating the first decoded representation generates the first decoded representation so that a first spectral portion of the first set of first spectral portions is placed, with respect to frequency, between two second spectral portions of the second set of second spectral portions.