Apparatus and method for encoding or decoding audio signal with intelligent gap filling in the spectral domain

• US 10,311,892 B2
• Filed: 12/07/2017
• Issued: 06/04/2019
• Est. Priority Date: 07/22/2013
• Status: Active Grant

• Current Assignee: Fraunhofer Gesellschaft Zur Foerderung Der Angewandten Forsching E.V.
• Original Assignee: Fraunhofer Gesellschaft Zur Foerderung Der Angewandten Forsching E.V.
• Inventors: Disch, Sascha, Nagel, Frederik, Geiger, Ralf, Thoshkahna, Balaji Nagendran, Schmidt, Konstantin, Bayer, Stefan, Neukam, Christian, Edler, Bernd, Helmrich, Christian
• Primary Examiner(s): Serrou, Abdelali
• Application Number:
  US15/834,260
• Publication Number:
  US 20180102134A1
• Time in Patent Office: 544 Days
• CPC Class Codes: G10L 19/008 Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing
• CPC Class Codes: G10L 19/02 using spectral analysis, e.g. transform vocoders or subband vocoders
• CPC Class Codes: G10L 19/0204 using subband decomposition
• CPC Class Codes: G10L 19/0208 Subband vocoders
• CPC Class Codes: G10L 19/0212 using orthogonal transformation
• CPC Class Codes: G10L 19/022 Blocking, i.e. grouping of samples in time; Choice of analysis windows; Overlap factoring
• CPC Class Codes: G10L 19/025 Detection of transients or attacks for time/frequency resolution switching
• CPC Class Codes: G10L 19/03 Spectral prediction for preventing pre-echo; Temporary noise shaping [TNS], e.g. in MPEG2 or MPEG4
• CPC Class Codes: G10L 19/032 Quantisation or dequantisation of spectral components
• CPC Class Codes: G10L 19/06 Determination or coding of the spectral characteristics, e.g. of the short-term prediction coefficients
• CPC Class Codes: G10L 21/0388 Details of processing therefor
• CPC Class Codes: G10L 25/06 the extracted parameters being correlation coefficients
• CPC Class Codes: G10L 25/18 the extracted parameters being spectral information of each sub-band
• CPC Class Codes: G10L 25/21 the extracted parameters being power information
• CPC Class Codes: H04S 1/007 in which the audio signals are in digital form data reduction aspects thereof based on psychoacoustics G10L19/02

Abstract

An apparatus for decoding an encoded audio signal, includes a spectral domain audio decoder for generating a first decoded representation of a first set of first spectral portions, the decoded representation having a first spectral resolution; a parametric decoder for generating a second decoded representation of a second set of second spectral portions having a second spectral resolution being lower than the first spectral resolution; a frequency regenerator for regenerating every constructed second spectral portion having the first spectral resolution using a first spectral portion and spectral envelope information for the second spectral portion; and a spectrum time converter for converting the first decoded representation and the reconstructed second spectral portion into a time representation.

Claims

1. Audio encoder for encoding an audio signal to obtain an encoded audio signal, comprising a processor and a memory including instructions that, when executed by the processor, cause the encoder to:

convert, by a time-spectrum converter an audio signal comprising a sampling rate into a spectral representation;

analyze, by a spectral analyzer, the spectral representation for determining a first set of first spectral portions to be encoded with a first spectral resolution and a different second set of second spectral portions to be encoded with a second spectral resolution, the second spectral resolution being smaller than the first spectral resolution, wherein a first spectral portion of the first set of first spectral portions is placed, with respect to frequency, between two second spectral portions of the second set of second spectral portions;

generate by, a spectral domain audio encoder, a first encoded representation of the first set of first spectral portions comprising the first spectral resolution, wherein the first encoded representation comprises an encoded representation of the first spectral portion of the first set of first spectral portions that is placed, with respect to frequency, between the two second spectral portions of the second set of second spectral portions

calculate, by a parametric coder, spectral envelope information for the second set of second spectral portions, the spectral envelope information comprising the second spectral resolution, wherein the spectral envelope information comprises spectral envelope information of the two second spectral portions of the second set of second spectral portions; and

transmitting, by an output interface, the encoded audio signal to a decoder, wherein the encoded audio signal comprises the first encoded representation, and the spectral envelope information, and wherein one or more of the time-spectrum converter, the spectral analyzer, the spectral domain audio encoder, and the parametric coder is implemented, at least in part, by one or more hardware elements of the audio encoder.

2. Audio encoder of claim 1, wherein the parametric coder is configured for calculating similarities between source ranges comprising first spectral portions of the first set of first spectral portions and target ranges comprising second spectral portions of the second set of second spectral portions and for determining, based on calculated similarities, for a second spectral portion of the second set of second spectral portions, a first spectral portion of the first set of first spectral portions matching with the second spectral portion of the second set of second spectral portions and for providing the matching information on the first spectral portion of the first set of first spectral portions matching with the second spectral portion of the second set of second spectral portions into an encoded representation.

3. Audio encoder of claim 2,

wherein the spectral analyzer is configured for analyzing the spectral representation up to a maximum analysis frequency being at least one quarter of a sampling frequency of the audio signal.

4. Audio encoder of claim 1,

wherein the time-spectrum converter is configured for windowing the audio signal with overlapping windows to acquire a sequence of windowed frames, a windowed frame comprising a first number of samples, and for converting the sequence of frames into the spectral representation to acquire spectral frames, a spectral frame comprising a second number of spectral samples, the second number being smaller than the first number.

5. Audio encoder of claim 1,

wherein the spectral domain audio encoder is configured to process a sequence of frames of spectral values for quantization and entropy coding, wherein, in a frame of the sequence of frames, spectral values of the second set of second portions are set to zero or wherein, in a frame, spectral values of the first set of first spectral portions and the second set of the second spectral portions are present and wherein, during processing or subsequent to processing, spectral values in the second set of second spectral portions are set to zero.

6. Audio encoder of claim 1,

wherein the spectral domain audio encoder is configured to generate the first encoded representation of the first set of first spectral portions comprising a Nyquist frequency defined by the sampling rate of the audio signal.

7. Audio encoder of claim 1,

wherein the spectral domain audio encoder is configured to provide the first encoded representation so that, for a frame of a sampled audio signal, the first encoded representation comprises the first set of first spectral portions and the second set of second spectral portions, wherein the spectral values in the second set of second spectral portions are encoded as zero values or as noise values.

8. Audio encoder of claim 1,

wherein the spectral analyzer is configured to analyze the spectral representation starting, with respect to frequency, with a frequency gap filling start frequency, and ending, with respect to frequency, with a maximum frequency represented by a maximum frequency comprised in the spectral representation, and

wherein a spectral portion extending from a minimum frequency up to the frequency gap filling start frequency belongs to the first set of first spectral portions to be encoded by the spectral domain audio encoder.

9. Audio encoder of claim 1,

wherein the spectral analyzer is configured to apply a tonal mask processing to at least a portion of the spectral representation so that tonal components and non-tonal components are separated from each other, wherein the first set of the first spectral portions comprises the tonal components and wherein the second set of the second spectral portions comprises the non-tonal components.

10. Audio encoder of claim 1,

wherein the spectral domain audio encoder comprises a psycho-acoustic module for quantizing the first set of first spectral portions under consideration of a masking threshold determined in the psycho-acoustic module.

11. Audio encoder of claim 1,

wherein the time-spectrum converter is configured to apply a Modified Discrete Cosine Transform.

12. Audio encoder of claim 1,

wherein the spectral analyzer is configured to separate spectral portions having tonal components from spectral portions having non-tonal components in the spectral representation,

wherein the spectral analyzer is configured to further analyze the spectral portions having the non-tonal components to be reconstructed by using a spectral portion from the first set of spectral portions, wherein the spectral analyzer is configured to determine noise-like spectral portions in the non-tonal components to be reconstructed by noise filling,

wherein the first set of first spectral portions comprises the tonal components, wherein the second set of second spectral portions comprises the spectral portions having the non-tonal components, and wherein a third set of third spectral portions comprises the noise-like spectral portions to be reconstructed by noise filling,

wherein the parametric coder is configured for introducing an energy information for the noise-like spectral portions into the second encoded representation.

13. Audio encoder of claim 1,

wherein the spectral analyzer is configured to analyze the spectral representation in a frequency range starting from a frequency gap filling start frequency and extending to frequencies higher than the frequency gap filling start frequency,

wherein the spectral domain audio encoder is configured to encode at least a third spectral portion in the spectral representation comprising frequencies below the frequency gap filling start frequency with a spectral resolution lower than the first spectral resolution by setting spectral values in at least the third spectral portion to zero and by calculating and encoding the spectral envelope information indicating an energy in at least the third spectral portion.

14. Method for encoding an audio signal to obtain an encoded audio signal, comprising:

converting an audio signal comprising a sampling rate into a spectral representation;

analyzing the spectral representation for determining a first set of first spectral portions to be encoded with a first spectral resolution and a different second set of second spectral portions to be encoded with a second spectral resolution, the second spectral resolution being smaller than the first spectral resolution, wherein a first spectral portion of the first set of first spectral portions is placed, with respect to frequency, between two second spectral portions of the second set of second spectral portions;

generating a first encoded representation of the first set of first spectral portions comprising the first spectral resolution, wherein the first encoded representation comprises an encoded representation of the first spectral portion of the first set of first spectral portions that is placed, with respect to frequency, between the two second spectral portions of the second set of second spectral portions;

calculating spectral envelope information for the second set of second spectral portions, the spectral envelope information comprising the second spectral resolution, wherein the spectral envelope information comprises spectral envelope information of the two second spectral portions of the second set of second spectral portions; and

transmitting, by an output interface, the encoded audio signal to a decoder, wherein the encoded audio signal comprises the first encoded representation, and the spectral envelope information, and wherein one or more of the converting, the analyzing, the generating, and the calculating is implemented, at least in part, by one or more hardware elements of an audio signal processing device.

15. Non-transitory digital storage medium having computer-readable code stored thereon to perform, when running on a computer or a processor, a method for encoding an audio signal to obtain an encoded audio signal, the method comprising:

converting an audio signal comprising a sampling rate into a spectral representation;

analyzing the spectral representation for determining a first set of first spectral portions to be encoded with a first spectral resolution and a different second set of second spectral portions to be encoded with a second spectral resolution, the second spectral resolution being smaller than the first spectral resolution, wherein a first spectral portion is placed, with respect to frequency, between two second spectral portions;

generating a first encoded representation of the first set of spectral portions comprising the first spectral resolution, wherein the first encoded representation comprises an encoded representation of the first spectral portion of the first set of first spectral portions that is placed, with respect to frequency, between the two second spectral portions of the second set of second spectral portions; and

calculating spectral envelope information for the second set of second spectral portions, the spectral envelope information comprising the second spectral resolution, wherein the spectral envelope information comprises spectral envelope information of the two second spectral portions of the second set of second spectral portions; and

transmitting, by an output interface, the encoded audio signal to a decoder, wherein the encoded audio signal comprises the first encoded representation, and the spectral envelope information.