APPARATUS AND METHOD FOR GENERATING AUDIO OUTPUT SIGNALS USING OBJECT BASED METADATA

• US 20100014692A1
• Filed: 10/09/2008
• Published: 01/21/2010
• Est. Priority Date: 07/17/2008
• Status: Active Grant

• Current Assignee: Fraunhofer Gesellschaft Zur Foerderung Der Angewandten Forsching E.V.
• Original Assignee: Fraunhofer Gesellschaft Zur Foerderung Der Angewandten Forsching E.V.
• Inventors: SCHREINER, Stephan, FIESEL, Wolfgang, SPERSCHNEIDER, Ralph, NEUSINGER, Matthias, HELLMUTH, Oliver
• Granted Patent: US 8,315,396 B2
• Time in Patent Office: Days
• US Class Current: 381/119
• CPC Class Codes: H04S 3/008 in which the audio signals are in digital form, i.e. employing more than two discrete digital channels data reduction aspects thereof based on psychoacoustics G10L19/02

Abstract

An apparatus for generating at least one audio output signal representing a superposition of at least two different audio objects comprises a processor for processing an audio input signal to provide an object representation of the audio input signal, where this object representation can be generated by a parametrically guided approximation of original objects using an object downmix signal. An object manipulator individually manipulates objects using audio object based metadata referring to the individual audio objects to obtain manipulated audio objects. The manipulated audio objects are mixed using an object mixer for finally obtaining an audio output signal having one or several channel signals depending on a specific rendering setup.

Claims

1. Apparatus for generating at least one audio output signal representing a superposition of at least two different audio objects, comprising:

a processor for processing an audio input signal to provide an object representation of the audio input signal, in which the at least two different audio objects are separated from each other, the at least two different audio objects are available as separate audio object signals, and the at least two different audio objects are manipulatable independently from each other;

an object manipulator for manipulating the audio object signal or a mixed audio object signal of at least one audio object based on audio object based metadata referring to the at least one audio object to obtain a manipulated audio object signal or a manipulated mixed audio object signal for the at least one audio object; and

an object mixer for mixing the object representation by combining the manipulated audio object with an unmodified audio object or with a manipulated different audio object manipulated in a different way as the at least one audio object.

2. Apparatus in accordance with claim 1, which is adapted to generate m output signals, m being an integer greater than 1,

wherein the processor is operative to provide an object representation having k audio objects, k being an integer and greater than m,

wherein the object manipulator is adapted to manipulate at least two objects different from each other based on metadata associated with at least one object of the at least two objects, and

wherein the object mixer is operative to combine the manipulated audio signals of the at least two different objects to obtain the m output signals so that each output signal is influenced by the manipulated audio signals of the at least two different objects.

3. Apparatus in accordance with claim 1,

in which the processor is adapted to receive the input signal, the input signal being a downmixed representation of a plurality of original audio objects,

in which the processor is adapted to receive audio object parameters for controlling a reconstruction algorithm for reconstructing an approximated representation of the original audio objects, and

in which the processor is adapted to conduct the reconstruction algorithm using the input signal and the audio object parameters to obtain the object representation comprising audio object signals being an approximation of audio object signals of the original audio objects.

4. Apparatus in accordance with claim 1,

in which the audio input signal is a downmixed representation of a plurality of original audio objects and comprises, as side information, object based metadata having information on one or more audio objects included in the downmix representation, and

in which the object manipulator is adapted to extract the object based metadata from the audio input signal.

5. Apparatus in accordance with claim 3, in which the audio input signal comprises, as side information, the audio object parameters, and in which the processor is adapted to extract the side information from the audio input signal.

6. Apparatus in accordance with claim 1,

in which the object manipulator is operative to manipulate the audio object signal, and

in which the object mixer is operative to apply a downmix rule for each object based on a rendering position for the object and a reproduction setup to obtain an object component signal for each audio output signal, and

wherein the object mixer is adapted to add object component signals from different objects for the same output channel to obtain the audio output signal for the output channel.

7. Apparatus in accordance with claim 1, in which the object manipulator is operative to manipulate each of a plurality of object component signals in the same manner based on metadata for the object to obtain object component signals for the audio object, and

in which the object mixer is adapted to add the object component signals from different objects for the same output channel to obtain the audio output signal for the output channel.

8. Apparatus in accordance with claim 1, further comprising an output signal mixer for mixing the audio output signal obtained based on a manipulation of at least one audio object and a corresponding audio output signal obtained without the manipulation of the at least one audio object.

9. Apparatus in accordance with claim 1, in which the metadata comprises the information on a gain, a compression, a level, a downmix setup or a characteristic specific for a certain object, and

wherein the object manipulator is adaptive to manipulate the object or other objects based on the metadata to implement, in an object specific way, a midnight mode, a high fidelity mode, a clean audio mode, a dialogue normalization, a downmix specific manipulation, a dynamic downmix, a guided upmix, a relocation of speech objects or an attenuation of an ambience object.

10. Apparatus in accordance with claim 1, in which the object parameters comprise, for a plurality of time portions of an object audio signal, parameters for each band of a plurality of frequency bands in the respective time portion, and

wherein the metadata only include non-frequency-selective information for an audio object.

11. Apparatus for generating an encoded audio signal representing a superposition of at least two different audio objects, comprising:

a data stream formatter for formatting a data stream so that the data stream comprises an object downmix signal representing a combination of the at least two different audio objects, and, as side information, metadata referring to at least one of the different audio objects.

12. Apparatus in accordance with claim 11, wherein the data stream formatter is operative to additionally introduce, as side information, parametric data allowing an approximation of the at least two different audio objects, into the data stream.

13. Apparatus in accordance with claim 11, the apparatus further comprising a parameter calculator for calculating parametric data for an approximation of the at least two different audio objects, a downmixer for downmixing the at least two different audio objects to obtain the downmix signal, and an input for metadata individually relating to the at least two different audio objects.

14. Method of generating at least one audio output signal representing a superposition of at least two different audio objects, comprising:

processing an audio input signal to provide an object representation of the audio input signal, in which the at least two different audio objects are separated from each other, the at least two different audio objects are available as separate audio object signals, and the at least two different audio objects are manipulatable independently from each other;

manipulating the audio object signal or a mixed audio object signal of at least one audio object based on audio object based metadata referring to the at least one audio object to obtain a manipulated audio object signal or a manipulated mixed audio object signal for the at least one audio object; and

mixing the object representation by combining the manipulated audio object with an unmodified audio object or with a manipulated different audio object manipulated in a different way as the at least one audio object.

16. A computer readable medium storing a computer program for performing, when being executed on a computer, a method for generating at least one audio output signal in accordance with claim 14.

15. Method of generating an encoded audio signal representing a superposition of at least two different audio objects, comprising:

formatting a data stream so that the data stream comprises an object downmix signal representing a combination of the at least two different audio objects, and, as side information, metadata referring to at least one of the different audio objects.

17. A computer readable medium storing a computer program for performing, when being executed on a computer, a method for generating an encoded audio signal in accordance with claim 15.