Apparatus and method for generating audio output signals using object based metadata

• US 8,824,688 B2
• Filed: 08/15/2012
• Issued: 09/02/2014
• Est. Priority Date: 07/17/2008
• Status: Active Grant

• Current Assignee: Fraunhofer Gesellschaft Zur Foerderung Der Angewandten Forsching E.V.
• Original Assignee: Fraunhofer Gesellschaft Zur Foerderung Der Angewandten Forsching E.V.
• Inventors: Schreiner, Stephan, Fiesel, Wolfgang, Neusinger, Matthias, Hellmuth, Oliver, Sperschneider, Ralph
• Primary Examiner(s): Tornow, Mark
• Assistant Examiner(s): WARD, ERIC A
• Application Number:
  US13/585,875
• Publication Number:
  US 20120308049A1
• Time in Patent Office: 748 Days
• Field of Search: 381/1, 381/2, 381 17- 23, 381/119, 700500-502, 700/94, 369 1- 12, 704500-502
• US Class Current: 381/20
• CPC Class Codes: H04S 3/008 in which the audio signals are in digital form, i.e. employing more than two discrete digital channels data reduction aspects thereof based on psychoacoustics G10L19/02

Abstract

An apparatus for generating at least one audio output signal representing a superposition of at least two different audio objects comprises a processor for processing an audio input signal to provide an object representation of the audio input signal, where this object representation can be generated by a parametrically guided approximation of original objects using an object downmix signal. An object manipulator individually manipulates objects using audio object based metadata referring to the individual audio objects to obtain manipulated audio objects. The manipulated audio objects are mixed using an object mixer for finally obtaining an audio output signal having one or several channel signals depending on a specific rendering setup.

Claims

1. Apparatus for generating at least one audio output signal representing a superposition of at least two different audio objects, the at least two different audio objects comprising a first audio object and a second audio object, the apparatus comprising:

a processor arranged to process an audio input signal, the audio input signal being an object downmix comprising the first audio object and the second audio object, to provide an object representation of the audio input signal, in which the first audio object and the second audio object are separated from each other, in which the first audio object and the second audio object are available as a first audio object signal and a separate second audio object signal, and in which the first audio object signal and the second audio object signal are manipulatable independently from each other;

an object manipulator arranged to manipulate the first audio object signal based on audio object based metadata referring to the first audio object to obtain a manipulated first audio object signal for the first audio object; and

an object mixer arranged to combine the manipulated first audio object signal with the second audio object signal, the second audio object signal not being manipulated by the object manipulator or arranged to combine the manipulated first audio object signal with a manipulated second audio object signal, the manipulated second audio object signal being manipulated by the object manipulator based on audio object based metadata referring to the second audio object in a different way as compared to the manipulated first audio object signal.

2. Apparatus in accordance with claim 1,

in which the audio input signal is a downmixed representation of a plurality of original audio objects comprising the first audio object and the second audio object and comprises, as side information, object based metadata having information on one or more original audio objects of the plurality of original audio objects included in the downmixed representation, and

in which the object manipulator is adapted to extract the object based metadata from the audio input signal.

3. Apparatus in accordance with claim 1, in which the metadata comprises information on a gain, a compression, a level, a downmix setup or a characteristic specific for a certain object, and

wherein the object manipulator is adaptive to manipulate the object or other objects based on the metadata to implement, in an object specific way, a midnight mode, a high fidelity mode, a clean audio mode, a dialogue normalization, a downmix specific manipulation, a dynamic downmix, a guided upmix, a relocation of speech objects or an attenuation of an ambience object.

4. Method of generating at least one audio output signal representing a superposition of at least two different audio objects, the at least two different audio objects comprising a first audio object and a second audio object, the method comprising:

processing an audio input signal, the audio input signal being an object downmix comprising the first audio object and the second audio object, to provide an object representation of the audio input signal, in which the first audio object and the second audio object are separated from each other, in which the first audio object and the second audio object are available as a first audio object signal and a separate second audio object signal, and in which the first audio object signal and the second audio object signal are manipulatable independently from each other;

manipulating the first audio object signal based on audio object based metadata referring to the first audio object to obtain a manipulated first audio object signal for the first audio object; and

combining the manipulated first audio object signal with the second audio object signal, the second audio object signal not being manipulated by the manipulating;

or

combining the manipulated first audio object signal with a manipulated second audio object signal, the manipulated second audio object signal being manipulated by the manipulating based on audio object based metadata referring to the second audio object in a different way compared to the manipulated first audio object signal.

5. A non-transitory computer readable medium storing a computer program for performing, when being executed on a computer, a method for generating at least one audio output signal in accordance with claim 4.

6. Apparatus for generating at least one audio output signal representing a superposition of at least two different audio objects, the at least two different audio objects comprising a first audio object and a second audio object, the apparatus comprising:

a processor arranged to process an audio input signal to provide an object representation of the audio input signal, in which the first audio object and the second audio object are separated from each other, in which the first audio object and the second audio object are available as a first audio object signal and a separate second audio object signal, and in which the first audio object signal and the second audio object signal are manipulatable independently from each other;

a first object downmixer arranged to distribute the first audio object signal into output channels using rendering information to obtain a first plurality of first object component signals;

a second object downmixer arranged to distribute the second audio object signal into the output channels using the rendering information to obtain a second plurality of second object component signals;

an object manipulator arranged to manipulate each of the first object component signals of the first plurality in a same manner based on audio object based metadata referring to the first audio object to obtain manipulated first object component signals for the first audio object, and

an object mixer arranged to add, per output channel, the manipulated first object component signals with the second object component signals not being manipulated by the object manipulator or arranged to add, per output channel, the manipulated first object component signals with manipulated second object component signals, the manipulated second object component signals being manipulated by the object manipulator in a same manner based on audio object based metadata referring to the second audio object, the manipulated second object component signals being manipulated in a different way compared to the manipulated first object component signals.

7. Method of generating at least one audio output signal representing a superposition of at least two different audio objects, the at least two different audio objects comprising a first audio object and a second audio object, the method comprising:

processing an audio input signal to provide an object representation of the audio input signal, in which the first audio object and the second audio object are separated from each other, in which the first audio object and the second audio object are available as a first audio object signal and a separate second audio object signal, and in which the first audio object signal and the second audio object signal are manipulatable independently from each other;

distributing the first audio object signal into output channels using rendering information to obtain a first plurality of first object component signals;

distributing the second audio object signal into the output channels using the rendering information to obtain a second plurality of second object component signals;

manipulating each of the first object component signals of the first plurality in a same manner based on audio object based metadata referring to the first audio object to obtain manipulated first object component signals for the first audio object; and

adding, per output channel, the manipulated first object component signals with the second object component signals not being manipulated by the manipulating, or adding, per output channel, the manipulated first object component signals with manipulated second object component signals, the manipulated second object component signals being manipulated by the manipulating in a same manner based on audio object based metadata referring to the second audio object, the manipulated second object component signals being manipulated in a different way compared to the manipulated first object component signals.

8. A non-transitory computer readable medium storing a computer program for performing, when being executed on a computer, a method for generating at least one audio output signal in accordance with claim 7.