Apparatus and method for generating, storing, or editing an audio representation of an audio scene

• US 20050105442A1
• Filed: 08/04/2004
• Published: 05/19/2005
• Est. Priority Date: 08/04/2003
• Status: Active Grant

• Current Assignee: Fraunhofer Gesellschaft Zur Foerderung Der Angewandten Forsching E.V.
• Original Assignee: Fraunhofer Gesellschaft Zur Foerderung Der Angewandten Forsching E.V.
• Inventors: Langhammer, Jan, Brix, Sandra, Melchior, Frank, Reichelt, Katrin, Roeder, Thomas
• Granted Patent: US 7,680,288 B2
• Time in Patent Office: Days
• US Class Current: 369/83
• CPC Class Codes: G10L 19/008 Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing
• CPC Class Codes: G10L 19/20 using sound class specific coding, hybrid encoders or object based coding
• CPC Class Codes: H04R 2430/20 Processing of the output signals of the acoustic transducers of an array for obtaining a desired directivity characteristic H04R2203/12 takes precedence
• CPC Class Codes: H04R 3/12 for distributing signals to two or more loudspeakers specially adapted for hearing aids H04R25/407
• CPC Class Codes: H04S 2420/13 Application of wave-field synthesis in stereophonic audio systems
• CPC Class Codes: H04S 3/008 in which the audio signals are in digital form, i.e. employing more than two discrete digital channels data reduction aspects thereof based on psychoacoustics G10L19/02

Abstract

An apparatus for generating, storing or editing an audio representation of an audio scene includes audio processing means for generating a plurality of speaker signals from a plurality of input channels as well as means for providing an object-oriented description of the audio scene, wherein the object-oriented description of the audio scene includes a plurality of audio objects, wherein an audio object is associated with an audio signal, a starting time instant and an end time instant. The apparatus for generating further distinguishes itself by mapping means for mapping the object-oriented description of the audio scene to the plurality of input channels, wherein an assignment of temporally overlapping audio objects to parallel input channels is performed by the mapping means, whereas temporally sequential audio objects are associated with the same channel. With this, an object-oriented representation is transferred into a channel-oriented representation, whereby on the object-oriented side the optimal representation of a scene may be used, whereas on channel-oriented side the channel-oriented concept users are used to may be maintained.

Claims

1. An apparatus for generating, storing, or editing an audio representation of an audio scene, comprising:

an audio processor for generating a plurality of speaker signals from a plurality of input channels;

a provider for providing an object-oriented description of the audio scene, wherein the object-oriented description of the audio scene includes a plurality of audio objects, wherein an audio object is associated with an audio signal, a starting time instant, and an end time instant; and

a mapper for mapping the object-oriented description of the audio scene to the plurality of input channels of the audio processor, wherein the mapper is configured to assign a first audio object to an input channel, and to assign a second audio object whose starting time instant lies after the end time instant of the first audio object to the same input channel, and to assign a third audio object whose starting time instant lies after the starting time instant of the first audio object and before the end time instant of the first audio object to another of the plurality of input channels.

2. The apparatus of claim 1, wherein the audio processor includes a wave-field synthesizer configured to calculate the plurality of speaker signals for the speakers, knowing positions of a plurality of speakers.

3. The apparatus of claim 1, wherein the audio object is further associated with a virtual position, and wherein the audio processor is configured to take the virtual positions of the audio objects into account in generating the plurality of speaker signals.

4. The apparatus of claim 1, wherein the audio processor is coupled to the provider exclusively via the mapper, to receive audio object data to be processed.

5. The apparatus of claim 1, wherein a number of input channels of the audio processor is predetermined and is smaller than an allowed number of audio objects in the audio scene, wherein at least two audio objects are present that do not overlap temporally.

6. The apparatus of claim 1, further comprising a user interface, wherein the user interface comprises a number of separate user interface channels, wherein a user interface channel is associated with an input channel of the audio processor, and wherein the user interface is coupled to the mapper to identify the audio object just assigned to the user interface channel at a time instant.

7. The apparatus of claim 6, wherein the user interface is configured to identify user interface channels associated with input channels of the audio processor to which currently one audio object is assigned.

8. The apparatus of claim 7, wherein the user interface is configured as hardware mixing console having hardware manipulators for each user interface channel, and wherein each hardware manipulator is associated with an indicator to identify a currently active user interface channel.

9. The apparatus of claim 7, wherein the user interface comprises a graphical user interface configured to display, on an electrical display device, only the user interface channels associated with an input channel of the audio processor to which currently one audio object is assigned.

10. The apparatus of claim 6, wherein the user interface further comprises a manipulator for a user interface channel, which is configured to manipulate an audio object assigned to the input channel of the audio processor, which corresponds to the user interface channel, wherein the user interface is coupled to the provider to replace an audio object by a manipulated version thereof, and wherein the mapper is configured to assign, instead of the audio object, the manipulated version thereof to an input channel of the audio processor.

11. The apparatus of claim 10, wherein the manipulator is configured to change position, type, or audio signal of an audio object.

12. The apparatus of claim 6, wherein the user interface is configured to illustrate a temporal occupation for a user interface channel, wherein the temporal occupation represents a temporal sequence of the audio objects assigned to a user interface channel, and wherein the user interface is further configured to mark a current time instant in the temporal occupation.

13. The apparatus of claim 12, wherein the user interface is configured to illustrate the temporal occupation as time axis, which comprises the assigned audio objects proportional to their length as well as an indicator moving with time proceeding.

14. The apparatus of claim 1,

wherein the provider is configured to permit grouping of audio objects such that the audio objects that are grouped are marked by group information with regard to their group membership, and

wherein the mapper is configured to preserve the group information so that manipulation of a group property affects all members of the group, independently of the fact which input channel of the audio processor the audio objects of the group are associated with.

15. A method of generating, storing, or editing an audio representation of an audio scene, comprising:

generating a plurality of speaker signals from a plurality of input channels;

providing an object-oriented description of the audio scene, wherein the object-oriented description of the audio scene includes a plurality of audio objects, wherein an audio object is associated with an audio signal, a starting time instant, and an end time instant; and

mapping the object-oriented description of the audio scene to the plurality of input channels of the audio processor by assigning a first audio object to an input channel, and by assigning a second audio object whose starting time instant lies after the end time instant of the first audio object to the same input channel, and by assigning a third audio object whose starting time instant lies after the starting time instant of the first audio object and before the end time instant of the first audio object to another of the plurality of input channels.

16. A computer program with a program code for performing, when the program is executed on a computer, the method of generating, storing, or editing an audio representation of an audio scene, comprising:

generating a plurality of speaker signals from a plurality of input channels;

providing an object-oriented description of the audio scene, wherein the object-oriented description of the audio scene includes a plurality of audio objects, wherein an audio object is associated with an audio signal, a starting time instant, and an end time instant; and

mapping the object-oriented description of the audio scene to the plurality of input channels of the audio processor by assigning a first audio object to an input channel, and by assigning a second audio object whose starting time instant lies after the end time instant of the first audio object to the same input channel, and by assigning a third audio object whose starting time instant lies after the starting time instant of the first audio object and before the end time instant of the first audio object to another of the plurality of input channels.