System and tools for enhanced 3D audio authoring and rendering

• US 9,838,826 B2
• Filed: 12/02/2016
• Issued: 12/05/2017
• Est. Priority Date: 07/01/2011
• Status: Active Grant

• Current Assignee: Dolby Laboratories Licensing Corporation (Dolby Laboratories Incorporated)
• Original Assignee: Dolby Laboratories Licensing Corporation (Dolby Laboratories Incorporated)
• Inventors: Tsingos, Nicolas R., Robinson, Charles Q., Scharpf, Jurgen W.
• Primary Examiner(s): King, Simon
• Application Number:
  US15/367,937
• Publication Number:
  US 20170086007A1
• Time in Patent Office: 368 Days
• CPC Class Codes: H04R 5/02 Spatial or constructional arrangements of loudspeakers
• CPC Class Codes: H04S 2400/01 Multi-channel, i.e. more than two input channels, sound reproduction with two speakers wherein the multi-channel information is substantially preserved
• CPC Class Codes: H04S 2400/11 Positioning of individual sound objects, e.g. moving airplane, within a sound field H04S2420/13 takes precedence
• CPC Class Codes: H04S 3/00 Systems employing more than two channels, e.g. quadraphonic H04S5/00, H04S7/00 take precedence
• CPC Class Codes: H04S 3/008 in which the audio signals are in digital form, i.e. employing more than two discrete digital channels data reduction aspects thereof based on psychoacoustics G10L19/02
• CPC Class Codes: H04S 5/00 Pseudo-stereo systems, e.g. in which additional channel signals are derived from monophonic signals by means of phase shifting, time delay or reverberation
• CPC Class Codes: H04S 7/307 Frequency adjustment, e.g. tone control H04S7/301 takes precedence
• CPC Class Codes: H04S 7/308 Electronic adaptation dependent on speaker or headphone connection
• CPC Class Codes: H04S 7/40 Visual indication of stereophonic sound image

Abstract

Improved tools for authoring and rendering audio reproduction data are provided. Some such authoring tools allow audio reproduction data to be generalized for a wide variety of reproduction environments. Audio reproduction data may be authored by creating metadata for audio objects. The metadata may be created with reference to speaker zones. During the rendering process, the audio reproduction data may be reproduced according to the reproduction speaker layout of a particular reproduction environment.

Claims

1. A method, comprising:

receiving audio reproduction data comprising one or more audio objects and metadata associated with each of the one or more audio objects;

receiving reproduction environment data comprising an indication of a number of reproduction speakers in the reproduction environment and an indication of the location of each reproduction speaker within the reproduction environment; and

rendering the audio objects into one or more speaker feed signals by applying an amplitude panning process to each audio object, wherein the amplitude panning process is based, at least in part, on the metadata associated with each audio object and the location of each reproduction speaker within the reproduction environment, and wherein each speaker feed signal corresponds to at least one of the reproduction speakers within the reproduction environment;

wherein the metadata associated with each audio object includes audio object coordinates indicating the intended reproduction position of the audio object within the reproduction environment and zone constraint metadata indicating whether rendering the audio object involves imposing speaker zone constraints.

2. The method of claim 1, wherein imposing speaker zone constraints comprises disabling one or more reproduction speakers within speaker zones indicated by the zone constraint metadata.

3. The method of claim 2, wherein the speaker zones indicated by the zone constraint metadata correspond to one or more of a front area, a left area, a right area, a left rear area, a right rear area, an upper area, and a back area.

4. The method of claim 3, wherein the front area corresponds to an area of a cinema reproduction environment in which a screen is located or to an area of a home in which a television screen is located.

5. The method of claim 2, wherein disabling one or more reproduction speakers within speaker zones indicated by the zone constraint metadata comprises applying panning equations to compute gains by regarding the one or more reproduction speakers within speaker zones indicated by the zone constraint metadata as being off.

6. An apparatus, comprising:

an interface system; and

a logic system configured for;

receiving, via the interface system, audio reproduction data comprising one or more audio objects and metadata associated with each of the one or more audio objects;

receiving, via the interface system, reproduction environment data comprising an indication of a number of reproduction speakers in the reproduction environment and an indication of the location of each reproduction speaker within the reproduction environment; and

rendering the audio objects into one or more speaker feed signals by applying an amplitude panning process to each audio object, wherein the amplitude panning process is based, at least in part, on the metadata associated with each audio object and the location of each reproduction speaker within the reproduction environment, and wherein each speaker feed signal corresponds to at least one of the reproduction speakers within the reproduction environment;

wherein the metadata associated with each audio object includes audio object coordinates indicating the intended reproduction position of the audio object within the reproduction environment and zone constraint metadata indicating whether rendering the audio object involves imposing speaker zone constraints.

7. The apparatus of claim 6, wherein imposing speaker zone constraints comprises disabling one or more reproduction speakers within speaker zones indicated by the zone constraint metadata.

8. The apparatus of claim 7, wherein the speaker zones indicated by the zone constraint metadata correspond to one or more of a front area, a left area, a right area, a left rear area, a right rear area, an upper area, and a back area.

9. The apparatus of claim 8, wherein the front area corresponds to an area of a cinema reproduction environment in which a screen is located or to an area of a home in which a television screen is located.

10. The apparatus of claim 7, wherein disabling one or more reproduction speakers within speaker zones indicated by the zone constraint metadata comprises applying panning equations to compute gains by regarding the one or more reproduction speakers within speaker zones indicated by the zone constraint metadata as being off.

11. A non-transitory medium comprising a sequence of instructions, wherein the instructions, when executed by an audio signal processing device, cause the audio signal processing device to perform a method comprising:

receiving audio reproduction data comprising one or more audio objects and metadata associated with each of the one or more audio objects;

receiving reproduction environment data comprising an indication of a number of reproduction speakers in the reproduction environment and an indication of the location of each reproduction speaker within the reproduction environment; and

rendering the audio objects into one or more speaker feed signals by applying an amplitude panning process to each audio object, wherein the amplitude panning process is based, at least in part, on the metadata associated with each audio object and the location of each reproduction speaker within the reproduction environment, and wherein each speaker feed signal corresponds to at least one of the reproduction speakers within the reproduction environment;

wherein the metadata associated with each audio object includes audio object coordinates indicating the intended reproduction position of the audio object within the reproduction environment and zone constraint metadata indicating whether rendering the audio object involves imposing speaker zone constraints.

12. The medium of claim 11, wherein imposing speaker zone constraints comprises disabling one or more reproduction speakers within speaker zones indicated by the zone constraint metadata.

13. The medium of claim 12, wherein the speaker zones indicated by the zone constraint metadata correspond to one or more of a front area, a left area, a right area, a left rear area, a right rear area, an upper area, and a back area.

14. The medium of claim 13, wherein the front area corresponds to an area of a cinema reproduction environment in which a screen is located or to an area of a home in which a television screen is located.

15. The medium of claim 12, wherein disabling one or more reproduction speakers within speaker zones indicated by the zone constraint metadata comprises applying panning equations to compute gains by regarding the one or more reproduction speakers within speaker zones indicated by the zone constraint metadata as being off.