Processing object-based audio signals

• US 10,251,010 B2
• Filed: 09/26/2018
• Issued: 04/02/2019
• Est. Priority Date: 06/01/2015
• Status: Active Grant

• Current Assignee: Dolby Laboratories Licensing Corporation (Dolby Laboratories Incorporated)
• Original Assignee: Dolby Laboratories Licensing Corporation (Dolby Laboratories Incorporated)
• Inventors: Seefeldt, Alan J., Lu, Lie, Zhang, Chen
• Primary Examiner(s): Islam, Mohammad K
• Application Number:
  US16/143,351
• Publication Number:
  US 20190037333A1
• Time in Patent Office: 188 Days
• CPC Class Codes: G10L 19/008 Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing
• CPC Class Codes: H04S 2400/11 Positioning of individual sound objects, e.g. moving airplane, within a sound field H04S2420/13 takes precedence
• CPC Class Codes: H04S 3/008 in which the audio signals are in digital form, i.e. employing more than two discrete digital channels data reduction aspects thereof based on psychoacoustics G10L19/02
• CPC Class Codes: H04S 7/30 Control circuits for electronic adaptation of the sound field
• CPC Class Codes: H04S 7/302 Electronic adaptation of stereophonic sound system to listener position or orientation H04S7/301 takes precedence

Abstract

An audio processing system and method which calculates, based on spatial metadata of the audio object, a panning coefficient for each of the audio objects in relation to each of a plurality of predefined channel coverage zones. Converts the audio signal into submixes in relation to the predefined channel coverage zones based on the calculated panning coefficients and the audio objects. Each of the submixes indicating a sum of components of the plurality of the audio objects in relation to one of the predefined channel coverage zones. Generating a submix gain by applying an audio processing to each of the submix and controls an object gain applied to each of the audio objects. The object gain being as a function of the panning coefficients for each of the audio objects and the submix gains in relation to each of the predefined channel coverage zones.

Claims

1. A method of processing an audio signal, the audio signal having a plurality of audio objects, the method comprising:

receiving spatial metadata corresponding to the audio objects;

converting the audio signal into submixes of the audio objects of the audio signal, wherein each submix relates to corresponding audio objects of the plurality of audio objects, wherein each of the submix further corresponds to zone constraints of the corresponding audio objects of the submix;

determining a corresponding submix gain for each of the submixes; and

rendering each of the submixes of the audio objects, wherein the rendering includes rendering each of the corresponding audio of the submix, wherein the rendering of the audio objects is further based on the spatial metadata, and wherein the rendering of each of the audio objects includes applying an object gain that is based at least in part on the submix gain corresponding to the submix of the corresponding audio objects.

2. The method according to claim 1, further comprising determining a weighted average of the plurality of audio objects for each of the submix.

3. The method according to claim 2, further comprising determining a weight corresponding to a submix based on the weighted average, wherein the weight relates to a panning coefficient for each of the corresponding audio objects of the submix.

4. The method according to claim 1, further comprising:

determining whether one or more of the audio objects belongs to a dialog object; and

in response to the audio object being determined to be the dialog object, clustering the audio object to a dialog submix.

5. The method according to claim 1, wherein converting the audio signal into submixes further comprises:

converting the audio signal into a front submix in relation to a front zone based on the panning coefficients for the audio objects;

converting the audio signal into a center submix in relation to a center zone based on the panning coefficients for the audio objects;

converting the audio signal into a surround submix in relation to a surround zone based on the panning coefficients for the audio objects; and

converting the audio signal into a height submix in relation to a height zone based on the panning coefficients for the audio objects.

6. The method according to claim 1, further comprising:

for each of the audio objects, identifying a type of the audio object; and

generating the submix gain by applying an audio processing to each of the submixes based on the identified type of the audio object.

7. A computer program product for rendering an audio signal, the computer program product being tangibly stored on a non-transient computer-readable medium and comprising machine executable instructions which, when executed, cause the machine to perform steps of the method according to claim 1.

8. A system for processing an audio signal, the audio signal having a plurality of audio objects, the system comprising:

a receiver for receiving spatial metadata corresponding to the audio objects;

a converter for converting the audio signal into submixes of the audio objects of the audio signal, wherein each submix relates to corresponding audio objects of the plurality of audio objects, wherein each of the submix further corresponds to zone constraints of the corresponding audio objects of the submix;

a processor for determining a corresponding submix gain for each of the submixes; and

a renderer for rendering each of the submixes of the audio objects, wherein the rendering includes rendering each of the corresponding audio of the submix, wherein the rendering of the audio objects is further based on the spatial metadata, and wherein the rendering of each of the audio objects includes applying an object gain that is based at least in part on the submix gain corresponding to the submix of the corresponding audio objects.

9. The system according to claim 8, wherein the processor is further configured to determine a weighted average of the plurality of audio objects for each of the submix.

10. The system according to claim 9, wherein the processor is further configured to determine a weight corresponding to a submix based on the weighted average, wherein the weight relates to a panning coefficient for each of the corresponding audio objects of the submix.

11. The system according to claim 8, wherein the processor is further configured to:

determine whether one or more of the audio objects belongs to a dialog object, and

in response to the audio object being determined to be the dialog object, cluster the audio object to a dialog submix.

12. The system according to claim 8, wherein the converter is further configured to convert the audio signal into submixes by:

converting the audio signal into a front submix in relation to a front zone based on the panning coefficients for the audio objects;

converting the audio signal into a center submix in relation to a center zone based on the panning coefficients for the audio objects;

converting the audio signal into a surround submix in relation to a surround zone based on the panning coefficients for the audio objects; and

converting the audio signal into a height submix in relation to a height zone based on the panning coefficients for the audio objects.

13. The system according to claim 8, wherein the processor is further configured to:

for each of the audio objects, identify a type of the audio object, and

generate the submix gain by applying an audio processing to each of the submixes based on the identified type of the audio object.