PROCESSING OBJECT-BASED AUDIO SIGNALS

• US 20180152803A1
• Filed: 05/26/2016
• Published: 05/31/2018
• Est. Priority Date: 06/01/2015
• Status: Active Grant

• Current Assignee: Dolby Laboratories Licensing Corporation (Dolby Laboratories Incorporated)
• Original Assignee: Dolby Laboratories Licensing Corporation (Dolby Laboratories Incorporated)
• Inventors: SEEFELDT, Alan J., LU, Lie, ZHANG, Chen
• Granted Patent: US 10,111,022 B2
• Time in Patent Office: Days
• CPC Class Codes: G10L 19/008 Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing
• CPC Class Codes: H04S 2400/11 Positioning of individual sound objects, e.g. moving airplane, within a sound field H04S2420/13 takes precedence
• CPC Class Codes: H04S 3/008 in which the audio signals are in digital form, i.e. employing more than two discrete digital channels data reduction aspects thereof based on psychoacoustics G10L19/02
• CPC Class Codes: H04S 7/30 Control circuits for electronic adaptation of the sound field
• CPC Class Codes: H04S 7/302 Electronic adaptation of stereophonic sound system to listener position or orientation H04S7/301 takes precedence

Abstract

An audio processing system and method which calculates, based on spatial metadata of the audio object, a panning coefficient for each of the audio objects in relation to each of a plurality of predefined channel coverage zones. Converts the audio signal into submixes in relation to the predefined channel coverage zones based on the calculated panning coefficients and the audio objects. Each of the submixes indicating a sum of components of the plurality of the audio objects in relation to one of the predefined channel coverage zones. Generating a submix gain by applying an audio processing to each of the submix and controls an object gain applied to each of the audio objects. The object gain being as a function of the panning coefficients for each of the audio objects and the submix gains in relation to each of the predefined channel coverage zones.

Claims

1. A method of processing an audio signal, the audio signal having a plurality of audio objects, the method comprising:

calculating, based on spatial metadata of the audio object, a panning coefficient for each of the audio objects in relation to each of a plurality of predefined channel coverage zones, the predefined channel coverage zones being defined by a plurality of endpoints distributed in a sound field;

converting the audio signal into submixes in relation to the predefined channel coverage zones based on the calculated panning coefficients and the audio objects, each of the submixes indicating a sum of components of the plurality of the audio objects in relation to one of the predefined channel coverage zones;

generating a submix gain by applying an audio processing to each of the submixes; and

controlling an object gain applied to each of the audio objects, the object gain being as a function of the panning coefficients for each of the audio objects and the submix gains in relation to each of the predefined channel coverage zones.

2. The method according to claim 1, further comprising:

rendering the audio signal based on the audio objects and the object gain.

3. The method according to claim 1, wherein each of the submixes is converted as a weighted average of the plurality of audio objects, with the weight being the panning coefficient for each of the audio objects.

4. The method according to claim 1, wherein the number of the predefined channel coverage zones is equal to the number of the converted submixes.

5. The method according to claim 1, further comprising:

determining whether the audio object belongs to a dialog object; and

in response to the audio object being determined to be a dialog object, clustering the audio object to a dialog submix.

6. The method according to claim 5, wherein whether the audio object belongs to a dialog object is estimated with a confidence score, and the method further comprises generating the submix gain for the dialog submix based on the estimated confidence score.

7. The method according to claim 1, wherein the predefined channel coverage zones comprise:

a front zone defined by a front left channel and a front right channel,

a center zone defined by a center channel,

a surround zone defined by a surround left channel and a surround right channel, and

a height zone defined by a height channel.

8. The method according to claim 7, wherein converting the audio signal into submixes further comprises:

converting the audio signal into a front submix in relation to the front zone based on the panning coefficients for the audio objects;

converting the audio signal into a center submix in relation to the center zone based on the panning coefficients for the audio objects;

converting the audio signal into a surround submix in relation to the surround zone based on the panning coefficients for the audio objects; and

converting the audio signal into a height submix in relation to the height zone based on the panning coefficients for the audio objects.

9. The method according to claim 8, further comprising:

merging the center submix and the front submix; and

replacing the center submix by the dialog submix.

10. The method according to claim 8, further comprising:

applying a same audio processing algorithm on the surround submix and the height submix to generate the corresponding submix gains.

11. The method according to claim 1, further comprising:

for each of the audio objects, identifying a type of the audio object; and

generating the submix gain by applying an audio processing to each of the submixes based on the identified type of the audio object.

23. A computer program product for rendering an audio signal, the computer program product being tangibly stored on a non-transient computer-readable medium and comprising machine executable instructions which, when executed, cause the machine to perform steps of the method according to claim 1.

12. A system for processing an audio signal, the audio signal having a plurality of audio objects, the system comprising:

a panning coefficient calculating unit configured to calculate, based on spatial metadata of the audio object, a panning coefficient for each of the audio objects in relation to each of a plurality of predefined channel coverage zones, the predefined channel coverage zones being defined by a plurality of endpoints distributed in a sound field;

a submix converting unit configured to convert the audio signal into submixes in relation to all of the predefined channel coverage zones based on the calculated panning coefficients and the audio objects, each of the submixes indicating a sum of components of the plurality of the audio objects in relation to one of the predefined channel coverage zones;

a submix gain generating unit configured to generate a submix gain by applying an audio processing to each of the submixes; and

an object gain controlling unit configured to control an object gain applied to each of the audio objects, the object gain being as a function of the panning coefficients for each of the audio objects and the submix gains in relation to each of the predefined channel coverage zones.

13. The system according to claim 12, further comprising:

an audio signal rendering unit configured to render the audio signal based on the audio objects and the object gain.

14. The system according to claim 12, wherein each of the submixes is converted as a weighted average of the plurality of audio objects, with the weight being the panning coefficient for each of the audio objects.

15. The system according to claim 12, wherein the number of the predefined channel coverage zones is equal to the number of the converted submixes.

16. The system according to claim 12, further comprising:

a dialog determining unit configured to determine whether the audio object belongs to a dialog object;

a dialog object clustering unit configured to cluster the audio object to a dialog submix in response to the audio object being determined to be a dialog object.

17. The system according to claim 16, wherein whether the audio object belongs to a dialog object is estimated with a confidence score, and the system further comprises a dialog submix gain generating unit configured to generate the submix gain for the dialog submix based on the estimated confidence score.

18. The system according to claim 12, wherein the predefined channel coverage zones comprise:

a front zone defined by a front left channel and a front right channel,

a center zone defined by a center channel,

a surround zone defined by a surround left channel and a surround right channel, and

a height zone defined by a height channel.

19. The system according to claim 18, further comprising:

a front submix converting unit configured to convert the audio signal into a front submix in relation to the front zone based on the panning coefficients for the audio objects;

a center submix converting unit configured to convert the audio signal into a center submix in relation to the center zone based on the panning coefficients for the audio objects;

a surround submix converting unit configured to convert the audio signal into a surround submix in relation to the surround zone based on the panning coefficients for the audio objects; and

a height submix converting unit configured to convert the audio signal into a height submix in relation to the height zone based on the panning coefficients for the audio objects.

20. The system according to claim 19, further comprising:

a merging unit configured to merge the center submix and the front submix; and

a replacing unit configured to replace the center submix by the dialog submix.

21. The system according to claim 19, wherein the surround submix and the height submix are applied with a same audio processing algorithm in order to generate the corresponding submix gains.

22. The system according to claim 12, further comprising:

an object type identifying unit configured, for each of the audio objects, to identify a type of the audio object, and

wherein the submix gain generating unit is configured to generate the submix gain by applying an audio processing to each of the submixes based on the identified type of the audio object.