Upmixing of audio signals

• US 10,362,426 B2
• Filed: 02/09/2016
• Issued: 07/23/2019
• Est. Priority Date: 02/09/2015
• Status: Active Grant

• Current Assignee: Dolby Laboratories Licensing Corporation (Dolby Laboratories Incorporated)
• Original Assignee: Dolby Laboratories Licensing Corporation (Dolby Laboratories Incorporated)
• Inventors: Wang, Jun, Lu, Lie, Chen, Lianwu, Hu, Mingqing
• Primary Examiner(s): Ton, David L
• Application Number:
  US15/538,892
• Publication Number:
  US 20180262856A1
• Time in Patent Office: 1,260 Days
• CPC Class Codes: H04R 1/323 for loudspeakers H04R1/34 and H04R1/40 take precedence
• CPC Class Codes: H04S 2400/11 Positioning of individual sound objects, e.g. moving airplane, within a sound field H04S2420/13 takes precedence
• CPC Class Codes: H04S 3/008 in which the audio signals are in digital form, i.e. employing more than two discrete digital channels data reduction aspects thereof based on psychoacoustics G10L19/02
• CPC Class Codes: H04S 5/00 Pseudo-stereo systems, e.g. in which additional channel signals are derived from monophonic signals by means of phase shifting, time delay or reverberation
• CPC Class Codes: H04S 5/005 of the pseudo five- or more-channel type, e.g. virtual surround
• CPC Class Codes: H04S 7/308 Electronic adaptation dependent on speaker or headphone connection

Abstract

Example embodiments disclosed herein relates to upmixing of audio signals. A method of upmixing an audio signal is described. The method includes decomposing the audio signal into a diffuse signal and a direct signal, generating an audio bed at least in part based on the diffuse signal, the audio bed including a height channel, extracting an audio object from the direct signal, estimating metadata of the audio object, the metadata including height information of the audio object; and rendering the audio bed and the audio object as an upmixed audio signal, wherein the audio bed is rendered to a predefined position and the audio object is rendered according to the metadata. Corresponding system and computer program product are described as well.

Claims

1. A method of upmixing an audio signal, comprising:

decomposing the audio signal into a diffuse signal and a direct signal;

determining complexity of the audio signal;

generating an audio bed at least in part based on the diffuse signal, the audio bed including a height channel;

extracting an audio object from the direct signal;

estimating metadata of the audio object, the metadata including height information of the audio object;

determining at least one of a diffuse gain for the audio signal based on the complexity, an object gain for the audio signal based on the complexity, or a height gain for the audio signal based on the complexity; and

rendering the audio bed and the audio object as an upmixed audio signal, wherein the audio bed is rendered to at least one predefined position and the audio object is rendered according to the metadata.

2. The method of claim 1, wherein the generating the audio bed comprises:

upmixing the diffuse signal to create the height channel; and

including a residual signal into the audio bed, the residual signal obtained from the extracting of the audio object.

3. The method of claim 1, wherein the decomposing the audio signal comprises:

determining a diffuse gain for the audio signal based on the complexity, the diffuse gain indicating a proportion of the diffuse signal in the audio signal; and

decomposing the audio signal based on the diffuse gain.

4. The method of claim 1, wherein the extracting the audio object comprises:

determining an object gain for the audio signal based on the complexity, the object gain indicating a probability that the audio signal contains an audio object; and

extracting the audio object based on the object gain.

5. The method of claim 1, wherein the extracting the metadata comprises:

determining a height gain for the audio object based on the complexity; and

modifying the height information of the audio object based on the height gain.

6. The method of claim 1, wherein the rendering the audio object comprises:

determining an object-rendering gain based on at least one of the following;

the number of active channels of the audio object,

a position of the audio object with respect to a user, and

energy distribution among channels for the audio object; and

controlling, based on the object-rendering gain, a level of rendering related to the audio object in the rendering.

9. A computer program product of upmixing an audio signal, the computer program product being tangibly stored on a non-transient computer-readable medium and comprising machine executable instructions which, when executed, cause the machine to perform steps of the method according to claim 1.

7. A method of upmixing an audio signal, comprising:

decomposing the audio signal into a diffuse signal and a direct signal;

generating an audio bed at least in part based on the diffuse signal, the audio bed including a height channel;

extracting an audio object from the direct signal;

estimating metadata of the audio object, the metadata including height information of the audio object and

rendering the audio bed and the audio object as an upmixed audio signal, wherein the audio bed is rendered to at least one predefined position and the audio object is rendered according to the metadata, wherein the decomposing the audio signal comprises;

applying a first decomposition process to obtain the diffuse signal; and

applying a second decomposition process to obtain the direct signal, the first decomposition process having less diffuse-to-direct leakage than the second decomposition process.

8. The method of claim 7, further comprising:

pre-upmixing the audio signal,

wherein the first and second decomposition processes are separately applied to the pre-upmixed audio signal.