Apparatus and method for multi-channel signal playback

• US 9,313,599 B2
• Filed: 08/15/2011
• Issued: 04/12/2016
• Est. Priority Date: 11/19/2010
• Status: Active Grant

• Current Assignee: Nokia Technologies Oy (Nokia Corporation)
• Original Assignee: Nokia Technologies Oy (Nokia Corporation)
• Inventors: Tammi, Mikko T., Vilermo, Miikka T.
• Primary Examiner(s): Nguyen, Duc
• Assistant Examiner(s): Monikang, George
• Application Number:
  US13/209,738
• Publication Number:
  US 20130044884A1
• Time in Patent Office: 1,702 Days
• Field of Search: 381/26, 381/300, 381/303, 381/307, 381 17- 19, 381309-310, 381/92, 381/122
• US Class Current: 1/1
• CPC Class Codes: H04R 2227/005 Audio distribution systems for home, i.e. multi-room use
• CPC Class Codes: H04R 5/02 Spatial or constructional arrangements of loudspeakers
• CPC Class Codes: H04S 7/303 Tracking of listener position or orientation

Abstract

Techniques are presented for creating multichannel output signals from input audio signals. A first signal is determined based on a number of subbands into which the input audio signals are divided and based at least in part on a directional estimation wherein the subbands having dominant sound source directions are emphasized relative to subbands having directional estimates that deviate from directional estimates of the dominant sound source directions. A second signal is determined based on the number of subbands wherein an ambient component is introduced to create a perception of an externalization for a sound image. A resultant audio signal is created using the first and second signals. The resultant audio signal is one of a number of multichannel signals. Additionally, it is determined whether binaural audio output or multichannel audio output (or both) is to be output, and the appropriate number of audio output signals are determined and output.

Claims

1. An apparatus, comprising:

one or more processors; and

one or more memories including computer program code,

the one or more memories and the computer program code configured, with the one or more processors, to cause the apparatus to perform at least the following;

accessing at least two audio signals captured using at least two omnidirectional microphones and corresponding to a received sound, wherein the at least two omnidirectional microphones are effective for capturing the at least two audio signals coming from all directions;

determining a directional estimation based on subbands between the at least two audio signals and wherein one or more dominant sound source directions are determined, each of the one or more dominant sound source directions being dependent on a direction of the received sound, where the received sound can come from potentially all directions;

determining a first signal based at least in part on the directional estimation wherein the determined first signal comprises emphasized subbands having the one or more dominant sound source directions relative to subbands that deviate from the one or more dominant sound source directions, wherein the subbands that deviate from the one or more dominant sound source directions are attenuated, and wherein the emphasized subbands include a mid signal of the received sound;

determining a second signal wherein an ambience component is introduced for a sound image, wherein the ambience component includes a side signal of the received sound; and

creating a resultant audio signal for the sound image using the first and second signals wherein the resultant audio signal is at least one multichannel signal having the ambience component including the side signal, the emphasized subbands having the one or more dominant sound source directions including the mid signal and the attenuated subbands that deviate from the one or more dominant sound source directions.

2. The apparatus of claim 1, wherein the one or more memories and the computer program code are further configured, with the one or more processors, to cause the apparatus to perform at least the following:

performing the determining the first signal, the determining the second signal, and the creating the resultant audio signal for each said at least one multichannel signal.

3. The apparatus of claim 1, wherein:

determining the directional estimation further comprises determining the directional estimation for the at least two audio signals based on a plurality of subbands wherein the directional estimation is provided for subband pairs between the at least two audio signals, and determining subbands having the one or more dominant sound source directions;

determining a first signal further comprises determining the first signal based on the plurality of subbands wherein the subbands having the one or more dominant sound source directions are emphasized relative to subbands having the directional estimates that deviate from the directional estimates of the one or more dominant sound source directions;

in response to the directional estimation for a selected subband pair meeting a predetermined criteria indicating the first and second audio signals are dissimilar, setting to zero a delay used to shift a time-shifted version of the second signal in the selected subband pair; and

determining the first signal using an average for each subband of the first audio signal and the time-shifted version of the second audio signal.

4. The apparatus of claim 3, wherein determining a first signal further comprises:

Determining the mid signal using the plurality of subbands;

determining gain values for each of the plurality of subbands in the mid signal, the gain values at least partially determined using the directional estimation for each subband; and

applying the gain values for each of the subbands to the mid signal to create the first signal.

5. The apparatus of claim 4, wherein:

determining a first signal further comprises;

for individual ones of the subband pairs, in response to the directional estimation for a selected subband meeting predetermined criteria indicating the at least two audio signals are dissimilar, marking the directional estimation as a predetermined value; and

determining gain values further comprises;

in response to the directional estimation for a selected subband being marked as the predetermined value, setting the gain value corresponding to the subband to a predetermined fixed gain.

6. The apparatus of claim 4, wherein:

determining the first signal further comprises prior to applying the gain values, applying a smoothing filter to the gain values in the plurality of subbands to create smoothed gain values; and

applying the gain values further comprises applying the smoothed gain values per subband to create the first signal.

7. The apparatus of claim 1, wherein determining a second signal further comprises:

Determining the side signal from a plurality of subbands; and

decorrelating the side signal, the decorrelating performed so that side signals for each of the multichannel signals have a predetermined low amount of cross-correlation with each other.

8. The apparatus of claim 7, wherein creating an audio signal further comprises:

delaying, by an amount corresponding to a time of decorrelation in the decorrelating, a time-domain version of the first signal to create a time delayed version of the first signal; and

adding a scaled version of the second signal to the time delayed version of the first signal to create the resultant audio signal.

9. The apparatus of claim 1, wherein the at least two audio signals are received from a wireless or wired network.

10. The apparatus of claim 1, wherein the apparatus further comprises at least two microphones, and wherein each of the at least two audio signals comprises a microphone signal from an individual one of the at least two microphones.

11. The apparatus of claim 1, wherein the ambience component is determined based on subbands between the at least two audio signals.

12. A method, comprising:

accessing at least two audio signals captured using at least two omnidirectional microphones and corresponding to a received sound, wherein the at least two omnidirectional microphones are effective for capturing the at least two audio signals coming from all directions;

determining a directional estimation based on subbands between the at least two signals and wherein one or more dominant sound source directions are determined, each of the one or more dominant sound source directions being dependent on a direction of the received sound, where the received sound can come from potentially all directions;

determining a first signal based at least in part on the directional estimation wherein the determined first signal comprises emphasized sub bands having the one or more dominant sound source directions relative to subbands that deviate from the one or more dominant sound source directions, wherein the subbands that deviate from the one or more dominant sound source directions are attenuated, and wherein the emphasized subbands include a mid signal of the received sound;

determining a second signal wherein an ambience component is introduced for a sound image, wherein the ambience component includes a side signal of the received sound; and

creating a resultant audio signal for the sound image using the first and second signals wherein the resultant audio signal is at least one multichannel signal having the ambience component including the side signal, the emphasized subbands having the one or more dominant sound source directions including the mid signal and the attenuated subbands that deviate from the one or more dominant sound source directions.

13. An apparatus, comprising:

one or more processors; and

one or more memories including computer program code,

the one or more memories and the computer program code configured to, with the one or more processors, cause the apparatus to perform at least the following;

determining a directional estimation based on subbands between at least two input audio signals captured using at least two omnidirectional microphones, wherein the at least two omnidirectional microphones are effective for capturing the at least two input audio signals coming from all directions, and wherein one or more dominant sound source directions are determined, each of the one or more dominant sound source directions being dependent on a direction of a received sound, where the received sound can come from potentially all directions;

determining a first signal comprised of subbands and based at least in part on the directional estimation wherein the determined first signal comprises emphasized subbands having the one or more dominant sound source directions relative to subbands that deviate from the one or more dominant sound source directions, wherein the subbands that deviate from the one or more sound source directions are attenuated, and wherein the emphasized subbands include a mid signal of the received sound;

and determining a second signal based on the plurality of subbands wherein an ambience component is introduced for a sound image;

determining whether one or both of binaural audio output or multi-channel audio output should be output;

in response to a determination binaural audio output should be output, synthesizing binaural signals from the first and the second signal, processing the binaural signals into two audio output signals, and outputting the two audio output signals; and

in response to a determination multi-channel audio output should be output, synthesizing at least two audio output signals for the sound image from the first signal and the second second signal, and outputting the at least two audio output signals having the ambience component including the side signal, the emphasized subbands having the one or more dominant sound source directions including the mid signal and the attenuated subbands that deviate from the one or more dominant sound source directions.

14. The apparatus of claim 13, wherein the at least two audio output signals comprise audio output signals for at least the following channels:

center channel, front-left channel, front-right channel, rear-left strange, and rear-right channel.

15. The apparatus of claim 13, wherein the two audio output signals are one of analog signals or digital signals.

16. The apparatus of claim 13, wherein:

outputting the two audio output signals further comprises outputting the two audio output signals into a file; and

outputting the at least two audio output signals further comprises outputting the at least two audio output signals into a file.

17. The apparatus of claim 16, wherein responsive to both the two audio output signals and the at least two audio output signals being output, the file containing the two audio output signals and the file containing the at least two audio output signals are a single file.

18. The apparatus of claim 16, wherein responsive to both the two audio output signals and the at least three audio output signals being output, the file containing the two audio output signals and the file containing the at least two audio output signals are different files.

19. The apparatus of claim 13, wherein:

the apparatus further comprises a two-channel output connection and a multi-channel output connection;

determining whether one or both of binaural audio output or multi-channel audio output should be created further comprises;

determining whether the two-channel audio output or the multi-channel audio output is being used;

in response to the two-channel audio output being used, making a determination the two audio output signals should be output; and

in response to the multi-channel audio output being used, making a determination the at least two audio output signals should be output;

outputting the two audio output signals further comprises outputting the two audio output signals on the two-channel audio output;

outputting the at least two audio signals further comprises outputting the at least two audio signals on the multi-channel audio output; and

responsive to only one of the two-channel audio output or the multi-channel audio output is being used, performing only a corresponding one of the outputting of the two audio output signals or the outputting of the at least two audio signals.

20. The apparatus of claim 13, wherein determining whether one or both of binaural audio output or multi-channel audio output should be output further comprises:

allowing a user to select which one or both of the binaural audio output or the multi-channel audio output should be output; and

performing the determining based on a selection made by the user.