Controllable playback system offering hierarchical playback options

• US 9,055,371 B2
• Filed: 02/03/2012
• Issued: 06/09/2015
• Est. Priority Date: 11/19/2010
• Status: Active Grant

• Current Assignee: Nokia Technologies Oy (Nokia Corporation)
• Original Assignee: Nokia Technologies Oy (Nokia Corporation)
• Inventors: Tammi, Mikko T., Vilermo, Miikka T.
• Primary Examiner(s): Paul, Disler
• Application Number:
  US13/365,468
• Publication Number:
  US 20130202114A1
• Time in Patent Office: 1,222 Days
• Field of Search: 381/1, 381 22- 23, 381/92, 381/17, 381/18
• US Class Current: 1/1
• CPC Class Codes: H04R 1/406 microphones
• CPC Class Codes: H04R 2201/40 Details of arrangements for obtaining desired directional characteristic by combining a number of identical transducers covered by H04R1/40 but not provided for in any of its subgroups
• CPC Class Codes: H04R 3/005 for combining the signals of two or more microphones specially adapted for hearing aids H04R25/407
• CPC Class Codes: H04R 3/12 for distributing signals to two or more loudspeakers specially adapted for hearing aids H04R25/407
• CPC Class Codes: H04R 5/027 Spatial or constructional arrangements of microphones, e.g. in dummy heads
• CPC Class Codes: H04R 5/04 Circuit arrangements, e.g. for selective connection of amplifier inputs/outputs to loudspeakers, for loudspeaker detection, or for adaptation of settings to personal preferences or hearing impairments combinations of amplifiers H03F3/68; stereophonic systems H04S
• CPC Class Codes: H04S 2420/01 Enhancing the perception of the sound image or of the spatial distribution using head related transfer functions [HRTF's] or equivalents thereof, e.g. interaural time difference [ITD] or interaural level difference [ILD]

Abstract

A first apparatus performs the following: determining, using at least two microphone signals corresponding to left and right microphone signals and using at least one further microphone signal, directional information of the left and right microphone signals; outputting a first signal corresponding to the left microphone signal; outputting a second signal corresponding to the right microphone signal; and outputting a third signal corresponding to the determined directional information. Another apparatus performs the following: performing at least one of the following: outputting first and second signals as stereo output signals; or converting the first and second signals to mid and side signals, and converting, using directional information for the first and second signals, the mid and side signals to at least one of binaural signals or multi-channel signals, and outputting the corresponding binaural signals or multi-channel signals. Additional apparatus, program products, and methods are disclosed.

Claims

1. An apparatus, comprising:

one or more processors; and

one or more memories including computer program code,

the one or more memories and the computer program code configured, with the one or more processors, to cause the apparatus to perform at least the following;

determining, using at least two microphone signals, directional information of a sound source, wherein mid signals of the at least two microphone signals represent the directional information of the sound source and side signals of the at least two microphone signals represent ambiance information of the sound source; and

outputting a multichannel output signal for an audio playback device, wherein the multichannel output signal comprises a number of output channels dependent on an availability of output channels of the audio playback device, wherein if the multichannel output signal is binaural signals, or if the multichannel output signal is multichannel signals greater than two channels, then the multichannel output signal is outputted based on the determined directional information and the ambience information using the mid and side signals.

2. The apparatus of claim 1, wherein the at least two microphone signals includes a first microphone signal generated by a left microphone and a second microphone signal generated by a right microphone, and a third microphone signal generated by a third microphone;

and wherein determining further comprises determining the directional information including whether the sound source is in one of two possible directions relative to at least one of the left microphone or the right microphone based on the third microphone signal.

3. The apparatus of claim 2, wherein determining further comprises assigning a first value to a first of the two possible directions and assigning a second value to a second of the two possible directions, wherein the first value is a zero and the second value is a one.

4. The apparatus of claim 1, wherein:

determining further comprises

determining high quality left and right signals using the mid and side signals, and the directional information of the sound source; and

wherein a first output signal of the multichannel output signal corresponds to a first microphone signal of said at least two microphone signals and comprises the high quality left signal, and a second output signal of the multichannel output signal corresponds to a second microphone signal of said at least two microphone signals and comprises the high quality right signal.

5. The apparatus of claim 4, wherein determining the high quality left and right signals using the mid and side signals further comprises, for each subband of a plurality of subbands of a frequency range into which frequency domain representations of the mid and side signals are arranged, creating a high quality left signal at least by multiplying the mid signal by a left panning factor and creating a high quality right signal at least by multiplying the mid signal by a right panning factor, wherein the left and right panning factors are outputs of a respective left or right panning function, and the left and right panning functions have the directional information as input.

6. The apparatus of claim 5, wherein creating the high quality left signal further comprises adding a first decorrelated side signal to the panned mid signal, and wherein creating the high quality right signal further comprises adding a second decorrelated side signal to the panned mid signal.

7. The apparatus according to claim 4, wherein the high quality left signal, the high quality right signal and the directional information are used to output the multichannel output signal comprising two or more channels dependent on the availability of output channels of the audio playback device.

8. The apparatus of claim 5, wherein creating the high quality left and right signals further comprises adding a decorrelated side signal to one of the panned mid signals for one of the high quality left signal or the high quality right signal and adding the side signal to the other of the high quality left signal or the high quality right signal.

9. The apparatus of claim 1, wherein the directional information is determined using said at least two microphone signals corresponding to left and right microphone signals, and using at least one further microphone signal.

10. The apparatus of claim 1, further comprising determining, using said at least two microphone signals, ambience information of the sound source;

and wherein the multichannel output signal comprises the determined directional information and the determined ambience information.

11. The apparatus of claim 1, wherein the number of output channels is automatically selected dependent on the availability of output channels of the audio playback equipment.

12. The apparatus of claim 1, wherein if the multichannel output signal is a stereo signal, the multichannel output signal is generated without using the mid and side signals.

13. The apparatus of claim 1, wherein if the multichannel output signal is binaural signals, signal levels and delays between two channels of the multichannel output signal forming the binaural signals are modified based on the mid and side signals.

14. An apparatus, comprising:

one or more processors; and

one or more memories including computer program code,

the one or more memories and the computer program code configured, with the one or more processors, to cause the apparatus to perform at least the following;

performing at least one of the following;

determining a type of playback for an audio playback device;

if the type of playback is a stereo playback, then

outputting first and second signals as stereo output signals of a multichannel output signal for an audio playback device based on a determined directional information of a sound source, wherein the multichannel output signal comprises a number of output channels dependent on an availability of output channels of the audio playback device;

if the type of playback is a binaural or greater than two channel multichannel playback;

then

converting the first and second signals to mid and side signals wherein the mid signals represent the determined directional information of the sound source and the side signals represent ambiance information of the sound source, and outputting corresponding binaural signals or multichannel signals greater than two channels as the multichannel output signal for the audio playback device based on the determined directional information and ambiance information, wherein the multichannel output signal comprises a number of output channels dependent on an availability of output channels of the audio playback device, and

wherein if the multichannel output signal is binaural signals, or if the multichannel output signal is multichannel signals greater than two channels, then the multichannel output signal is outputted based on the determined directional information and the ambience information using the mid and side signals.

15. The apparatus of claim 14, wherein the directional information includes whether the sound source is in one of two possible directions.

16. The apparatus of claim 15, wherein a first of the two possible directions has a first value and a second of the two possible directions has a second value, wherein the first value is a zero and the second value is a one.

17. The apparatus of claim 14, wherein the first signal comprises a high quality left signal and the second signal comprises a high quality right signal.

18. The apparatus of claim 17, wherein converting the first and second signals to a mid signal further comprises, for each of a plurality of frequency bins in each of a plurality subbands of a frequency range into which frequency domain representations of the first and second signals are arranged:

determining the mid signal at least by subtracting a decorrelated version of the high quality right signal from the high quality left signal to create a first result, subtracting a decorrelated version of a right panning factor from a left panning factor to create a second result, and dividing the first result by the second result to determine the mid signal, wherein the right and left panning factors are based on directional information for a corresponding subband;

determining the side signal by subtracting the left panning factor multiplied by the determined mid signal from the high quality left signal to create a third result and applying a decorrelation function to the third result to determine the side signal.

19. The apparatus of claim 18, wherein:

the decorrelated version of the high quality right signal is determined by applying an inverse of a right decorrelation function corresponding to the high quality right signal to the high quality right signal to create a fourth result and applying a left decorrelation function corresponding to the high quality left signal to the fourth result to create the decorrelated version of the high quality right signal;

the decorrelated version of right panning factor is determined by applying an inverse of the right decorrelation function to the right panning factor to create a fifth result and applying the left decorrelation function to the fifth result to create the decorrelated version of the right panning factor; and

the decorrelation function applied to the third result is an inverse of the left decorrelation function.

20. The apparatus of claim 17, wherein converting the first and second signals to a mid signal further comprises, for each of a plurality of frequency bins in each of a plurality of subbands of a frequency range into which frequency domain representations of the first and second signals are arranged:

determining the mid signal at least by subtracting a decorrelated version of the high quality right signal from the high quality left signal to create a first result, subtracting a decorrelated version of a right panning factor from a left panning factor to create a second result, and dividing the first result by the second result to determine the mid signal, wherein the right and left panning factors are based on directional information for a corresponding subband;

determining the side signal by subtracting the right panning factor multiplied by the determined mid signal from the high quality right signal to determine the side signal.

21. The apparatus of claim 20, wherein the decorrelated version of the high quality right signal is determined by applying a left decorrelation function corresponding to the high quality left signal to the high quality right signal, and wherein the decorrelated version of the right panning factor is determined by applying the left decorrelation function to the right panning factor.

22. The apparatus of claim 17, wherein converting the first and second signals to a mid signal further comprises, for each of a plurality of frequency bins in each of a plurality of subbands of a frequency range into which frequency domain representations of the first and second signals are arranged:

determining the mid signal at least by subtracting a decorrelated version of the high quality left signal from the high quality right signal to create a first result, subtracting a decorrelated version of a left panning factor from a right panning factor to create a second result, and dividing the first result by the second result to determine the mid signal, wherein the right and left panning factors are based on directional information for a corresponding subband; and

determining the side signal by subtracting the left panning factor multiplied by the determined mid signal from the high quality left signal to determine the side signal.

23. A method, comprising:

determining, using at least two microphone signals, directional information of a sound source, wherein mid signals of the at least two microphone signals represent the directional information of the sound source and side signals of the at least two microphone signals represent ambiance information of the sound source; and

outputting a multichannel output signal for an audio playback device based on the determined directional information of the sound source, wherein the multichannel output signal comprises a number of output channels dependent on an availability of output channels of the audio playback device, wherein if the multichannel output signal is binaural signals, or if the multichannel output signal is multichannel signals greater than two channels, then the multichannel output signal is outputted based on the determined directional information and the ambience information using the mid and side signals.