Converting multi-microphone captured signals to shifted signals useful for binaural signal processing and use thereof

• US 9,456,289 B2
• Filed: 11/19/2010
• Issued: 09/27/2016
• Est. Priority Date: 11/19/2010
• Status: Active Grant

• Current Assignee: Nokia Technologies Oy (Nokia Corporation)
• Original Assignee: Nokia Technologies Oy (Nokia Corporation)
• Inventors: Tammi, Mikko T., Vilermo, Miikka T.
• Primary Examiner(s): Chin, Vivian
• Assistant Examiner(s): SUTHERS, DOUGLAS JOHN
• Application Number:
  US12/927,663
• Publication Number:
  US 20120128174A1
• Time in Patent Office: 2,139 Days
• Field of Search: 381/92, 381/26, 381/94.1, 381/94.2, 381/94.3, 381/122, 381/20, 381/22, 381/23
• US Class Current: 1/1
• CPC Class Codes: G10L 19/008 Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing
• CPC Class Codes: H04R 2430/23 Direction finding using a sum-delay beam-former
• CPC Class Codes: H04S 1/002 Non-adaptive circuits, e.g. manually adjustable or static, for enhancing the sound image or the spatial distribution control circuits for electronic adaptation of the sound field H04S7/30
• CPC Class Codes: H04S 2400/01 Multi-channel, i.e. more than two input channels, sound reproduction with two speakers wherein the multi-channel information is substantially preserved
• CPC Class Codes: H04S 2400/15 Aspects of sound capture and related signal processing for recording or reproduction
• CPC Class Codes: H04S 2420/07 Synergistic effects of band splitting and sub-band processing

Abstract

A method includes, for each of a number of subbands of a frequency range and for at least first and second frequency-domain signals that are frequency-domain representations of corresponding first and second audio signals: determining a time delay of the first frequency-domain signal that removes a time difference between the first and second frequency-domain signals in the subband. The method includes forming a first resultant signal including, for each of the number of subbands, a sum of one of the first or second frequency-domain signals shifted by the time delay and of the other of the first or second frequency-domain signals; and forming a second resultant signal including, for each of the number of subbands, a difference between the shifted one of the first or second frequency-domain signals and the other of the first or second frequency-domain signals. Apparatus and program products are also disclosed.

Claims

1. A method comprising:

estimating directional information based on multiple input channel signals representing at least one arriving sound from a sound source captured by respective multiple microphones that have respective known locations relative to each other;

deriving a mid-signal and a side signal on basis of a first input channel signal, a second input channel signal and said estimated directional information; and

generating an output signal comprising a plurality of output channels using said mid-signal, said side signal and said estimated directional information such that the output signal retains a spatial representation of the captured at least one arriving sound, wherein said generating comprises processing the mid-signal and the side signal using said estimated directional information, and combining the processed mid-signal and the processed side signal to determine at least a left channel signal and a right channel signal of said output signal that retains the spatial representation of the captured at least one arriving sound.

2. The method as claimed in claim 1, wherein said estimating comprises finding a time delay that removes a time difference between said first and second input channel signals and wherein said deriving comprises;

deriving the mid-signal as a sum of one of said first and second input channel signals shifted by said time delay and the other one of said first and second input channel signals; and

deriving the side signal as a difference between the shifted one of said first and second input channel signals and the other one of said first and second input channel signals.

3. The method as claimed in claim 1, wherein said estimating comprises determining an angle that represents direction of said sound source with respect to said known locations.

4. The method as claimed in claim 1, wherein said estimating comprises estimating the directional information separately in a plurality of subbands of said multiple input channel signals;

and said deriving comprises deriving the mid-signal and the side signals in said plurality of subbands.

5. The method as claimed in claim 1, wherein said estimating and said deriving are carried out on frequency-domain signals.

6. The method as claimed in claim 1, wherein said generating comprises encoding the mid-signal to obtain an encoded mid-signal;

encoding the side signal to obtain an encoded side signal; and

encoding the estimated directional information to obtain encoded directional information.

7. The method as claimed in claim 6, further comprising transmitting the encoded mid-signal, the encoded side signal and the encoded directional information.

8. The method as claimed in claim 7, further comprising receiving the encoded mid-signal, the encoded side signal and the encoded directional information and wherein said generating further comprises decoding the encoded mid-signal to obtain the mid-signal;

decoding the encoded side signal to obtain the side-signal; and

decoding the encoded directional information to obtain the estimated directional information.

9. The method as claimed in claim 1, wherein said output signal consists of two output channels.

10. The method as claimed in claim 1, wherein processing comprises applying, to subbands of said mid-signal below a certain frequency, left and right head related transfer functions to determine respective subbands of the left and right mid-signals;

applying, to subbands of the mid-signal above said certain frequency, magnitude of said left and right head related transfers functions and a fixed delay corresponding to said head related transfer functions to determine the respective subbands of the left and right mid-signal; and

applying, to subbands of the side signal, said fixed delay to determine left and right side signals, and wherein combining comprises combining the left mid-signal with the left side signal and combining the right mid-signal with the right side signal.

11. The method as claimed in claim 10, wherein said combining comprises returning an average energy of said mid-signal to its original level while maintaining a level difference between said left and right channel signals.

12. The method as claimed in claim 1, wherein said multiple microphones comprise at least three microphones arranged in a geometrical shape of a triangle.

13. A computer program product embodied on a non-transitory computer-readable medium in which a computer program is stored that, when being executed by a computer, is configured to perform the method of claim 1.

19. The method as claimed in claim 10, wherein said combining further comprises decorrelating the side signal so as to enhance the externalisation of the generated output signal and delaying the left and right mid-signals by an average group delay of a decorrelation filter.

14. An apparatus, comprising

at least one processor,

and at least one non-transitory computer readable medium including computer program code,

the at least one non-transitory computer readable medium and the computer program code configured to, with the at least one processor, cause the apparatus at least to perform:

estimating directional information based on multiple input channel signals representing at least one arriving sound from a sound source captured by respective multiple microphones that have respective known locations relative to each other;

deriving a mid-signal and a side signal on basis of a first input channel signal, a second input channel signal and said estimated directional information and

generating an output signal comprising a plurality of output channels using said mid-signal, said side signal and said estimated directional information such that the output signal retains a spatial representation of the captured at least one arriving sound, wherein said generating comprises processing the mid-signal and the side signal using said estimated directional information, and combining the processed mid-signal and the processed side signal to determine at least a left channel signal and a right channel signal of said output signal that retains the spatial representation of the captured at least one arriving sound.

15. The apparatus as claimed in claim 14, wherein said estimating comprises finding a time delay that removes a time difference between said first and second input channel signals and wherein said deriving comprises;

deriving the mid-signal as a sum of one of said first and second input channel signals shifted by said time delay and the other one of said first and second input channel signals; and

deriving the side signal as a difference between the shifted one of said first and second input channel signals and the other one of said first and second input channel signals.

16. The apparatus as claimed in claim 14, wherein said estimating comprises determining an angle that represents direction of said sound source with respect to said known locations.

17. The apparatus as claimed in claim 14, wherein said estimating comprises estimating the directional information separately in a plurality of subbands of said multiple input channel signals;

and said deriving comprises deriving the mid-signal and the side signals in said plurality of subbands.

18. The apparatus as claimed in claim 14, wherein said estimating and said deriving are carried out on frequency-domain signals.

20. The apparatus as claimed in claim 14, wherein said output signal consists of two output channels.