Spatial audio processing apparatus

• US 10,154,361 B2
• Filed: 12/22/2011
• Issued: 12/11/2018
• Est. Priority Date: 12/22/2011
• Status: Active Grant

• Current Assignee: Nokia Technologies Oy (Nokia Corporation)
• Original Assignee: Nokia Technologies Oy (Nokia Corporation)
• Inventors: Tammi, Mikko, Vilermo, Miikka, Ugur, Kemal
• Primary Examiner(s): Patel, Yogeshkumar
• Application Number:
  US14/367,912
• Publication Number:
  US 20150139426A1
• Time in Patent Office: 2,546 Days
• Field of Search: 381 1
• CPC Class Codes: H04R 2201/401 2D or 3D arrays of transducers
• CPC Class Codes: H04R 2430/23 Direction finding using a sum-delay beam-former
• CPC Class Codes: H04R 29/005 Microphone arrays
• CPC Class Codes: H04R 3/005 for combining the signals of two or more microphones specially adapted for hearing aids H04R25/407
• CPC Class Codes: H04S 7/30 Control circuits for electronic adaptation of the sound field

Abstract

An apparatus comprising: a directional analyser configured to determine a directional component of at least two audio signals; an estimator configured to determine at least one virtual position or direction relative to the actual position of the apparatus; and a signal generator configured to generate at least one further audio signal dependent on the at least one virtual position or direction relative to the actual position of the apparatus and the directional component of at least two audio signals.

Claims

1. An apparatus comprising at least one processor and at least one memory including computer code for one or more programs, the at least one memory and the computer code configured to with the at least one processor cause the apparatus at least to:

determine a direction of each of at least two audio sources based on at least three microphone audio signals, wherein a respective direction of each audio source is relative to an actual position of a recording device comprising the at least three microphone audio signals;

cause to display a visual image for each audio source;

receive an indication to select a first audio source of the at least two audio sources based on the displayed visual image for each audio source;

generate a modified version of the selected first audio source, wherein the selected first audio source is modified at its determined direction with respect to a second audio source of the at least two audio sources;

process the at least three microphone audio signals based on the modified version of the selected first audio source and the second audio source, wherein the second audio source is unmodified; and

reproduce the processed at least three microphone audio signals in playback.

2. The apparatus as claimed in claim 1, wherein determining the direction of each of the at least two audio sources based on the at least three microphone audio signals further comprises providing a directional analysis using the at least three microphone audio signals.

3. The apparatus as claimed in claim 2, wherein providing the directional analysis comprises:

dividing the at least three microphone audio signals into frequency bands; and

performing the directional analysis based on the frequency bands.

4. The apparatus as claimed in claim 2, wherein providing the directional analysis further comprises determining an ambient sound signal associated with the at least two audio sources.

5. The apparatus as claimed in claim 1, wherein processing the at least three microphone audio signals further comprises generating at least one further audio signal based on the received indication.

6. The apparatus as claimed in claim 5, wherein generating the at least one further audio signal comprises one of:

a multichannel audio signal;

at least one of the at least two audio sources;

at least one of the at least two audio sources with the determined direction;

or

an ambient audio signal associated with at least one of the at least two audio sources.

7. The apparatus as claimed in claim 5, wherein generating the at least one further audio signal further comprises generating a spatial filter;

and

applying the spatial filter to at least one of the at least three microphone audio signals to modify a spatial audio field of at least one of the at least two audio sources by the at least three microphone audio signals.

8. The apparatus as claimed in claim 7, wherein generating the spatial filter comprises at least one of:

determining the spatial filter is dependent on a user input;

determining the spatial filter is dependent on a position of each audio source of the visual image;

or

determining the spatial filter is dependent on a recognized position of at least one of the at least two audio sources.

9. The apparatus as claimed in claim 1, wherein the apparatus is further caused to determine a position of each audio source of the visual image for at least one of the at least two audio sources relative to an actual position of the recording device based on the determined direction of each of the at least two audio sources.

10. The apparatus as claimed in claim 1, wherein a virtual position associated with each audio source of the displayed visual image is modified based on the received indication.

11. The apparatus as claimed in claim 1, wherein the selected first audio source is modified based on the received indication by changing a sound parameter of the first audio source.

12. The apparatus as claimed in claim 1, wherein causing to display the visual image comprises:

determining a virtual position associated with the visual image associated with each audio source;

causing to display the virtual position associated with the visual image which is an actual position of each of the at least two audio sources; and

receiving a user input to modify the virtual position of the visual image.

13. The apparatus as claimed in claim 12, wherein processing the at least three microphone audio signals further comprises modifying a sound parameter of at least one of the at least two audio sources based on the received user input, wherein the modified sound parameter virtually changes the actual position of the at least one of the at least two audio sources so as to match the virtual position of the at least one of the at least two audio sources to the modified virtual position of the visual image provided by the user input.

14. The apparatus as claimed in claim 1, wherein the apparatus is further caused to generate a first audio signal of the at least three microphone audio signals from a first microphone located at a first position in the recording device and a second audio signal of the at least three microphone audio signals from a second microphone located at a second position in the recording device.

15. The apparatus as claimed in claim 1, wherein processing the at least three microphone audio signals further comprises:

amplifying at least one of the at least two audio sources by processing at least one audio signal of the at least three microphone audio signals; and

attenuating at least one of the at least two audio sources by processing the at least one audio signal of the at least three microphone audio signals.

16. The apparatus as claimed in claim 1, wherein the apparatus comprises:

an estimator configured to determine the direction of each of the at least two audio sources relative to the actual position of the recording device; and

a signal generator configured to generate at least one further audio signal associated with at least one of the at least two audio sources based on the at least three microphone audio signals, wherein the at least one further audio signal is processed based on the received indication.

17. A method comprising:

determining a direction of each of at least two audio sources based on at least three microphone audio signals, wherein a respective direction of each audio source is relative to an actual position of a recording device comprising the at least three microphone audio signals;

causing to display a visual image for each audio source;

receiving an indication to select a first audio source of the at least two audio sources based on the displayed visual image for each audio source;

generating a modified version of the selected first audio source, wherein the selected first audio source is modified at its determined direction with respect to a second audio source of the at least two audio sources;

processing the at least three microphone audio signals based on the modified version of the selected first audio source and the second audio source, wherein the second audio source is unmodified; and

reproducing the processed at least three microphone audio signals in playback.

18. The method as claimed in claim 17, wherein the processing

the at least three microphone audio signals further comprises one of:

amplifying at least one of the at least two audio sources by processing at least one of the at least three microphone audio signals;

or

attenuating at least one of the at least two audio sources by processing at least one of the at least three microphone audio signals.

19. The method as claimed in claim 17, the method further comprising:

determining a virtual position associated with each audio source of the visual image associated with each of the at least two audio sources;

displaying the virtual position associated with each audio source of the visual image which is an actual position of each audio source of the at least two audio sources relative to the recording device; and

receiving a user input for modifying the virtual position of each audio source of the visual image.

20. The method as claimed in claim 19, wherein processing the at least three microphone audio signals further comprises modifying a sound parameter of at least one of the at least two audio sources based on an input for virtually changing an actual position of at least one of the at least two audio sources to match the virtual position of at least one of the at least two audio sources to the changed position of the at least one of the at least two audio sources of the visual image.