Focusing on a portion of an audio scene for an audio signal

• US 8,509,454 B2
• Filed: 11/01/2007
• Issued: 08/13/2013
• Est. Priority Date: 11/01/2007
• Status: Active Grant

• Current Assignee: Piece Future Pte., Ltd.
• Original Assignee: Nokia Corporation
• Inventors: Kirkeby, Ole, Virolainen, Jussi
• Primary Examiner(s): Jamal, Alexander
• Application Number:
  US11/933,638
• Publication Number:
  US 20090116652A1
• Time in Patent Office: 2,112 Days
• Field of Search: 381/310, 381/17, 381/106, 381/309, 381/1, 381/92, 700/94, 379/202.01, 348/14.8
• US Class Current: 381/92
• CPC Class Codes: H04S 2400/11 Positioning of individual sound objects, e.g. moving airplane, within a sound field H04S2420/13 takes precedence
• CPC Class Codes: H04S 2400/15 Aspects of sound capture and related signal processing for recording or reproduction
• CPC Class Codes: H04S 2420/01 Enhancing the perception of the sound image or of the spatial distribution using head related transfer functions [HRTF's] or equivalents thereof, e.g. interaural time difference [ITD] or interaural level difference [ILD]
• CPC Class Codes: H04S 2420/11 Application of ambisonics in stereophonic audio systems
• CPC Class Codes: H04S 3/00 Systems employing more than two channels, e.g. quadraphonic H04S5/00, H04S7/00 take precedence
• CPC Class Codes: H04S 7/303 Tracking of listener position or orientation

Abstract

Aspects of the invention provide methods, computer-readable media, and apparatuses for spatially manipulating sound that is played back to a listener over a set of output transducers, e.g., headphones. The listener can direct spatial attention to focus on a portion of an audio scene, analogous to a magnifying glass being used to pick out details in a picture. An input multi-channel audio signal that is generated by audio sources is obtained, and directional information is determined for each of the audio sources. The user provides a desired direction of spatial attention so that audio processing can focus on the desired direction and render a corresponding multi-channel audio signal to the user. A region of an audio scene is expanded around the desired direction while the audio scene is compressed in another portion of the audio scene.

Claims

1. A method comprising:

obtaining a single-channel or multi-channel input audio signal having a plurality of audio sources;

generating, with a processor, a single-channel or multi-channel output audio signal including the plurality of audio sources spatialized in a sound field;

obtaining at least one parameter from a user controlled input device, the at least one parameter indicating at least one direction of spatial attention in the sound field; and

expanding a first region of the sound field around the indicated at least one direction of spatial attention to produce a modified sound field in the output audio signal to a user, wherein the expanding includes;

for any sound source in the first region and not centered in the indicated at least one direction of spatial attention in the sound field, moving the sound source in the modified sound field according to a re-mapping function; and

for any sound source centered in the indicated at least one direction of spatial attention in the sound field, maintaining the sound source centered in the indicated at least one direction of spatial attention in the modified sound field.

2. The method of claim 1, further comprising:

compressing a second region of the sound field to produce the modified sound field.

3. The method of claim 1, further comprising:

re-mapping an azimuth value of each sound source in the first region and not centered in the indicated at least one direction of spatial attention to a new azimuth value in the modified sound field.

4. The method of claim 1, further comprising:

utilizing a remapping function to re-map each azimuth value, wherein the re-mapping function is characterized by a non-linearity and has a derivative greater than one for a range of possible new azimuth values.

5. The method of claim 1, further comprising:

preserving an overall loudness of the plurality of audio sources when moving one or more of the plurality of audio sources in the modified sound field of the output audio signal.

6. The method of claim 1, further comprising:

amplifying one or more of the audio sources positioned within the first region of the sound field.

7. The method of claim 1, the output audio signal comprising a binaural audio signal.

8. The method of claim 1, further comprising:

obtaining the at least one parameter indicating the at least one direction of spatial attention from a headtracker configured to be fastened to the user.

18. The method of claim 1, further comprising:

determining directional information for each of the plurality of audio sources in the input audio signal; and

generating the output audio signal by positioning the plurality of audio sources in the sound field based on the directional information.

20. The method of claim 1, further comprising:

rendering the output audio signal from one or more speakers.

22. The method of claim 1, further comprising:

for any sound source in the first region and not centered in the indicated at least one direction of spatial attention in the sound field, moving the sound source in the modified sound field away from the at least one direction of spatial attention according to the re-mapping function.

9. An apparatus comprising:

at least one processor;

and memory having computer executable instructions stored thereon, that when executed, cause the apparatus to;

obtain a single-channel or multi-channel input audio signal having a plurality of audio sources;

generate a single-channel or multi-channel output audio signal including the plurality of audio sources spatialized in a sound field;

obtain at least one parameter from a user controlled input device, the at least one parameter indicating at least one direction of spatial attention in the sound field; and

expand a first region of the sound field around the indicated at least one direction of spatial attention to produce a modified sound field in the output audio signal to a user, wherein the expanding includes;

for any sound source in the first region and not centered in the indicated at least one direction of spatial attention in the sound field, move the sound source in the modified sound field according to a re-mapping function; and

for any sound source centered in the indicated at least one direction of spatial attention in the sound field, maintain the sound source centered in the indicated at least one direction of spatial attention in the modified sound field.

10. The apparatus of claim 9, wherein the computer executable instructions, when executed, cause the apparatus to:

compress a second region of the sound field to produce the modified sound field.

11. The apparatus of claim 9, wherein the computer executable instructions, when executed, cause the apparatus to:

re-map an azimuth value of each sound source in the first region and not centered in the indicated at least one direction of spatial attention to a new azimuth value in the modified sound field.

12. The apparatus of claim 11, wherein the computer executable instructions, when executed, cause the apparatus to:

utilize a re-mapping function to re-map each azimuth value, wherein the re-mapping function is characterized by a non-linearity and has a derivative greater than one for a range of possible new azimuth values.

19. The apparatus of claim 9, wherein the computer executable instructions, when executed, cause the apparatus to:

determine directional information for each of the plurality of audio sources in the input audio signal; and

generate the output audio signal by positioning the plurality of audio sources in the sound field based on directional information.

21. The apparatus of claim 9, wherein the computer executable instructions, when executed, cause the apparatus to:

render the output audio signal from one or more speakers.

13. A method comprising:

obtaining a single-channel or multi-channel input audio signal having a plurality of audio sources;

generating a single-channel or multi-channel output audio signal including the plurality of audio sources spatialized in a sound field;

obtaining at least one parameter from a user control led in gut device, the at least one parameter indicating at least one direction of spatial attention in the sound field; and

expanding a first region of the sound field around the indicated at least one direction of spatial attention to produce a modified sound field in the output audio signal to a user, wherein the expanding includes;

for each sound source in the first region, modifying in the modified sound field an azimuth angle between the sound source and the indicated at least one direction of spatial attention according to a re-mapping function having a non-zero derivative at the indicated at least one direction of spatial attention.

14. The method of claim 13, further comprising:

compressing a second region of the sound field to produce the modified sound field.

15. The method of claim 13, wherein the re-mapping function is characterized by a non-linearity and has a derivative greater than one for a range of possible new azimuth values.

16. An apparatus comprising:

at least one processor;

and memory having computer executable instructions stored thereon, that when executed, cause the apparatus to;

obtain a single-channel or multi-channel audio signal having a plurality of audio sources;

generate a single-channel or multi-channel output audio signal including the plurality of audio sources spatialized in a sound field;

obtain at least one parameter from a user controlled input device, the at least one parameter indicating at least one direction of spatial attention in the sound field; and

expand a first region of the sound field around the indicated at least one direction of spatial attention to produce a modified sound field in the output audio signal to a user, wherein the expanding includes;

for each sound source in the first region, modifying in the modified sound field an azimuth angle between the sound source and the indicated at least one direction of spatial attention according to a re-mapping function having a non-zero derivative at the indicated at least one direction of spatial attention.

17. The apparatus of claim 16, wherein the computer executable instructions, when executed, cause the apparatus to:

compress a second region of the sound field to produce the modified sound field.