Distributed audio mixing

• US 10,448,186 B2
• Filed: 12/08/2017
• Issued: 10/15/2019
• Est. Priority Date: 12/14/2016
• Status: Active Grant

• Current Assignee: Nokia Technologies Oy (Nokia Corporation)
• Original Assignee: Nokia Technologies Oy (Nokia Corporation)
• Inventors: Lehtiniemi, Arto, Eronen, Antti, Leppanen, Jussi, Arrasvuori, Juha
• Primary Examiner(s): Blair, Kile O
• Application Number:
  US15/835,790
• Publication Number:
  US 20180167755A1
• Time in Patent Office: 676 Days
• CPC Class Codes: G10L 19/008 Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing
• CPC Class Codes: H04H 60/04 Studio equipment; Interconnection of studios
• CPC Class Codes: H04S 2400/11 Positioning of individual sound objects, e.g. moving airplane, within a sound field H04S2420/13 takes precedence
• CPC Class Codes: H04S 2400/13 Aspects of volume control, not necessarily automatic, in stereophonic sound systems
• CPC Class Codes: H04S 2400/15 Aspects of sound capture and related signal processing for recording or reproduction
• CPC Class Codes: H04S 3/008 in which the audio signals are in digital form, i.e. employing more than two discrete digital channels data reduction aspects thereof based on psychoacoustics G10L19/02
• CPC Class Codes: H04S 7/30 Control circuits for electronic adaptation of the sound field
• CPC Class Codes: H04S 7/304 For headphones
• CPC Class Codes: H04S 7/305 Electronic adaptation of stereophonic audio signals to reverberation of the listening space H04S7/301 takes precedence
• CPC Class Codes: H04S 7/40 Visual indication of stereophonic sound image

Abstract

Systems and methods for distributed audio mixing are disclosed, comprising providing one or more predefined constellations, each constellation defining a spatial arrangement of points forming a shape or pattern and receiving positional data indicative of the spatial positions of a plurality of audio sources in a capture space. A correspondence may be identified between a subset of the audio sources and a constellation based on the relative spatial positions of audio sources in the subset. Responsive to said correspondence, at least one action may be applied, for example an audio, video and/or controlling action to audio sources of the subset.

Claims

1. A method comprising:

providing one or more predefined constellations, each constellation defining a spatial arrangement of points forming a shape or pattern;

receiving positional data indicative of spatial positions of a plurality of audio sources in a capture space;

identifying a correspondence between a subset of the audio sources and a constellation based on relative spatial positions of audio sources in the subset, where identifying the correspondence comprises at least comparing a shape or pattern formed with the spatial positions of the audio sources of the subset with the constellation; and

responsive to said correspondence, applying at least one action.

2. The method of claim 1, wherein the at least one action is applied to selected ones of the audio sources.

3. The method of claim 1, wherein the action applied is one or more of:

an audio action,

a visual action, or

a controlling action.

4. The method of claim 3, wherein the audio action is applied to audio signals of selected audio sources, comprising one or more of:

reducing the audio volume,

muting the audio volume,

increasing the audio volume,

distortion, or

reverberation.

5. The method of claim 3, wherein the controlling action is applied to control at least one of:

the spatial position(s) of selected audio source(s);

or

movement of one or more capture devices in the capture space.

6. The method of claim 1, wherein each constellation defines one or more of a line, arc, circle, cross or polygon.

7. The method of claim 1, wherein the positional data is derived from positioning tags, where the audio sources carry the positioning tags in the capture space.

8. The method of claim 1, wherein the correspondence is identified if the relative spatial positions of the audio sources in the subset comprise substantially the same shape or pattern of the constellation, or deviate therefrom no more than a predetermined distance.

9. The method of claim 1, wherein definition of each constellation comprises receiving, through a user interface, a user-defined spatial arrangement of points forming a shape or pattern.

10. An apparatus having at least one processor and at least one non-transitory memory having computer-readable code stored thereon which when executed controls the apparatus to:

provide one or more predefined constellations, each constellation defining a spatial arrangement of points forming a shape or pattern;

receive positional data indicative of spatial positions of a plurality of audio sources in a capture space;

identify a correspondence between a subset of the audio sources and a constellation based on relative spatial positions of audio sources in the subset, where identification of the correspondence comprises at least comparison of a shape or pattern formed with the spatial positions of the audio sources of the subset with the constellation; and

responsive to said correspondence, apply at least one action.

11. The apparatus of claim 10, wherein the at least one action is applied to selected ones of the audio sources.

12. The apparatus of claim 11, wherein definition of each constellation comprises means of receiving, through a user interface, a user-defined spatial arrangement of points forming a shape or pattern.

13. The apparatus of claim 10, wherein the action applied is one or more of:

an audio action,

a visual action, or

a controlling action.

14. The apparatus of claim 13, wherein the audio action is applied to audio signals of selected audio sources, comprising one or more of:

reducing the audio volume,

muting the audio volume,

increasing the audio volume,

distortion, or

reverberation.

15. The apparatus of claim 13, wherein the controlling action is applied to control the spatial position(s) of selected audio source(s).

16. The apparatus of claim 13, wherein the controlling action is applied to control movement of one or more capture devices in the capture space.

17. The apparatus of claim 13, wherein each constellation defines one or more of a line, arc, circle, cross or polygon.

18. The apparatus of claim 13, wherein the positional data is derived from positioning tags, where the audio sources carry the positioning tags in the capture space.

19. The apparatus of claim 13, wherein the correspondence is identified if the relative spatial positions of the audio sources in the subset have substantially the same shape or pattern of the constellation, or deviate therefrom no more than a predetermined distance.

20. A non-transitory computer-readable storage medium having stored thereon computer-readable code, which, when at least one processor executes the computer-readable code, causes the at least one processor to perform:

providing one or more predefined constellations, each constellation defining a spatial arrangement of points forming a shape or pattern;

receiving positional data indicative of spatial positions of a plurality of audio sources in a capture space;

identifying a correspondence between a subset of the audio sources and a constellation based on relative spatial positions of audio sources in the subset, where identifying the correspondence comprises at least comparing a shape or pattern formed with the spatial positions of the audio sources of the subset with the constellation; and

responsive to said correspondence, applying at least one action.