Audio mixing based upon playing device location

• US 10,038,957 B2
• Filed: 03/19/2013
• Issued: 07/31/2018
• Est. Priority Date: 03/19/2013
• Status: Active Grant

• Current Assignee: Nokia Technologies Oy (Nokia Corporation)
• Original Assignee: Nokia Technologies Oy (Nokia Corporation)
• Inventors: Laaksonen, Lasse Juhani, Ali-Yrkko, Olli, Hagqvist, Jari
• Primary Examiner(s): Pham, Quang D
• Application Number:
  US13/847,158
• Publication Number:
  US 20140285312A1
• Time in Patent Office: 1,960 Days
• CPC Class Codes: H04R 2420/07 Applications of wireless loudspeakers or wireless microphones
• CPC Class Codes: H04R 2499/11 Transducers incorporated or for use in hand-held devices, e.g. mobile phones, PDA's, camera's
• CPC Class Codes: H04R 3/12 for distributing signals to two or more loudspeakers specially adapted for hearing aids H04R25/407
• CPC Class Codes: H04R 5/02 Spatial or constructional arrangements of loudspeakers
• CPC Class Codes: H04S 7/302 Electronic adaptation of stereophonic sound system to listener position or orientation H04S7/301 takes precedence

Abstract

A method including determining location of at least one second device relative to a first device, where at least two of the devices are configured to play audio sounds based upon audio signals; and mixing at least two of the audio signals based, at least partially, upon the determined location(s).

Claims

1. A method for creating a spatial audio mix of audio objects at a first electronic device based on locations of the first electronic device and at least one second electronic device, the method comprising:

assigning, by the first electronic device wirelessly connected to the at least one second electronic device via a short-range communications system, at least one of the audio objects to each of the first electronic device and the at least one second electronic device;

initiating, by the first electronic device, mixing of the assigned audio objects;

determining, by the first electronic device, a location of the at least one second electronic device relative to the first electronic device;

mixing, by the first electronic device, the assigned audio objects to create the spatial audio mix based on the determined relative location of the at least one second electronic device and the first electronic device, wherein the mixing comprises defining a spatial location of each of the audio objects based on the determined relative location of the at least one second electronic device and the first electronic device and at least one new determined relative location resulting from a movement of at least one of the first electronic device and the at least one second electronic device;

updating, by the first electronic device, the spatial location of each of the assigned audio objects in real time based upon the at least one new determined relative location while the mixing is being performed;

rendering, by the first electronic device, the spatial audio mix while the mixing is being performed; and

in response to a user input to end the mixing, creating a file comprising the spatial audio mix.

2. The method as in claim 1 further comprising coupling the first electronic device and the at least one second electronic device by a short range wireless link of the short range communication system, and wherein at least one of the assigned audio objects is shared by the first electronic device and the at least one second electronic device.

3. The method as in claim 1 further comprising using a user interface on the first electronic device to adjust an output of the audio object assigned to the at least one second electronic device.

4. The method according to claim 1, wherein the mixing comprises synchronizing the first electronic device and the at least one second electronic device to start playback of the assigned audio objects when the mixing is initiated.

5. The method according to claim 1, wherein mixing the spatial audio mix comprises at least one of:

adjusting a sound level of at least one of the audio objects; and

adding a sound effect to at least one of the audio objects.

6. The method according to claim 1, wherein the spatial audio mix comprises a nested spatial audio mix such that at least one of the assigned audio objects comprises a combined representation of a plurality of audio objects.

7. The method as in claim 1, wherein the first electronic device and the at least one second electronic device are mobile electronic devices.

8. A non-transitory program storage device readable by a machine, tangibly embodying a program of instructions executable by the machine for performing operations for creating a spatial audio mix of audio objects at a first electronic device based on locations of the first electronic device and at least one second electronic device, the operations comprising:

assigning, by the first electronic device wirelessly connected to the at least one second electronic device via a short-range communications system, at least one of the audio objects to each of the first electronic device and the at least one second electronic device;

initiating, by the first electronic device, mixing of the assigned audio objects;

determining, by the first electronic device, a location of the at least one second electronic device relative to the first electronic device;

mixing, by the first electronic device, the assigned audio objects to create the spatial audio mix based on the determined relative location of the at least one second electronic device and the first electronic device, wherein the mixing comprises defining a spatial location of each of the audio objects based on the determined relative location of the at least one second electronic device and the first electronic device and at least one new determined relative location resulting from a movement of at least one of the first electronic device and the at least one second electronic device;

updating, by the first electronic device, the spatial location of each of the assigned audio objects in real time based upon the at least one new determined relative location while the mixing is being performed;

rendering, by the first electronic device, the spatial audio mix while the mixing is being performed; and

in response to a user input to end the mixing, creating a file comprising the spatial audio mix.

9. The non-transitory program storage device as in claim 8, wherein the operations further comprise:

coupling the electronic device and the at least one second electronic device by a short range wireless link of the short range communication system, and wherein at least one of the assigned audio objects is shared by the first electronic device and the at least one second electronic device.

10. The non-transitory program storage device as in claim 8, wherein the mixing comprises synchronizing the first electronic device and the at least one second electronic device to start playback of the assigned audio objects when the mixing is initiated.

11. The non-transitory program storage device as in claim 8, wherein mixing the spatial audio mix comprises at least one of:

adjusting a sound level of at least one of the audio objects; and

adding a sound effect to at least one of the audio objects.

12. The non-transitory program storage device as in claim 8, wherein the spatial audio mix comprises a nested spatial audio mix such that at least one of the assigned audio objects comprises a combined representation of a plurality of audio objects.

13. The non-transitory program storage device as in claim 8, wherein the first electronic device and the at least one second electronic device are mobile electronic devices.

14. An electronic device for creating a spatial audio mix of audio objects at the electronic device based on locations of the electronic device and at least one other electronic device, wherein the electronic device is wirelessly connected to the at least one other electronic device via a short-range communications system, the electronic device comprising:

at least one processor, and

at least one non-transitory memory comprising computer program code, the at least one memory and the computer program code configured to, with the at least one processor, cause the electronic device at least to;

assign at least one of the audio objects to each of the electronic device and the at least one other electronic device;

initiate mixing of the assigned audio objects;

determine a location of the at least one other electronic device relative to the electronic device;

mix the assigned audio objects to create the spatial audio mix based on the determined relative locations of the at least one other electronic device and the electronic device, wherein the mix comprises definition of a spatial location of each of the audio objects based on the determined relative location of the at least one other electronic device and the electronic device and at least one new determined relative location resulting from a movement of at least one of the electronic device and the at least other electronic device;

update the spatial location of each of the assigned audio objects in real time based upon the at least one new determined relative location while the mixing is being performed;

render the spatial audio mix while the mixing is being performed; and

in response to a user input to end the mixing, creating a file comprising the spatial audio mix.

15. The electronic device as in claim 14 where the electronic device is configured to couple the electronic device and the at least one other electronic device by the short-range communication system, and wherein at least one of the assigned audio objects is shared by the at least one other electronic device and the electronic device.

16. The electronic device as in claim 14 where the electronic device is configured to mix the assigned audio objects based upon at least one of:

a movement of the at least one other electronic device relative to the electronic device; and

a relative movement of the electronic device and the at least one other electronic device relative to each other.

17. The electronic device as in claim 14, wherein the electronic device and the at least one other electronic device are mobile electronic devices.