Synchronized spatial audio presentation

• US 10,694,311 B2
• Filed: 03/15/2018
• Issued: 06/23/2020
• Est. Priority Date: 03/15/2018
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Tomlin, Arthur Charles, York, Kendall Clark, Sipko, Jeffrey, Hernandez Santisteban, Adolfo, Krauss, Aaron Daniel, Muehlhausen, Andrew Frederick
• Primary Examiner(s): Fischer, Mark
• Application Number:
  US15/922,791
• Publication Number:
  US 20190289417A1
• Time in Patent Office: 831 Days
• CPC Class Codes: G06F 3/165 Management of the audio stream, e.g. setting of volume, audio stream path
• CPC Class Codes: H04S 2400/01 Multi-channel, i.e. more than two input channels, sound reproduction with two speakers wherein the multi-channel information is substantially preserved
• CPC Class Codes: H04S 2400/11 Positioning of individual sound objects, e.g. moving airplane, within a sound field H04S2420/13 takes precedence
• CPC Class Codes: H04S 2420/01 Enhancing the perception of the sound image or of the spatial distribution using head related transfer functions [HRTF's] or equivalents thereof, e.g. interaural time difference [ITD] or interaural level difference [ILD]
• CPC Class Codes: H04S 3/008 in which the audio signals are in digital form, i.e. employing more than two discrete digital channels data reduction aspects thereof based on psychoacoustics G10L19/02
• CPC Class Codes: H04S 7/303 Tracking of listener position or orientation

Abstract

Examples are disclosed relating to providing spatialized audio to multiple users. In one example, a computing device presents spatialized audio to multiple users within an environment via communicative connection to one or more wearable spatial audio output devices. For each communicatively connected wearable spatial audio output device, a user-specific subset of audio tracks is generated from a set of audio tracks for a dynamic audio object positioned within the environment based on one or more user-specific parameters. A location of the wearable spatial audio output device is determined relative to the dynamic audio object, and based upon this location, a device-specific spatialized audio mix is generated that includes the user-specific subset of audio tracks. The device-specific spatialized audio mixes are sent to the wearable spatial output devices, and playback of the device-specific spatialized audio mixes are synchronously initiated at each wearable spatial audio output device.

Claims

1. A computing device for presenting spatialized audio to two or more users within an environment, comprising:

a logic machine; and

a storage machine comprising instructions executable by the logic machine to;

communicatively connect to one or more wearable spatial audio output devices, and

for each wearable spatial audio output device communicatively connected to the computing device,

generate, from a set of audio tracks for a dynamic audio object positioned within the environment, a user-specific subset of audio tracks based on one or more user-specific parameters,

determine a location of the wearable spatial audio output device relative to the dynamic audio object positioned within the environment,

generate, based upon the location of the wearable spatial audio output device relative to the dynamic audio object positioned within the environment, a device-specific spatialized audio mix including the user-specific subset of audio tracks,

send the device-specific spatialized audio mix to the wearable spatial output audio device, and

synchronously signal to initiate playback of the device-specific spatialized audio mixes at each wearable spatial audio output device.

2. The computing device of claim 1, wherein the computing device comprises a wearable spatial audio output device.

3. The computing device of claim 1, wherein the user-specific parameters comprise a preferred language.

4. The computing device of claim 1, wherein the user-specific subset of audio tracks includes time-stamped audio tracks.

5. The computing device of claim 1, wherein the storage machine further comprises instructions executable by the logic machine to:

update the device-specific spatialized audio mix based on determining a change in location of the wearable spatial audio output device relative to the dynamic audio object positioned within the environment.

6. The computing device of claim 1, wherein the storage machine further comprises instructions executable by the logic machine to:

update the device-specific spatialized audio mix based on recognizing a change in a condition within the environment.

7. The computing device of claim 1, wherein the storage machine further comprises instructions executable by the logic machine to:

update the device-specific spatialized audio mix based on determining a change in location of one or more other wearable spatial audio output devices relative to the dynamic audio object positioned within the environment.

8. The computing device of claim 1, wherein the dynamic audio object is a virtual object encoded as instructions stored in the storage machine that are readable by the logic machine.

9. A wearable spatial audio output device, comprising:

one or more audio speakers;

a logic machine; and

a storage machine holding instructions executable by the logic machine to;

obtain shared map data for a real-world environment, the shared map data shared with one or more additional wearable spatial audio output devices within the real- world environment,

receive location information for a plurality of dynamic audio objects positioned at predetermined locations in the real-world environment based on the shared map data, each dynamic audio object associated with a set of audio tracks,

for a selected dynamic audio object, receive a user-specific subset of audio tracks based on one or more user-specific parameters, and

generate, based on the shared map data, a device-specific spatialized audio mix including the user-specific subset of audio tracks for synchronous playback at each wearable spatial audio output device.

10. The wearable spatial audio output device of claim 9, wherein the shared map data is generated based on data received from two or more wearable spatial audio output devices.

11. The wearable spatial audio output device of claim 9, wherein the shared map data is generated based on data received from one or more sensor systems external to the wearable spatial audio output device.

12. The wearable spatial audio output device of claim 9, wherein the user-specific subset of audio tracks is further based on the shared map data.

13. The wearable spatial audio output device of claim 9, wherein the storage machine further holds instructions executable by the logic machine to

receive updated map data at the wearable spatial audio output device;

generate updated shared map data based on the received updated map data; and

update the user-specific subset of audio tracks based on the updated shared map data.

14. The wearable spatial audio output device of claim 13, wherein the storage machine further holds instructions executable by the logic machine to update the device-specific spatialized audio mix based on the updated shared map data.

15. The wearable spatial audio output device of claim 9, wherein the user-specific subset of audio tracks includes time-stamped audio tracks.

16. A method for presenting spatialized audio to two or more users within a real-world environment, comprising:

generating map data for the real-world environment;

assigning a plurality of dynamic audio objects to predetermined locations in the real-world environment based on the generated map data, each dynamic audio object associated with a set of audio tracks;

detecting the location of one or more wearable spatial audio output devices within the real-world environment; and

for each detected wearable spatial audio output device,

determining a position of the wearable spatial audio output device relative to one or more dynamic audio objects;

for a selected dynamic audio object, selecting audio tracks for a user-specific subset of audio tracks based on the position of the wearable spatial audio output device relative to the selected dynamic audio object; and

generating a device-specific spatialized audio mix including the user-specific subset of audio tracks based on the position of the wearable spatial audio output device relative to the selected dynamic audio object for playback of the device-specific spatialized audio mix.

17. The method of claim 16, wherein the set of audio tracks comprises audio tracks shared between wearable spatial audio output devices.

18. The method of claim 16, wherein the set of audio tracks comprises audio tracks that are included in each user-specific subset of audio tracks regardless of the position of the wearable spatial audio output device.

19. The method of claim 16, wherein initiating playback of the device-specific spatialized audio mix comprises initiating playback of the device-specific spatialized audio mix responsive to a position of the wearable spatial audio output device relative to the selected dynamic audio object.

20. The method of claim 16, wherein the plurality of dynamic audio objects comprise dynamic audio objects associated with live, scheduled events.