Remote multi-dimensional audio

• US 10,674,305 B2
• Filed: 03/15/2018
• Issued: 06/02/2020
• Est. Priority Date: 03/15/2018
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: York, Kendall Clark, Sipko, Jeffrey, Krauss, Aaron, Muehlhausen, Andrew F., Hernandez Santisteban, Adolfo, Tomlin, Arthur C.
• Primary Examiner(s): Flanders, Andrew C
• Application Number:
  US15/922,515
• Publication Number:
  US 20190289416A1
• Time in Patent Office: 810 Days
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 3/165 Management of the audio stream, e.g. setting of volume, audio stream path
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: H01Q 1/273 Adaptation for carrying or wearing by persons or animals
• CPC Class Codes: H04B 1/385 Transceivers carried on the body, e.g. in helmets
• CPC Class Codes: H04R 1/323 for loudspeakers H04R1/34 and H04R1/40 take precedence
• CPC Class Codes: H04R 1/403 loud-speakers
• CPC Class Codes: H04R 2420/07 Applications of wireless loudspeakers or wireless microphones
• CPC Class Codes: H04R 2460/07 Use of position data from wide-area or local-area positioning systems in hearing devices, e.g. program or information selection
• CPC Class Codes: H04R 27/00 Public address systems circuits for preventing acoustic reaction H04R3/02; circuits for distributing signals to loudspeakers H04R3/12; monitoring or testing arrangements for public address systems H04R29/007; amplifiers H03F
• CPC Class Codes: H04R 3/005 for combining the signals of two or more microphones specially adapted for hearing aids H04R25/407
• CPC Class Codes: H04R 3/12 for distributing signals to two or more loudspeakers specially adapted for hearing aids H04R25/407
• CPC Class Codes: H04R 5/02 Spatial or constructional arrangements of loudspeakers
• CPC Class Codes: H04R 5/04 Circuit arrangements, e.g. for selective connection of amplifier inputs/outputs to loudspeakers, for loudspeaker detection, or for adaptation of settings to personal preferences or hearing impairments combinations of amplifiers H03F3/68; stereophonic systems H04S
• CPC Class Codes: H04S 2400/01 Multi-channel, i.e. more than two input channels, sound reproduction with two speakers wherein the multi-channel information is substantially preserved
• CPC Class Codes: H04S 2400/11 Positioning of individual sound objects, e.g. moving airplane, within a sound field H04S2420/13 takes precedence
• CPC Class Codes: H04S 7/303 Tracking of listener position or orientation
• CPC Class Codes: H04S 7/304 For headphones
• CPC Class Codes: H04S 7/308 : Electronic adaptation dependent on speaker or headphone connection
• CPC Class Codes: H04W 4/02 : Services making use of location information
• CPC Class Codes: H04W 4/023 : using mutual or relative location information between multiple location based services [LBS] targets or of distance thresholds
• CPC Class Codes: H04W 4/026 : using orientation information, e.g. compass

Abstract

The disclosed technology provides multi-dimensional audio output by providing a relative physical location of an audio transmitting device relative to an audio outputting device in a shared map of physical space shared between the audio transmitting device and the audio outputting device. An orientation of the audio outputting device relative to the audio transmitting device is determined and an audio signal received from the audio transmitting device via a communication network is processed using the determined orientation of the audio outputting device relative to the audio transmitting device and the relative physical location of the audio transmitting device to create an augmented audio signal. The augmented audio signal is output through at least one audio output on the audio outputting device in a manner indicating a relative physical direction of the audio transmitting device to the audio outputting device in the shared map of the physical space.

Claims

1. An audio outputting device configured to execute a process comprising:

determining a relative physical location of an audio transmitting device relative to the audio outputting device in a shared map of a physical space, the shared map being shared between the audio transmitting device and the audio outputting device;

determining an orientation of an axis of the audio outputting device relative to the audio transmitting device;

processing an audio signal received from the audio transmitting device via a communication network using the determined orientation of the axis of the audio outputting device relative to the audio transmitting device and the relative physical location of the audio transmitting device to create an augmented audio signal; and

outputting the augmented audio signal through at least one audio output on the audio outputting device, the augmented audio signal being output in a manner that indicates a relative physical direction of the audio transmitting device to the audio outputting device in the shared map of the physical space.

2. The audio outputting device of claim 1, wherein the augmented audio signal is output in a manner that further indicates a three-dimensional relative physical direction of the audio transmitting device to the audio outputting device in the shared map of the physical space.

3. The audio outputting device of claim 1, the process further comprising:

selecting the audio outputting device based on its location relative to the audio transmitting device.

4. The audio outputting device of claim 1, the process further comprising:

selecting the audio outputting device based on a user input received by the audio transmitting device.

5. The audio outputting device of claim 1, wherein the audio transmitting device contributes mapping information continuously to the shared map based on information from a first position sensor located in the audio transmitting device and the audio outputting device contributes mapping information continuously to the shared map based on information from a second position sensor located in the audio outputting device.

6. The audio outputting device of claim 1, wherein the audio transmitting device contributes mapping information continuously to the shared map based on information from a first orientation sensor located in the audio transmitting device and the audio outputting device contributes mapping information continuously to the shared map based on information from a second orientation sensor located in the audio outputting device.

7. The audio outputting device of claim 1, wherein the shared map is created by information received from external sensors, the external sensors being in communication with the audio transmitting device and the audio outputting device.

8. The audio outputting device of claim 1, wherein the augmented audio signal is output in a manner that further indicates the determined orientation of the axis of the audio outputting device to the audio transmitting device in the shared map of the physical space.

9. A method comprising:

determining a relative physical location of an audio transmitting device relative to an audio outputting device in a shared map of a physical space, the shared map being shared between the audio transmitting device and the audio outputting device;

determining an orientation of an axis of the audio outputting device relative to the audio transmitting device;

processing an audio signal received from the audio transmitting device via a communication network using the determined orientation of the axis of the audio outputting device relative to the audio transmitting device and the relative physical location of the audio transmitting device to create an augmented audio signal; and

outputting the augmented audio signal through at least one audio output on the audio outputting device, the augmented audio signal being output in a manner that indicates a relative physical direction of the audio transmitting device to the audio outputting device in the shared map of the physical space.

10. The method of claim 9, wherein the augmented audio signal is output in a manner that further indicates a three-dimensional relative physical direction of the audio transmitting device to the audio outputting device in the shared map of the physical space.

11. The method of claim 9, further comprising:

selecting the audio outputting device based on its location relative to the location and orientation of the axis of the audio transmitting device.

12. The method of claim 9, further comprising:

selecting the audio outputting device based on a user input received by the audio transmitting device.

13. The method of claim 9, wherein the audio transmitting device contributes mapping information continuously to the shared map based on information from a first position sensor located in the audio transmitting device and the audio outputting device contributes mapping information continuously to the shared map based on information from a second position sensor located in the audio outputting device.

14. The method of claim 9, wherein the audio transmitting device contributes mapping information continuously to the shared map based on information from a first orientation sensor located in the audio transmitting device and the audio outputting device contributes mapping information continuously to the shared map based on information from a second orientation sensor located in the audio outputting device.

15. The method of claim 9, wherein the shared map is created by information received from external sensors, the external sensors being in communication with the audio transmitting device and the audio outputting device.

16. One or more tangible processor-readable storage media embodied with instructions for executing on one or more processors and circuits of a computing device a process comprising:

determining a relative physical location of an audio transmitting device relative to an audio outputting device in a shared map of a physical space, the shared map being shared between the audio transmitting device and the audio outputting device;

determining an orientation of an axis of the audio outputting device relative to the audio transmitting device;

processing an audio signal received from the audio transmitting device via a communication network using the determined orientation of the axis of the audio outputting device relative to the audio transmitting device and the relative physical location of the audio transmitting device to create an augmented audio signal; and

outputting the augmented audio signal through at least one audio output on the audio outputting device, the augmented audio signal being output in a manner that indicates a relative physical direction of the audio transmitting device to the audio outputting device in the shared map of the physical space.

17. The one or more tangible processor-readable storage media of claim 16, wherein the process further comprises:

selecting the audio outputting device based on its location relative to the location and orientation of the axis of the audio transmitting device.

18. The one or more tangible processor-readable storage media of claim 16, wherein the process further comprises:

selecting the audio outputting device based on a user input received by the audio transmitting device.

19. The one or more tangible processor-readable storage media of claim 16, wherein the audio transmitting device contributes mapping information continuously to the shared map based on information from a first position sensor located in the audio transmitting device and the audio outputting device contributes mapping information continuously to the shared map based on information from a second position sensor located in the audio outputting device.

20. The one or more tangible processor-readable storage media of claim 16, wherein the audio transmitting device contributes mapping information continuously to the shared map based on information from a first orientation sensor located in the audio transmitting device and the audio outputting device contributes mapping information continuously to the shared map based on information from a second orientation sensor located in the audio outputting device.

21. The one or more tangible processor-readable storage media of claim 16, wherein the shared map is created by information received from external sensors, the external sensors being in communication with the audio transmitting device and the audio outputting device.