SYSTEM AND METHOD FOR DETERMINING AUDIO CONTEXT IN AUGMENTED-REALITY APPLICATIONS

• US 20170208415A1
• Filed: 07/09/2015
• Published: 07/20/2017
• Est. Priority Date: 07/23/2014
• Status: Abandoned Application

• Current Assignee: PCMS Holdings Incorporated (InterDigital, Inc.)
• Original Assignee: PCMS Holdings Incorporated (InterDigital, Inc.)
• Inventors: Ojala, Pasi Sakari
• Application Number:
  US15/327,314
• Publication Number:
  US 20170208415A1
• Time in Patent Office: Days
• CPC Class Codes: A63F 13/215 comprising means for detecting acoustic signals, e.g. using a microphone
• CPC Class Codes: A63F 13/54 involving acoustic signals, e.g. for simulating revolutions per minute [RPM] dependent engine sounds in a driving game or reverberation against a virtual wall
• CPC Class Codes: A63F 2300/6081 generating an output signal, e.g. under timing constraints, for spatialization
• CPC Class Codes: G10L 2021/02166 Microphone arrays; Beamforming
• CPC Class Codes: G10L 21/043 by changing speed
• CPC Class Codes: H04R 5/033 Headphones for stereophonic communication details thereof, e.g. relating to batteries, cables or control elements H04R1/10
• CPC Class Codes: H04S 2400/01 Multi-channel, i.e. more than two input channels, sound reproduction with two speakers wherein the multi-channel information is substantially preserved
• CPC Class Codes: H04S 2400/11 Positioning of individual sound objects, e.g. moving airplane, within a sound field H04S2420/13 takes precedence
• CPC Class Codes: H04S 2420/01 Enhancing the perception of the sound image or of the spatial distribution using head related transfer functions [HRTF's] or equivalents thereof, e.g. interaural time difference [ITD] or interaural level difference [ILD]
• CPC Class Codes: H04S 2420/07 Synergistic effects of band splitting and sub-band processing
• CPC Class Codes: H04S 7/00 Indicating arrangements; Control arrangements, e.g. balance control
• CPC Class Codes: H04S 7/30 Control circuits for electronic adaptation of the sound field
• CPC Class Codes: H04S 7/304 For headphones
• CPC Class Codes: H04S 7/306 For headphones

Abstract

An augmented-reality audio system generates information regarding the acoustic environment by sampling audio signals. Using a Gaussian mixture model or other technique, the system identifies the location of one or more audio sources, with each source contributing an audio component to the sampled audio signals. The system determines a reverberation time for the acoustic environment using the audio components. In determining the reverberation time, the system may discard audio components from sources that are determined to be in motion, such as components with an angular velocity above a threshold or components having a Doppler shift above a threshold. The system may also discard audio components from sources having an inter-channel coherence above a threshold. In at least one embodiment, the system renders sounds using the reverberation time at virtual locations that are separated from the locations of the audio sources.

Claims

1. A method comprising:

sampling an audio signal from a plurality of microphones of an augmented-reality headset;

determining by an augmented-reality headset at least one location of a respective audio source from the sampled audio signal; and

on a plurality of speakers of the augmented-reality headset, rendering by the augmented-reality headset an augmented-reality audio signal having a virtual location separated from the at least one determined location by at least a threshold separation.

3. The method of claim 1, wherein rendering comprises applying a head-related transfer function filtering.

4. The method of claim 1, wherein the determined location is an angular position, and wherein the threshold separation is a threshold angular distance.

5. The method of claim 4, wherein the threshold angular distance has a value selected from the group consisting of 5 degrees and 10 degrees.

6. The method of claim 1, wherein the at least one audio source comprises multiple audio sources, and wherein the virtual location is separated from each of the respective determined locations by at least the threshold separation.

7. The method of claim 6, further comprising distinguishing among the multiple audio sources based on one or more statistical properties selected from the group consisting of the range of harmonic frequencies, sound level, and coherence.

8. The method of claim 6, wherein each of the multiple audio sources contributes a respective audio component to the sampled audio signal, the method further comprising:

determining that each of the audio components has a respective coherence level that is above a predetermined coherence-level threshold.

9. The method of claim 6, further comprising identifying each of the multiple audio sources using a Gaussian mixture model.

10. The method of claim 6, further comprising identifying each of the multiple audio sources at least in part by determining a probability density function of direction of arrival data.

11. The method of claim 6, further comprising identifying each of the multiple audio sources at least in part by modeling a probability density function of direction of arrival data as a sum of probability distribution functions of the multiple audio sources.

12. The method of claim 1, wherein the sampled audio signal is not a test signal.

13. The method of claim 1, wherein the location determination is performed using binaural cue coding.

14. The method of claim 1, wherein the location determination is performed by analyzing a sub-band in the frequency domain.

15. The method of claim 1, wherein the location determination is performed using inter-channel time difference.

2. (canceled)

16. An augmented-reality headset comprising:

a plurality of microphones;

at least one audio-output device;

a processor; and

data storage containing instructions executable by the processor for causing the augmented-reality headset to carry out a set of functions, the set of functions including;

sampling an audio signal from the plurality of microphones;

determining a respective location of at least one audio source from the sampled audio signal;

rendering, via the at least one audio-output device, an augmented-reality audio signal having a virtual location separated from the at least one determined location by at least a threshold separation.

17. A method comprising:

sampling at least one audio signal from a plurality of microphones;

determining a reverberation time based on the sampled at least one audio signal;

modifying an augmented-reality audio signal based at least in part on the determined reverberation time; and

rendering the modified augmented-reality audio signal.

18. The method of claim 17, wherein modifying the augmented-reality audio signal based at least in part on the determined reverberation time comprises applying to the augmented-reality audio signal a reverberation corresponding to the determined reverberation time.

19. The method of claim 17, wherein modifying the augmented-reality audio signal based at least in part on the determined reverberation time comprises applying to the augmented-reality audio signal a reverberation filter corresponding to the determined reverberation time.

20. The method of claim 17, wherein modifying the augmented-reality audio signal based at least in part on the determined reverberation time comprises slowing down the augmented-reality audio signal by an amount determined based at least in part on the determined reverberation time.