CONTEXTUAL AUDIO DUCKING WITH SITUATION AWARE DEVICES

• US 20140006026A1
• Filed: 06/29/2012
• Published: 01/02/2014
• Est. Priority Date: 06/29/2012
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Robert L. Crocco Jr
• Inventors: Lamb, Mathew J., Sugden, Ben J., Crocco, Robert L. Jr., Keane, Brian E., Miles, Christopher E., Perez, Kathryn Stone, Massey, Laura K., Kipman, Alex Aben-Athar
• Granted Patent: US 9,384,737 B2
• Time in Patent Office: Days
• US Class Current: 704/246
• CPC Class Codes: G10L 17/00 Speaker identification or verification techniques
• CPC Class Codes: H03G 3/10 having semiconductor devices

Abstract

A system for generating one or more enhanced audio signals such that one or more sound levels corresponding with sounds received from one or more sources of sound within an environment may be dynamically adjusted based on contextual information is described. The one or more enhanced audio signals may be generated by a head-mounted display device (HMD) worn by an end user within the environment and outputted to earphones associated with the HMD such that the end user may listen to the one or more enhanced audio signals in real-time. In some cases, each of the one or more sources of sound may correspond with a priority level. The priority level may be dynamically assigned depending on whether the end user of the HMD is focusing on a particular source of sound or has specified a predetermined level of importance corresponding with the particular source of sound.

Claims

1. A method for adjusting sound levels based on contextual information, comprising:

capturing at a mobile device one or more sounds from an environment;

identifying one or more sources of sound within the environment based on the one or more sounds, the one or more sources of sound include a first sound source and a second sound source different from the first sound source;

assigning a first priority level to the first sound source;

assigning a second priority level to the second sound source;

determining a first sound level associated with the first sound source;

determining a second sound level associated with the second sound source;

determining one or more weighting coefficients based on the first priority level and the second priority level;

generating one or more enhanced audio signals based on the one or more weighting coefficients; and

outputting the one or more enhanced audio signals from the mobile device.

2. The method of claim 1, wherein:

the assigning a first priority level to the first sound source includes determining whether an end user of the mobile device is focusing on the first sound source.

3. The method of claim 1, wherein:

the assigning a first priority level to the first sound source includes determining whether an end user of the mobile device has focused on the first sound source for at least a particular period of time.

4. The method of claim 1, wherein:

the first sound source comprises a particular person; and

the assigning a first priority level to the first sound source includes identifying the particular person and acquiring a user profile associated with the particular person.

5. The method of claim 1, wherein:

the identifying one or more sources of sound includes identifying the first sound source using voice recognition techniques.

6. The method of claim 1, further comprising:

determining a first direction associated with the first sound source; and

determining a second direction associated with the second sound source, the generating the one or more enhanced audio signals includes generating a first new audio signal based on the first direction using beamforming techniques and generating a second new audio signal based on the second direction using beamforming techniques, the generating the one or more enhanced audio signals includes generating a combined audio signal based on the first new audio signal and the second new audio signal.

7. The method of claim 1, wherein:

the determining one or more weighting coefficients includes determining the one or more weighting coefficients such that a minimum volume ratio is maintained.

8. The method of claim 1, wherein:

the first sound source comprises a first person;

the second sound source comprises a second person different from the first person;

the determining one or more weighting coefficients is performed by the mobile device;

the generating the one or more enhanced audio signals is performed by the mobile device; and

the mobile device comprises an HMD.

9. One or more storage devices containing processor readable code for programming one or more processors to perform a method for adjusting sound levels based on contextual information comprising the steps of:

detecting a reading object within an environment;

identifying an AR tag associated with the reading object,

playing a first portion of an audio recording corresponding with the AR tag, the first portion of the audio recording is associated with a first sound level and a first priority level, the playing a first portion is performed by a mobile device;

identifying a second sound source within the environment;

assigning a second priority level to the second sound source subsequent to the playing a first portion;

determining a first weighting coefficient based on the first priority level and the second priority level;

adjusting the first sound level based on the first weighting coefficient; and

playing a second portion of the audio recording corresponding with the AR tag at the first sound level subsequent to the adjusting the first sound level, the playing a second portion is performed by the mobile device.

10. The one or more storage devices of claim 9, wherein:

the assigning a second priority level includes determining whether an end user of the mobile device is focusing on the second sound source.

11. The one or more storage devices of claim 9, wherein:

the assigning a second priority level includes determining whether an end user of the mobile device has focused on the second sound source for at least a particular period of time.

12. The one or more storage devices of claim 9, wherein:

the second sound source comprises a particular person; and

the assigning a second priority level includes identifying the particular person and acquiring a user profile associated with the particular person.

13. The one or more storage devices of claim 9, wherein:

the identifying a second sound source includes identifying the second sound source using voice recognition techniques.

14. The one or more storage devices of claim 9, wherein:

the reading object comprises an augmentable book; and

the audio recording comprises a particular song associated with the augmentable book.

15. The one or more storage devices of claim 9, wherein:

the reading object comprises an augmentable book;

the AR tag corresponds with a particular page of the augmentable book; and

the mobile device comprises an HMD.

16. The one or more storage devices of claim 9, wherein:

the determining a first weighting coefficient includes determining the first weighting coefficient such that a minimum volume ratio is maintained.

17. An electronic device for adjusting sound levels based on contextual information, comprising:

a plurality of microphones, the plurality of microphones captures one or more sounds from an environment;

one or more processors in communication with the plurality of microphones, the one or more processors identify one or more sources of sound within the environment based on the one or more sounds, the one or more sources of sound include a first sound source and a second sound source different from the first sound source, the one or more processors assign a first priority level to the first sound source and a second priority level to the second sound source, the one or more processors determine a first sound level associated with the first sound source and a second sound level associated with the second sound source, the one or more processors determine one or more weighting coefficients based on the first priority level and the second priority level, the one or more processors generate one or more enhanced audio signals based on the one or more weighting coefficients; and

an audio output device in communication with the one or more processors, the audio output device outputs the one or more enhanced audio signals.

18. The electronic device of claim 17, wherein:

the one or more processors assign the first priority level to the first sound source based on whether an end user of the electronic device is focusing on the first sound source.

19. The electronic device of claim 17, wherein:

the one or more processors assign the first priority level to the first sound source based on whether an end user of the electronic device has focused on the first sound source for at least a particular period of time.

20. The electronic device of claim 17, wherein:

the audio output device comprises earphones;

the first sound source comprises a particular person; and

the one or more processors identify the first sound source using voice recognition techniques.