Augmented reality sound system

• US 9,648,436 B2
• Filed: 09/01/2015
• Issued: 05/09/2017
• Est. Priority Date: 04/08/2014
• Status: Active Grant

• Current Assignee: Dolby Laboratories Licensing Corporation (Dolby Laboratories Incorporated)
• Original Assignee: Doppler Labs, Inc.
• Inventors: Lanman, III, Richard Fritz, Kraft, Noah, Wiggins, Daniel C., Parks, Anthony, Klimanis, Gints, Baker, Jeff
• Primary Examiner(s): Edun, Muhammad N
• Application Number:
  US14/842,594
• Publication Number:
  US 20150373474A1
• Time in Patent Office: 616 Days
• CPC Class Codes: H04R 1/1083 Reduction of ambient noise active noise reduction per se G10K11/175; protective devices for the ear, e.g. providing acoustic protection A61F11/06
• CPC Class Codes: H04S 5/00 Pseudo-stereo systems, e.g. in which additional channel signals are derived from monophonic signals by means of phase shifting, time delay or reverberation
• CPC Class Codes: H04W 4/02 Services making use of location information
• CPC Class Codes: H04W 4/024 Guidance services
• CPC Class Codes: H04W 4/029 Location-based management or tracking services

Abstract

An augmented reality sound systems is disclosed. An augmented reality sound system includes a at least one microphone for receiving ambient sound and a memory storing one or more augmented reality sound profiles and a respective set of processing parameters. The system further includes a processor coupled to the memory and configured to generate augmented ambient sound from the ambient sound by reproducing the ambient sound in conjunction with processed sound superimposed over the ambient sound as directed by one or more of the set of processing parameters retrieved from the memory based on a selected augmented reality sound profile.

Claims

1. An augmented reality sound system, comprising:

at least one microphone for receiving ambient sound;

a memory storing one or more augmented reality sound profiles each associated with one or more processing parameters; and

a processor coupled to the memory and configured to;

analyze the ambient sound to produce processed sound that at least emphasizes one or more particular aspects of the ambient sound; and

generate augmented ambient sound from the ambient sound by reproducing the ambient sound in conjunction with the processed sound superimposed over the ambient sound as directed by one or more of the set of processing parameters retrieved from the memory based on a selected augmented reality sound profile.

2. The augmented reality sound system of claim 1, further comprising a plurality of microphones, in addition to the at least one microphone and wherein the processor is further configured to:

receive ambient sound from the plurality of microphones; and

automatically select an augmented reality sound profile at least in part based upon the ambient sound received from the plurality of microphones.

3. The augmented reality sound system of claim 2 wherein the plurality of microphones are dispersed between more than one individual.

4. The augmented reality sound system of claim 2 further comprising a personal computing device, coupled to the processor via a wireless communication link, the personal computing device including at least one of the plurality of microphones.

5. The augmented reality sound system of claim 1, wherein the processor is further configured to:

analyze digitized ambient sound to generate an ambient sound profile,

compare the ambient sound profile to at least one of the one or more augmented reality sound profiles, and

select the selected augmented reality sound profile of the one or more augmented reality sound profiles that is appropriate for the ambient sound profile according to predetermined criteria.

6. The augmented reality sound system of claim 1, wherein the selected augmented reality sound profile is selected by someone other than a wearer of the at least one microphone.

7. The augmented reality sound system of claim 1, wherein the one or more of augmented reality sound profiles that create processed sound include at least one of the following:

increased volume of at least one specific frequency range of the ambient sound, decreased volume of at least one specific frequency range of the ambient sound, insert pre-recorded sound as a part of the processed sound superimposed over the ambient sound, pre-recorded sound providing additional information about an identified ambient sound, pre-recorded sound identifying a sound in the ambient sound, a live audio feed directly from audio equipment of a live audio performance, pre-recorded sound selected based upon a user location, pre-recorded sound selected based upon the presence of another in proximity to a user of the system, locational feedback to the user of the system in response to a particular ambient sound in a specific frequency range so as to better-identify a direction of the particular ambient sound to the user of the system, auditory directions to the user of the system so as to enable them to navigate to a location while reproducing the ambient sound so as to not impede user hearing of the ambient sound, and reproduction of live audio superimposed over the ambient sound.

8. The augmented reality sound system of claim 1 further comprising:

a second at least one microphone, a second memory and a second processor as described in claim 1 used to generate a second augmented ambient sound;

the second at least one microphone for receiving a second ambient sound; and

a third processor for combining the ambient sound from the at least one microphone and the second ambient sound from the second at least one microphone to create a multi-source ambient sound and using the multi-source ambient sound to select the selected ambient sound profile.

9. The augmented reality sound system of claim 8, wherein the third processor is distinct from the first and second processors and located in one of a personal computing device and a server.

10. The augmented reality sound system of claim 1, further comprising:

a housing enclosing at least a portion of the processor and at least a portion of the memory;

a preamplifier, and an analog to digital converter to convert the ambient sound into digitized ambient sound input to the processor; and

a digital to analog converter, an amplifier, and a speaker to convert digitized processed sound output from the processor into processed output sound, wherein

the at least one microphone, the preamplifier, the analog to digital converter, the digital to analog converter, the amplifier, and the speaker are disposed within the housing.

11. The augmented reality sound system of claim 10, wherein:

the housing is an earbud housing configured to fit, at least partially, within and be supported by a user'"'"'s ear.

12. The augmented reality sound system of claim 1, wherein the processor is further configured to generate augmented ambient sound from the ambient sound by reproducing the ambient sound in conjunction with informational feedback and the processed sound superimposed over the ambient sound.

13. The augmented reality sound system of claim 12, wherein the informational feedback includes information regarding a location of one or more ambient sound features within the ambient sound.

14. A method for processing ambient sound to provide augmented reality sound, comprising:

storing, in a memory, one or more augmented reality sound profiles, each associated with one or more processing parameters;

analyzing the ambient sound to produce processed sound that at least emphasizes one or more particular aspects of the ambient sound; and

generating augmented ambient sound from the ambient sound by reproducing the ambient sound received by at least one microphone in conjunction with the processed sound superimposed over the ambient sound as directed by one or more of the set of processing parameters retrieved from the memory based on a selected augmented reality sound profile.

15. The method of claim 14, further comprising:

receiving ambient sound from a plurality of microphones that are separate from the at least one microphone; and

automatically selecting an augmented reality sound profile at least in part based upon the ambient sound received from the plurality of microphones.

16. The method of claim 15, wherein the plurality of microphones are dispersed between more than one individual.

17. The method of claim 16, wherein a personal computing device includes at least one of the plurality of microphones and is coupled to the processor via a wireless communication link.

18. The method of claim 14 further comprising:

analyzing digitized ambient sound to generate an ambient sound profile,

comparing the ambient sound profile to at least one of the one or more augmented reality sound profiles, and

selecting an augmented reality sound profile of the one or more augmented reality sound profiles that is appropriate for the ambient sound profile according to predetermined criteria.

19. The method of claim 14, wherein the selected augmented reality sound profile is selected by someone other than a wearer of the at least one microphone.

20. The method of claim 14, wherein the one or more augmented reality sound profiles that create processed sound include at least one of the following:

increased volume of at least one specific frequency range of the ambient sound, decreased volume of at least one specific frequency range of the ambient sound, insert pre-recorded sound as a part of the processed sound superimposed over the ambient sound, pre-recorded sound providing additional information about an identified ambient sound, pre-recorded sound identifying a sound in the ambient sound, a live audio feed directly from audio equipment of a live audio performance, pre-recorded sound selected based upon a user location, pre-recorded sound selected based upon the presence of another in proximity to a user, locational feedback to the user in response to a particular ambient sound in a specific frequency range so as to better-identify a direction of the particular ambient sound to the user, auditory directions to the user so as to enable them to navigate to a location while reproducing the ambient sound so as to not impede user hearing of the ambient sound, and reproduction of live audio superimposed over the ambient sound.

21. The method of claim 14 further comprising:

receiving a second ambient sound from at least one second microphone; and

combining the ambient sound from the at least one microphone and the second ambient sound from the at least one second microphone to create a multi-source ambient sound and using the multi-source ambient sound to select the selected ambient sound profile.

22. The method of claim 21, wherein the combining is performed on a processor located in one of a personal computing device and a server.