Combination of immersive and binaural sound

• US 10,575,094 B1
• Filed: 12/13/2018
• Issued: 02/25/2020
• Est. Priority Date: 12/13/2018
• Status: Active Grant

• Current Assignee: DTS, Inc. (Xperi Inc.)
• Original Assignee: DTS, Inc. (Xperi Inc.)
• Inventors: Slack, Brian
• Primary Examiner(s): Anwah, Olisa
• Application Number:
  US16/219,180
• Time in Patent Office: 439 Days
• CPC Class Codes: H04R 2205/022 Plurality of transducers corresponding to a plurality of sound channels in each earpiece of headphones or in a single enclosure
• CPC Class Codes: H04R 2205/024 Positioning of loudspeaker enclosures for spatial sound reproduction
• CPC Class Codes: H04R 2420/05 Detection of connection of loudspeakers or headphones to amplifiers
• CPC Class Codes: H04R 3/12 for distributing signals to two or more loudspeakers specially adapted for hearing aids H04R25/407
• CPC Class Codes: H04R 5/02 Spatial or constructional arrangements of loudspeakers
• CPC Class Codes: H04R 5/033 Headphones for stereophonic communication details thereof, e.g. relating to batteries, cables or control elements H04R1/10
• CPC Class Codes: H04R 5/04 Circuit arrangements, e.g. for selective connection of amplifier inputs/outputs to loudspeakers, for loudspeaker detection, or for adaptation of settings to personal preferences or hearing impairments combinations of amplifiers H03F3/68; stereophonic systems H04S
• CPC Class Codes: H04S 1/005 For headphones
• CPC Class Codes: H04S 2400/01 Multi-channel, i.e. more than two input channels, sound reproduction with two speakers wherein the multi-channel information is substantially preserved
• CPC Class Codes: H04S 2400/03 Aspects of down-mixing multi-channel audio to configurations with lower numbers of playback channels, e.g. 7.1 -> 5.1 H04S2400/01 takes precedence
• CPC Class Codes: H04S 2400/11 Positioning of individual sound objects, e.g. moving airplane, within a sound field H04S2420/13 takes precedence
• CPC Class Codes: H04S 2420/01 Enhancing the perception of the sound image or of the spatial distribution using head related transfer functions [HRTF's] or equivalents thereof, e.g. interaural time difference [ITD] or interaural level difference [ILD]
• CPC Class Codes: H04S 3/002 Non-adaptive circuits, e.g. manually adjustable or static, for enhancing the sound image or the spatial distribution control circuits for electronic adaptation of the sound field H04S7/30
• CPC Class Codes: H04S 3/004 For headphones
• CPC Class Codes: H04S 5/005 of the pseudo five- or more-channel type, e.g. virtual surround
• CPC Class Codes: H04S 7/304 For headphones
• CPC Class Codes: H04S 7/308 Electronic adaptation dependent on speaker or headphone connection

Abstract

The present subject matter provides a technical solution to the technical problems facing sound localization by separating sounds and reproducing the separated sounds using a set of loudspeakers and a set of headphones. A general soundtrack that is meant to be experienced throughout the room would play through the loudspeakers, and specific sounds that are meant to be experienced near the listener would be played through a binaural representation in the headphones. The headphones may be selected to avoid occluding the ear, allowing sound produced at the loudspeakers to be heard clearly. This separation and reproduction of sounds using a combination of a loudspeaker and headphone provides a technical solution to the technical problem facing typical surround sound systems by localizing sounds for listeners in any location within a room. This improves reproduction accuracy of location-specific audio objects, including audio objects above or below a coplanar speaker configuration.

Claims

1. An immersive sound system comprising:

one or more processors;

a storage device comprising instructions, which when executed by the one or more processors, configure the one or more processors to;

receive a surround sound audio input;

decompose the surround sound audio input into a scene sound component specific to a room and a user sound component specific to each headphone user;

output the scene sound component to a plurality of loudspeakers; and

output the user sound component to a user headphone.

2. The system of claim 1, the instructions further configuring the one or more processors to detect a headphone connection, wherein the decomposition of the surround sound audio input is responsive to the detection of the headphone connection.

3. The system of claim 1, the instructions further configuring the one or more processors to:

detect a headphone disconnection; and

output, responsive to the detection of the headphone disconnection, the scene sound component and the user sound component to the plurality of loudspeakers.

4. The system of claim 1, the instructions further configuring the one or more processors to:

determine a plurality of audio channels associated with surround sound audio input, each of the plurality of audio channels having an associated loudspeaker location;

receive loudspeaker configuration information, the loudspeaker configuration information indicating the number and location of each of the plurality of loudspeakers;

identify one or more unmatched channels based on a comparison between the plurality of audio channels and the loudspeaker configuration information; and

output the one or more unmatched channels to the user headphone.

5. The system of claim 1, wherein the user sound component includes a moving sound object.

6. The system of claim 1, wherein the user sound component includes an elevated sound object, the elevated sound object having an associated 3-D position above a listener location.

7. The system of claim 1, wherein the user headphone includes a bone conduction headphone.

8. The system of claim 1, wherein the user headphone includes stereo headphones, and wherein a head related transfer function (HRTF) is used to create a perception of sound from a 3-D location around the user headphone.

9. The system of claim 1, wherein the decomposition of the surround sound audio input includes instructions further configuring the one or more processors to:

decompose audio objects to the scene sound component, each audio object including an associated 3-D audio object position; and

decompose a sound source to the user sound component, the sound source including a playback audio signal in a final mix with an associated rendering method.

10. The system of claim 1, wherein the decomposition of the surround sound audio input includes instructions further configuring the one or more processors to:

decompose egocentric audio to the scene sound component, the egocentric audio including audio specific to each headphone user; and

decompose allocentric audio to the user sound component, the allocentric audio including audio specific to a room.

11. The system of claim 1, wherein the decomposition of the surround sound audio input includes instructions further configuring the one or more processors to:

decompose diegetic audio to the scene sound component, the diegetic audio including audio visible on a video screen or implied to be present on a scene displayed on the video screen; and

decompose non-diegetic audio to the user sound component, the non-diegetic audio not visible on the video screen or not implied to be present on the scene displayed on the video screen.

12. An immersive sound system method comprising:

receiving a surround sound audio input;

decomposing the surround sound audio input into a scene sound component specific to a room and a user sound component specific to each headphone user;

outputting the scene sound component to a plurality of loudspeakers; and

outputting the user sound component to a user headphone.

13. The method of claim 12, further including detecting a headphone connection, wherein the decomposition of the surround sound audio input is responsive to the detection of the headphone connection.

14. The method of claim 12, further including:

detecting a headphone disconnection; and

outputting, responsive to the detection of the headphone disconnection, the scene sound component and the user sound component to the plurality of loudspeakers.

15. The method of claim 12, further including:

determining a plurality of audio channels associated with surround sound audio input, each of the plurality of audio channels having an associated loudspeaker location;

receiving loudspeaker configuration information, the loudspeaker configuration information indicating the number and location of each of the plurality of loudspeakers;

identifying one or more unmatched channels based on a comparison between the plurality of audio channels and the loudspeaker configuration information; and

outputting the one or more unmatched channels to the user headphone.

16. The method of claim 12, wherein the user headphone includes a bone conduction headphone.

17. The method of claim 12, wherein the user headphone includes stereo headphones, and wherein a head related transfer function (HRTF) is used to create a perception of sound from a 3-D location around the user headphone.

18. A non-transitory machine-readable storage medium comprising a plurality of instructions that, when executed with a processor of a device, cause the device to:

receive a surround sound audio input;

decompose the surround sound audio input into a scene sound component specific to a room and a user sound component specific to each headphone user;

output the scene sound component to a plurality of loudspeakers; and

output the user sound component to a user headphone.

19. The non-transitory machine-readable storage medium of claim 18, the instructions further causing the device to detect a headphone connection, wherein the decomposition of the surround sound audio input is responsive to the detection of the headphone connection.

20. The non-transitory machine-readable storage medium of claim 18, the instructions further causing the device to:

detect a headphone disconnection; and

output, responsive to the detection of the headphone disconnection, the scene sound component and the user sound component to the plurality of loudspeakers.