Binaural sound in visual entertainment media

• US 10,848,899 B2
• Filed: 10/13/2016
• Issued: 11/24/2020
• Est. Priority Date: 10/13/2016
• Status: Active Grant

• Current Assignee: Eight kHz, LLC
• Original Assignee: Glen A. Norris, Philip Scott Lyren
• Inventors: Lyren, Philip Scott, Norris, Glen A.
• Primary Examiner(s): Nguyen, Duc
• Assistant Examiner(s): Mohammed, Assad
• Application Number:
  US15/293,251
• Publication Number:
  US 20180109900A1
• Time in Patent Office: 1,503 Days
• CPC Class Codes: H04R 2499/11 Transducers incorporated or for use in hand-held devices, e.g. mobile phones, PDA's, camera's
• CPC Class Codes: H04S 1/005 For headphones
• CPC Class Codes: H04S 1/007 in which the audio signals are in digital form data reduction aspects thereof based on psychoacoustics G10L19/02
• CPC Class Codes: H04S 2400/01 Multi-channel, i.e. more than two input channels, sound reproduction with two speakers wherein the multi-channel information is substantially preserved
• CPC Class Codes: H04S 2400/11 Positioning of individual sound objects, e.g. moving airplane, within a sound field H04S2420/13 takes precedence
• CPC Class Codes: H04S 2400/13 Aspects of volume control, not necessarily automatic, in stereophonic sound systems
• CPC Class Codes: H04S 2420/01 Enhancing the perception of the sound image or of the spatial distribution using head related transfer functions [HRTF's] or equivalents thereof, e.g. interaural time difference [ITD] or interaural level difference [ILD]
• CPC Class Codes: H04S 7/304 For headphones

Abstract

A method provides binaural sound to a listener while the listener watches a movie so sounds from the movie localize to a location of a character in the movie. Sound is convolved with head related transfer functions (HRTFs) of the listener, and the convolved sound is provided to the listener who wears a wearable electronic device.

Claims

1. A method to provide binaural sound to a listener in a movie theater, the method comprising:

determining different head orientations of the listener with respect to an image of a character as the image of the character moves to different locations from one side of a movie screen to an opposite side of the movie screen while being displayed to the listener on the movie screen during a movie in the movie theater;

selecting, based on the head orientations of the listener with respect to the image of the character, head related transfer functions (HRTFs) so a voice of the character is heard to originate to the listener from the image of the character as the image of the character moves to the different locations from the one side of the movie screen to the opposite side of the movie screen;

convolving, with a digital signal processor, the voice of the character with the HRTFs;

providing, through a wearable electronic device and to the listener, the voice of the character convolved with the HRTFs so the voice of the character localizes to the listener as originating from the image of the character as the image of the character moves to the different locations from the one side of the movie screen to the opposite side of the movie screen;

determining when a position of the character is no longer located on the movie screen; and

switching, in response to a position of the character no longer being located on the movie screen, a point-of-view for sound being provided to the listener to another character that is on the movie screen so the listener continues to localize an origin of sound to the movie screen.

2. The method of claim 1, wherein the one side of the movie screen is a left side of the movie screen as seen from the listener and the opposite side of the movie screen is a right side of the movie screen as seen from the listener.

3. The method of claim 1 further comprising:

determining azimuth angles and elevation angles between a forward facing head orientation of the listener and the image of the character that is located on the movie screen as the image of the character moves to the different locations from the one side of the movie screen to the opposite side of the movie screen; and

selecting the HRTFs based on the azimuth angles and the elevation angles so the voice of the character localizes to the listener as originating from the image of the character as the image of the character moves to the different locations from the one side of the movie screen to the opposite side of the movie screen.

4. The method of claim 1 further comprising:

determining a distance from the listener to where the character is visually represented in a scene in the movie; and

adjusting a loudness of the voice of the character based on the distance.

5. The method of claim 1 further comprising:

providing the listener with a selection of different characters that appear in the movie shown on the movie screen;

receiving a selection of one of the characters; and

providing the listener with binaural sound from the point-of-view of the one of the characters so the listener hears sounds as if the listener were the one of the characters.

6. The method of claim 1 further comprising:

determining when a position of the character is no longer located on the movie screen; and

switching, in response to the position of the character no longer being located on the movie screen, the voice of the character from being provided in binaural sound to being provided to the listener in stereo sound.

7. A method to provide binaural sound to a listener from a point-of-view of a character in a feature length movie while the listener watches the feature length movie, the method comprising:

determining an azimuth angle between the character and a source of sound in the feature length movie;

selecting a head related transfer function (HRTF) that corresponds to the azimuth angle;

convolving, with a digital signal processor, sound from the source of the sound with the HRTF; and

providing, through a wearable electronic device and to the listener, the sound convolved with the HRTF so the listener hears the sound from the point-of-view of the character while the listener watches the feature length movie.

8. The method of claim 7 further comprising:

receiving, from the listener, a selection of the character indicating that the listener desires to hear sound from the point-of-view of the character.

9. The method of claim 7 further comprising:

tracking a head orientation of the listener; and

switching, while the listener watches the feature length movie, the sound from being provided in binaural sound to the listener to being provided in stereo sound to the listener when the head orientation of the listener is not directed to the feature length movie.

10. The method of claim 7 further comprising:

playing a signal sound to notify the listener when the point-of-view changes from the character to another character.

11. The method of claim 7 further comprising:

providing, while the listener watches the feature length movie, a voice of the character to the listener in stereo sound while other sounds are simultaneously provided to the listener in binaural sound.

12. The method of claim 7 further comprising:

providing, while the listener watches the feature length movie, a voice of the character to the listener in binaural sound so the voice of the character localizes to the listener one meter above the head of the listener while other sounds are simultaneously provided to the listener in stereo sound.

13. The method of claim 7 further comprising:

providing, through the wearable electronic device and to the listener, sounds that localize to the listener as originating behind the listener while the listener watches the feature length movie.

14. The method of claim 7 further comprising:

providing, through the wearable electronic device and to the listener, sounds that localize to the listener as originating between one to two meters away from the listener while the listener watches the feature length movie.

15. A method to provide binaural sound to a listener while the listener watches a movie so sounds from the movie localize behind the listener, the method comprising:

obtaining head related transfer functions (HRTFs) for the listener;

selecting a character in the movie as an audial point-of-view of the listener;

convolving, with a digital signal processor and with the HRTFs, a sound in the movie that originates from behind the character so the sound originates from behind the listener at a location that is in empty space not occupied by a tangible object; and

providing, through a wearable electronic device and to the listener, the sound convolved with the HRTFs so the listener localizes the sound to originate from behind the listener at the location that is in the empty space not occupied by a tangible object while the listener watches the movie.

16. The method of claim 15 further comprising:

convolving, with the digital signal processor and with the HRTFs, a voice in the movie that originates in front of the character so the voice originates in front of the listener in the empty space; and

providing, through the wearable electronic device and to the listener, the voice convolved with the HRTFs so the listener localizes the voice to originate in front of the listener in the empty space while the listener watches the movie.

17. The method of claim 15 further comprising:

convolving, with the digital signal processor and with the HRTFs, a voice in the movie that originates above the character so the voice originates above the listener in the empty space; and

providing, through the wearable electronic device and to the listener, the voice convolved with the HRTFs so the listener localizes the voice to originate above the listener in the empty space while the listener watches the movie.

18. The method of claim 15, wherein the location that is in the empty space is located one meter from a head of the listener.

19. The method of claim 15, wherein the location that is in the empty space is located at an azimuth angle with a value in a range between 135°

to 225°

when a line-of-sight of the listener is an azimuth angle of 0°

.