Wearable electronic device selects HRTFs based on eye distance and provides binaural sound

• US 10,595,143 B2
• Filed: 05/02/2019
• Issued: 03/17/2020
• Est. Priority Date: 06/23/2014
• Status: Active Grant

• Current Assignee: Glen A. Norris, Philip Scott Lyren
• Original Assignee: Glen A. Norris, Philip Scott Lyren
• Inventors: Norris, Glen A., Lyren, Philip Scott
• Primary Examiner(s): Zhu, Qin
• Application Number:
  US16/401,157
• Publication Number:
  US 20190261113A1
• Time in Patent Office: 320 Days
• Field of Search: None
• CPC Class Codes: G10L 21/028 using properties of sound source
• CPC Class Codes: H04M 1/6033 for providing handsfree use or a loudspeaker mode in telephone sets arrangements for preventing acoustic echo H04M9/08
• CPC Class Codes: H04M 1/605 involving control of the receiver volume to provide a dual operational mode at close or far distance from the user
• CPC Class Codes: H04M 1/72457 according to geographic location
• CPC Class Codes: H04M 3/53366 Message disposing or creating aspects
• CPC Class Codes: H04M 3/568 audio processing specific to telephonic conferencing, e.g. spatial distribution, mixing of participants echo suppression in two-way loud-speaking telephone systems H04M9/02; sound field processing per se H04S7/30
• CPC Class Codes: H04M 9/085 using digital techniques H04M9/082 takes precedence
• CPC Class Codes: H04R 2227/003 Digital PA systems using, e.g. LAN or internet
• CPC Class Codes: H04R 27/00 Public address systems circuits for preventing acoustic reaction H04R3/02; circuits for distributing signals to loudspeakers H04R3/12; monitoring or testing arrangements for public address systems H04R29/007; amplifiers H03F
• CPC Class Codes: H04R 5/027 Spatial or constructional arrangements of microphones, e.g. in dummy heads
• CPC Class Codes: H04R 5/033 Headphones for stereophonic communication details thereof, e.g. relating to batteries, cables or control elements H04R1/10
• CPC Class Codes: H04R 5/04 Circuit arrangements, e.g. for selective connection of amplifier inputs/outputs to loudspeakers, for loudspeaker detection, or for adaptation of settings to personal preferences or hearing impairments combinations of amplifiers H03F3/68; stereophonic systems H04S
• CPC Class Codes: H04S 1/002 Non-adaptive circuits, e.g. manually adjustable or static, for enhancing the sound image or the spatial distribution control circuits for electronic adaptation of the sound field H04S7/30
• CPC Class Codes: H04S 1/007 in which the audio signals are in digital form data reduction aspects thereof based on psychoacoustics G10L19/02
• CPC Class Codes: H04S 2400/01 Multi-channel, i.e. more than two input channels, sound reproduction with two speakers wherein the multi-channel information is substantially preserved
• CPC Class Codes: H04S 2400/11 Positioning of individual sound objects, e.g. moving airplane, within a sound field H04S2420/13 takes precedence
• CPC Class Codes: H04S 2400/15 Aspects of sound capture and related signal processing for recording or reproduction
• CPC Class Codes: H04S 2420/01 Enhancing the perception of the sound image or of the spatial distribution using head related transfer functions [HRTF's] or equivalents thereof, e.g. interaural time difference [ITD] or interaural level difference [ILD]
• CPC Class Codes: H04S 3/008 in which the audio signals are in digital form, i.e. employing more than two discrete digital channels data reduction aspects thereof based on psychoacoustics G10L19/02
• CPC Class Codes: H04S 7/303 Tracking of listener position or orientation
• CPC Class Codes: H04S 7/304 : For headphones
• CPC Class Codes: H04S 7/305 : Electronic adaptation of stereophonic audio signals to reverberation of the listening space H04S7/301 takes precedence
• CPC Class Codes: H04S 7/306 : For headphones

Abstract

A method provides a voice in binaural sound to a user. A wearable electronic device (WED) selects head-related transfer functions (HRTFs) based on a distance between eyes of the user. A digital signal processor (DSP) in the WED processes the voice with the HRTFs to generate the voice in the binaural sound that localizes at a sound localization point (SLP) at a location in empty space to the user.

Claims

1. A method to provide a voice in binaural sound to a user during an electronic call, the method comprising:

determining, with a wearable electronic device (WED) worn on a head of the user, a physical attribute of the user that includes a distance between eyes of the user;

selecting, based on the physical attribute of the user that includes the distance between the eyes of the user, user-specific head-related transfer functions (HRTFs) of the user;

processing, with a digital signal processor (DSP) during the electronic call, the voice with the user-specific HRTFs to generate the voice in the binaural sound that localizes at a sound localization point (SLP) at a location in empty space at least one meter away from the user;

detecting, with a camera in the WED, eye movement of the user; and

altering, in response to the detecting of the eye movement, the user-specific HRTFs being processed with the voice to move the voice from externally localizing to the SLP to externally localizing to another SLP.

2. The method of claim 1 further comprising:

detecting a body gesture of the user with a sensor in the WED; and

altering, in response to the detecting the body gesture, the voice from being provided to the user in the binaural sound to being provided to the user in stereo sound,

wherein the location in empty space includes a virtual image of a person with whom the user is talking during the electronic call.

3. The method of claim 1 further comprising:

recognizing, at the WED, a body gesture from the user that designates the location of the SLP in empty space at least one meter away from the user where the user will hear the voice.

4. The method of claim 1 further comprising:

receiving, at the WED, a body gesture from the user that designates the location of the SLP as being on a wall in a room where the user is located.

5. The method of claim 1 further comprising:

receiving, at the WED, a body gesture from the user that designates the location of the SLP as being on a physical chair in a room where the user is located.

6. The method of claim 1 further comprising:

receiving, at a microphone in the WED and from the user during the electronic call, a voice command to move the SLP of a window from a foreground to a background where the SLP is away from the user to make the binaural sound softer.

7. The method of claim 1 further comprising:

detecting, with the WED, a body gesture of the user that commands the WED to switch the voice from the binaural sound to stereo sound.

8. The method of claim 1 further comprising:

receiving, at a microphone in the WED and from the user during the electronic call, a voice command to move the voice from the SLP at the location in empty space at least one meter away from the user to another location in empty space at least one meter away from the user.

9. A non-transitory computer readable storage medium storing instructions that one or more electronic devices execute to provide a voice in binaural sound to a user, the method comprising:

measuring, with a wearable electronic device (WED) worn on a head of the user, a distance between eyes of the user;

selecting, based on the distance between the eyes of the user, head-related transfer functions (HRTFs) for the user;

processing, with a digital signal processor (DSP), the voice with the HRTFs to generate the voice in the binaural sound that localizes at a sound localization point (SLP) at a location in empty space at least one meter away from the head of the user;

maintaining the SLP at a fixed location with respect to the head of the user as the user moves around a room in which the user and the SLP are located such that an origin of the voice at the SLP follows the user as the user moves around the room;

displaying, with the WED, multiple screens with SLPs that externally localize as the binaural sound in empty space at each of the multiple screens; and

receiving, at the WED, a body gesture from the user that commands the WED to arrange the SLPs in an order that includes moving one of the SLPs to a foreground and one of the SLPs to a background.

10. The non-transitory computer readable storage medium of claim 9 further comprising:

receiving, by a natural language user interface in the WED, a voice command from the user to move the SLP from the fixed location having a first set of spherical coordinates with respect to the head of the user to a fixed location having a second set of spherical coordinates with respect to the head of the user; and

altering, in response to receiving the voice command, the HRTFs being processed with the voice to move the SLP from the fixed location having the first set of spherical coordinates with respect to the head of the user to the fixed location having the second set of spherical coordinates with respect to the head of the user.

11. The non-transitory computer readable storage medium of claim 9 further comprising:

detecting a second body gesture of the user with a sensor in the WED; and

altering, in response to the detecting the second body gesture, the voice from being provided to the user in the binaural sound to being provided to the user in stereo sound.

12. The non-transitory computer readable storage medium of claim 9 further comprising:

receiving, at a microphone in the WED, a voice command to an intelligent personal assistant; and

moving, by the intelligent personal assistant and in response to the voice command, the voice from one location in empty space to another location in empty space.

14. The non-transitory computer readable storage medium of claim 9 further comprising:

capturing, with the WED, an image of a room in which the user and the SLP are located;

generating, with the WED, a model of the room to determine sound reverberation of the room; and

altering, in response to the model of the room, processing of the voice to account for the sound reverberation of the room on the voice at the SLP.

15. The non-transitory computer readable storage medium of claim 9 further comprising:

capturing, with the WED, an image of a room in which the user and the SLP are located;

generating, with the WED, a model of the room to determine sound reflection of the room; and

altering, in response to the model of the room, processing of the voice to account for the sound reflection of the room on the voice at the SLP.

13. A non-transitory computer readable storage medium storing instructions that one or more electronic devices execute to provide a voice in binaural sound to a user, the method comprising:

measuring, with a wearable electronic device (WED) worn on a head of the user, a distance between eyes of the user;

selecting, based on the distance between the eyes of the user, head-related transfer functions (HRTFs) for the user;

processing, with a processor, the voice with the HRTFs to generate the voice in the binaural sound that localizes at a sound localization point (SLP) at a location in empty space at least one meter away from the head of the user;

maintaining the SLP at a fixed location with respect to the head of the user as the user moves around a room in which the user and the SLP are located such that an origin of the voice at the SLP follows the user as the user moves around the room;

displaying, with the WED and at the SLP at the location in empty space, an augmented reality image of a person that is an intelligent personal assistant;

receiving, at a microphone in the WED, a voice command directed to the intelligent personal assistant; and

moving, by the intelligent personal assistant and in response to receiving the voice command, the SLP at the fixed location with respect to the head of the user to another fixed location with respect to the head of the user.

16. A method to provide a voice in binaural sound to a user, the method comprising:

determining, from at least one of a sensor or a camera in a wearable electronic device (WED) while the WED is worn on a head of the user, a distance between eyes of the user;

selecting, based on the distance between the eyes of the user, user-specific head-related transfer functions (HRTFs) for the user;

receiving, at the WED and from the user, a body gesture that designates a sound localization point (SLP) in empty space where the voice externally localizes to the user as the binaural sound;

processing, with a digital signal processor (DSP), the voice with the user-specific HRTFs to generate the voice in the binaural sound that externally localizes at the SLP in empty space designated by the body gesture of the user;

receiving, at a microphone in the WED, a voice command to change the SLP; and

changing, in response to receiving the voice command, the SLP from externally localizing as the binaural sound to internally localizing as stereo sound.

17. The method of claim 16 further comprising:

recognizing, by a kinetic user interface in the WED, a head gesture of the user; and

altering, in response to the head gesture, the user-specific HRTFs being processed with the voice to move the voice from the SLP at the location in empty space to another location in empty space at least one meter away from the user.

18. The method of claim 16 further comprising:

receiving, with a microphone in the WED, a voice command to an intelligent personal assistant; and

moving, by the intelligent personal assistant and in response to the voice command, the SLP at the location in empty space that is above a physical object to another SLP at a location in empty space that is above another physical object where the user is located.