Controlling a location of binaural sound with a command

• US 10,645,512 B2
• Filed: 06/18/2019
• Issued: 05/05/2020
• Est. Priority Date: 06/23/2014
• Status: Active Grant

• Current Assignee: Glen A. Norris, Philip Scott Lyren
• Original Assignee: Glen A. Norris, Philip Scott Lyren
• Inventors: Norris, Glen A., Lyren, Philip Scott
• Primary Examiner(s): Zhu, Qin
• Application Number:
  US16/443,904
• Publication Number:
  US 20190306644A1
• Time in Patent Office: 322 Days
• Field of Search: None
• CPC Class Codes: G10L 21/028 using properties of sound source
• CPC Class Codes: H04M 1/6033 for providing handsfree use or a loudspeaker mode in telephone sets arrangements for preventing acoustic echo H04M9/08
• CPC Class Codes: H04M 1/605 involving control of the receiver volume to provide a dual operational mode at close or far distance from the user
• CPC Class Codes: H04M 1/72457 according to geographic location
• CPC Class Codes: H04M 3/53366 Message disposing or creating aspects
• CPC Class Codes: H04M 3/568 audio processing specific to telephonic conferencing, e.g. spatial distribution, mixing of participants echo suppression in two-way loud-speaking telephone systems H04M9/02; sound field processing per se H04S7/30
• CPC Class Codes: H04M 9/085 using digital techniques H04M9/082 takes precedence
• CPC Class Codes: H04R 2227/003 Digital PA systems using, e.g. LAN or internet
• CPC Class Codes: H04R 27/00 Public address systems circuits for preventing acoustic reaction H04R3/02; circuits for distributing signals to loudspeakers H04R3/12; monitoring or testing arrangements for public address systems H04R29/007; amplifiers H03F
• CPC Class Codes: H04R 5/027 Spatial or constructional arrangements of microphones, e.g. in dummy heads
• CPC Class Codes: H04R 5/033 Headphones for stereophonic communication details thereof, e.g. relating to batteries, cables or control elements H04R1/10
• CPC Class Codes: H04R 5/04 Circuit arrangements, e.g. for selective connection of amplifier inputs/outputs to loudspeakers, for loudspeaker detection, or for adaptation of settings to personal preferences or hearing impairments combinations of amplifiers H03F3/68; stereophonic systems H04S
• CPC Class Codes: H04S 1/002 Non-adaptive circuits, e.g. manually adjustable or static, for enhancing the sound image or the spatial distribution control circuits for electronic adaptation of the sound field H04S7/30
• CPC Class Codes: H04S 1/007 in which the audio signals are in digital form data reduction aspects thereof based on psychoacoustics G10L19/02
• CPC Class Codes: H04S 2400/01 Multi-channel, i.e. more than two input channels, sound reproduction with two speakers wherein the multi-channel information is substantially preserved
• CPC Class Codes: H04S 2400/11 Positioning of individual sound objects, e.g. moving airplane, within a sound field H04S2420/13 takes precedence
• CPC Class Codes: H04S 2400/15 Aspects of sound capture and related signal processing for recording or reproduction
• CPC Class Codes: H04S 2420/01 Enhancing the perception of the sound image or of the spatial distribution using head related transfer functions [HRTF's] or equivalents thereof, e.g. interaural time difference [ITD] or interaural level difference [ILD]
• CPC Class Codes: H04S 3/008 in which the audio signals are in digital form, i.e. employing more than two discrete digital channels data reduction aspects thereof based on psychoacoustics G10L19/02
• CPC Class Codes: H04S 7/303 Tracking of listener position or orientation
• CPC Class Codes: H04S 7/304 : For headphones
• CPC Class Codes: H04S 7/305 : Electronic adaptation of stereophonic audio signals to reverberation of the listening space H04S7/301 takes precedence
• CPC Class Codes: H04S 7/306 : For headphones

Abstract

A wearable electronic device (WED) worn on a head of a user displays first and second virtual images. A digital signal processor (DSP) processes sound to a first sound localization point (SLP) at the first virtual image and to a second SLP at the second virtual image. A command controls the first and second SLPs when a head orientation of the user is directed to the SLP.

Claims

1. A method executed by one or more electronic devices in an electronic system, the method comprising:

simultaneously displaying, with a display of a wearable electronic device (WED) worn on a head of a user, first and second virtual images that are spaced apart from each other;

processing, by a digital signal processor (DSP) and with one or more head-related transfer functions (HRTFs), sounds such that a first binaural sound originates to the user at a first sound localization point (SLP) in empty space at the first virtual image and a second binaural sound originates to the user at a second SLP in empty space at the second virtual image;

moving the first SLP and not the second SLP in response to a first voice command spoken by the user when a head orientation of the user is directed to the first virtual image; and

moving the second SLP and not the first SLP in response to a second voice command spoken by the user when the head orientation of the user is directed to the second virtual image.

2. The method of claim 1 further comprising:

displaying the first and second virtual images as windows, wherein the first voice command moves the first virtual image from a foreground to a background, and the second voice command moves the second virtual image from the background to the foreground.

3. The method of claim 1 further comprising:

displaying the first virtual image as a first screen that plays a movie and the second virtual image as a second screen that shows a game, wherein the first binaural sound from the movie externally localizes to the first SLP in empty space at the first image, and the second binaural sound from the game externally localizes to the second SLP in empty space at the second image.

4. The method of claim 1 further comprising:

sensing, with the WED, a command that is a body gesture from the user; and

moving, by the WED and in response to sensing the body gesture from the user, the SLP in empty space to another location in empty space.

5. The method of claim 1 further comprising:

sensing, with the WED, a command that is a head gesture from the user; and

moving, by the WED and in response to sensing the head gesture from the user, the SLP in empty space to another location in empty space.

6. The method of claim 1 further comprising:

executing a command received at the WED and from the user that moves the SLP in empty space of the binaural sound playing to the user to a foreground that is closer to the user.

7. The method of claim 1 further comprising:

executing a command received at the WED and from the user that moves the SLP in empty space of the binaural sound playing to the user to a background that is farther away from the user.

8. A non-transitory computer readable storage medium storing instructions that one or more electronic devices execute, the method comprising:

displaying, with a wearable electronic device (WED) worn on a head of a user, a first virtual image and a second virtual image that are spaced apart from each other in an environment of the user;

processing, by a digital signal processor (DSP), sounds such that a first binaural sound originates to the user at a first sound localization point (SLP) in empty space at the first virtual image and a second binaural sound originates to the user at a second SLP in empty space at the second virtual image;

moving, in response to a first command, the first SLP but not the second SLP when a head orientation of the user is directed toward the first virtual image when the user issues the first command; and

moving, in response to a second command, the second SLP but not the first SLP when the head orientation of the user is directed toward the second virtual image when the user issues the second command.

9. The non-transitory computer readable storage medium of claim 8 in which the method further comprises:

receiving, at the WED and from a handheld portable electronic device (HPED) held in a hand of the user, a wireless transmission that includes the binaural sound processed by the DSP located in the HPED.

10. The non-transitory computer readable storage medium of claim 8, wherein the first and second binaural sounds are voices, the first command moves the first SLP to a third SLP in empty space, and the second command moves the second SLP to a fourth SLP in empty space.

11. The non-transitory computer readable storage medium of claim 8, wherein the first virtual image is a person to a telephone call with the user, and the second virtual image is a screen playing video and the second binaural sound is audio of the video.

12. The non-transitory computer readable storage medium of claim 8, wherein the first virtual image is an intelligent personal assistant, and second virtual image is a person in a telephone call with the user.

13. The non-transitory computer readable storage medium of claim 8 in which the method further comprises:

receiving, at the WED and from the user, a designation to set a default location of an SLP for the binaural sound, wherein the default location is at a portable electronic device in a hand of the user.

14. A method comprising:

displaying, with a wearable electronic device (WED) worn on a head of a user, a first virtual image and a second virtual image that are spaced apart from each other;

processing, by a digital signal processor (DSP), a first binaural sound that has a first sound localization point (SLP) in empty space at coordinates of the first virtual image and a second binaural sound that has a second SLP in empty space at coordinates of the second virtual image; and

controlling the first and second SLPs with commands from the user by executing a first command that controls the first SLP and not the second SLP when a head orientation of the user is directed toward the first virtual image and by executing a second command that controls the second SLP and not the first SLP when the head orientation of the user is directed toward the second virtual image.

15. The method of claim 14 further comprising:

receiving, at the WED, a command that is a head gesture from the user; and

moving, in response to the command that is the head gesture from the user, the first SLP in empty space from one location outside the head of the user to another location outside the head of the user.

16. The method of claim 14 further comprising:

receiving, at the WED, a command that is a body gesture from the user; and

moving, in response to the command that is the body gesture from the user, the first SLP in empty space from a foreground to a background.

17. The method of claim 14 further comprising:

changing, by the WED, a location of the first SLP when the WED detects the first SLP conflicts with a location of a physical object.

18. The method of claim 14 further comprising:

altering, by the DSP and in response to a command from the user, processing of the first binaural sound to move the first SLP to a different location in empty space outside the head of the user, wherein the command is a head gesture from the head of the user and the first binaural sound is music.

19. The method of claim 14, wherein the first command moves the first SLP from one location in empty space to another location in empty space outside the head of the user.

20. The method of claim 14, wherein the first command moves the first SLP from one side of the head of the user to another side of the head of the user.