Sound localization for user in motion

• US 9,037,468 B2
• Filed: 05/23/2012
• Issued: 05/19/2015
• Est. Priority Date: 10/27/2008
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment Inc. (Sony Group Corp.)
• Original Assignee: Sony Computer Entertainment Incorporated (Sony Group Corp.)
• Inventors: Osman, Steven
• Primary Examiner(s): Jackson, Jakieda
• Application Number:
  US13/478,582
• Publication Number:
  US 20130041648A1
• Time in Patent Office: 1,091 Days
• Field of Search: 704/270
• US Class Current: 704/270
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: H04S 2400/11 Positioning of individual sound objects, e.g. moving airplane, within a sound field H04S2420/13 takes precedence
• CPC Class Codes: H04S 2420/01 Enhancing the perception of the sound image or of the spatial distribution using head related transfer functions [HRTF's] or equivalents thereof, e.g. interaural time difference [ITD] or interaural level difference [ILD]
• CPC Class Codes: H04S 7/302 Electronic adaptation of stereophonic sound system to listener position or orientation H04S7/301 takes precedence
• CPC Class Codes: H04S 7/304 For headphones

Abstract

Methods, apparatus, and computer programs for simulating the source of sound are provided. One method includes operations for determining a location in space of the head of a user utilizing face recognition of images of the user. Further, the method includes an operation for determining a sound for two speakers, and an operation for determining an emanating location in space for the sound, each speaker being associated with one ear of the user. The acoustic signals for each speaker are established based on the location in space of the head, the sound, the emanating location in space, and the auditory characteristics of the user. In addition, the acoustic signals are transmitted to the two speakers. When the acoustic signals are played by the two speakers, the acoustic signals simulate that the sound originated at the emanating location in space.

Claims

1. A method for simulating a source of sound, the method comprising:

determining, with a first device, a first location of a head of a first user;

receiving at the first device a second location corresponding to a head of a second user, wherein the second location is determined with a second device;

determining, by the first device, a third location corresponding to a location of the second device with reference to the first device;

establishing acoustic signals for a plurality of speakers based on the first location, the second location, the third location, and auditory characteristics of the first user; and

transmitting the acoustic signals from the first device to the plurality of speakers, wherein the acoustic signals when played by the plurality of speakers simulate that a sound associated with the acoustic signals originated at the second user.

2. The method as recited in claim 1, further including:

receiving speech created by the second user; and

setting the sound associated with the acoustic signals to be the received speech.

3. The method as recited in claim 1, further including:

receiving speech created by the second user;

translating the speech to a different language; and

setting the sound associated with the acoustic signals to be the translated speech.

4. The method as recited in claim 1 further including:

sending the first location and the third location to the second device.

5. The method as recited in claim 1, further including:

establishing updated acoustic signals for the plurality of speakers based on the first location, the third location, and the auditory characteristics of the first user, wherein the auditory characteristics characterize how a physical anatomy of a listener receives sound from a point in space, wherein establishing the updated acoustic signals further includes localizing the updated acoustic signals based on the physical anatomy of the user; and

transmitting the updated acoustic signals to the plurality of speakers, wherein the updated acoustic signals when played by the plurality of speakers simulate that a sound associated with the second device originated at the second device.

6. The method as recited in claim 1, wherein determining the third location further includes:

determining the third location using a global positioning system (GPS).

7. The method as recited in claim 1, wherein determining the first location further includes:

receiving the first location of the head of the first user from a global positioning system (GPS) module situated in headphones worn by the first user.

8. The method as recited in claim 1, wherein the first user is holding a controller whose location is trackable by the first device.