Automated user/sensor location recognition to customize audio performance in a distributed multi-sensor environment

• US 9,408,011 B2
• Filed: 05/21/2012
• Issued: 08/02/2016
• Est. Priority Date: 12/19/2011
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Kim, Lae-Hoon, Xiang, Pei, Visser, Erik
• Primary Examiner(s): Nguyen, Duc
• Assistant Examiner(s): BLAIR, KILE O
• Application Number:
  US13/476,815
• Publication Number:
  US 20130156198A1
• Time in Patent Office: 1,534 Days
• US Class Current: 1/1
• CPC Class Codes: G01S 3/808 using transducers spaced apart and measuring phase or time difference between signals therefrom, i.e. path-difference systems
• CPC Class Codes: H04R 1/323 for loudspeakers H04R1/34 and H04R1/40 take precedence
• CPC Class Codes: H04R 2430/20 Processing of the output signals of the acoustic transducers of an array for obtaining a desired directivity characteristic H04R2203/12 takes precedence
• CPC Class Codes: H04R 2499/13 Acoustic transducers and sound field adaptation in vehicles
• CPC Class Codes: H04R 29/002 Loudspeaker arrays
• CPC Class Codes: H04R 3/12 for distributing signals to two or more loudspeakers specially adapted for hearing aids H04R25/407
• CPC Class Codes: H04S 7/301 Automatic calibration of stereophonic sound system, e.g. with test microphone
• CPC Class Codes: H04S 7/302 Electronic adaptation of stereophonic sound system to listener position or orientation H04S7/301 takes precedence
• CPC Class Codes: H04S 7/303 Tracking of listener position or orientation

Abstract

A wireless device is provided that makes use of other nearby audio transducer devices to generate a surround sound effect for a targeted user. To do this, the wireless device first ascertains whether there are any nearby external microphones and/or loudspeaker devices. An internal microphone for the wireless device and any other nearby external microphones may be used to ascertain a location of the desired/targeted user as well as the nearby loudspeaker devices. This information is then used to generate a surround sound effect for the desired/targeted user by having the wireless device steer audio signals to its internal loudspeakers and/or the nearby external loudspeaker devices.

Claims

1. A method operational in a wireless device, comprising:

ascertaining a relative location of a target user with respect to the wireless device and a relative location of one or more external loudspeakers with respect to the wireless device, where the location of the target user relative to the wireless device is arbitrary, wherein ascertaining the relative location of the target user with respect to the wireless device comprises capturing audio signals from the target user, and wherein ascertaining the relative location of the one or more external loudspeakers with respect to the wireless device comprises obtaining information indicative of a location of one or more external devices relative to the wireless device, the information being based on an emitted audio signal that is emitted by the wireless device; and

wirelessly sending speaker-specific audio signals from the wireless device to the one or more external loudspeakers, the speaker-specific audio signals being based on the relative location of the target user with respect to the wireless device and the relative location of the one or more external loudspeakers with respect to the wireless device, each speaker-specific audio signal adapted to be played back at a corresponding external loudspeaker to achieve a surround sound experience aimed at the target user.

2. The method of claim 1, further comprising:

ascertaining the location for the one or more external devices relative to the wireless device, the one or more external devices including one or more external microphones and/or the one or more external loudspeakers.

3. The method of claim 2, wherein the audio signals from the target user are captured using one or more internal microphones.

4. The method of claim 3, wherein ascertaining the relative location of the target user with respect to the wireless device further comprises comparing captured audio from each of the one or more external microphones.

5. The method of claim 3, further comprising:

instructing the one or more external devices to cause the one or more external microphones to capture the audio signals from the target user to ascertain a direction of arrival for the captured audio signals from the target user.

6. The method of claim 2, wherein ascertaining the location for the one or more external devices relative to the wireless device comprises:

instructing the one or more external devices to cause the one or more external microphones to capture audio signals from the one or more external loudspeakers to ascertain one or more corresponding directions of arrival for the captured audio signals from the one or more external loudspeakers.

7. The method of claim 1, wherein ascertaining the relative location of the target user with respect to the wireless device includes:

ascertaining a location of the target user relative to an external device.

8. The method of claim 1, wherein the wireless device is a mobile telephone device.

9. The method of claim 1, wherein the speaker-specific audio signals from the wireless device to the one or more external loudspeakers are sent in inaudible form via a wireless communication interface.

10. A wireless device, comprising:

a communication interface to communicate with one or more local external devices; and

a processing circuit coupled to the communication interface, the processing circuit adapted to;

ascertain a relative location of a target user with respect to the wireless device and a relative location of one or more external loudspeakers with respect to the wireless device, where the location of the target user relative to the wireless device is arbitrary, wherein ascertaining the relative location of the target user with respect to the wireless device comprises capturing audio signals from the target user, and wherein ascertaining the relative location of the one or more external loudspeakers with respect to the wireless device comprises obtaining information indicative of a location of one or more external devices relative to the wireless device, the information being based on an emitted audio signal that is emitted by the wireless device; and

wirelessly send speaker-specific audio signals from the wireless device to the one or more external loudspeakers, the speaker-specific audio signals being based on the relative location of the target user with respect to the wireless device and the relative location of the one or more external loudspeakers with respect to the wireless device, each speaker-specific audio signal adapted to be played back at a corresponding external loudspeaker to achieve a surround sound experience aimed at the target user.

11. The wireless device of claim 10, further comprising:

one or more internal microphones which are also used to locate the target user relative to the wireless device; and

one or more internal loudspeakers which are also used to provide speaker-specific audio signals aimed at the target user.

12. The wireless device of claim 10, wherein the processing circuit is further adapted to:

ascertain the location for the one or more external devices relative to the wireless device, the one or more external devices including one or more external microphones and/or the one or more external loudspeakers.

13. The wireless device of claim 12, wherein the audio signals from the target user are captured using one or more internal microphones.

14. The wireless device of claim 10, wherein ascertaining the relative location of the target user with respect to the wireless device includes:

ascertaining a location of the target user relative to an external device.

15. The wireless device of claim 10, wherein the wireless device is a mobile telephone device.

16. The wireless device of claim 10, wherein the speaker-specific audio signals from the wireless device to the one or more external loudspeakers are sent in inaudible form via a wireless communication interface.

17. A wireless device, comprising:

means for communicating with one or more local external devices;

means for ascertaining a relative location of a target user with respect to the wireless device and a relative location of one or more external loudspeakers with respect to the wireless device, where the location of the target user relative to the wireless device is arbitrary, wherein the means for ascertaining the relative location of the target user with respect to the wireless device comprises means for capturing audio signals from the target user, and wherein the means for ascertaining the relative location of the one or more external loudspeakers with respect to the wireless device comprises means for obtaining information indicative of a location of one or more external devices relative to the wireless device, the information being based on an emitted audio signal that is emitted by the wireless device; and

means for wirelessly sending speaker-specific audio signals from the wireless device to the one or more external loudspeakers, the speaker-specific audio signals being based on the relative location of the target user with respect to the wireless device and the relative location of the one or more external loudspeakers with respect to the wireless device, each speaker-specific audio signal adapted to be played back at a corresponding external loudspeaker to achieve a surround sound experience aimed at the target user.

18. The wireless device of claim 17, further comprising:

means for ascertaining the location for the one or more external devices relative to the wireless device, the one or more external devices including one or more external microphones and/or the one or more external loudspeakers.

19. A non-transitory processor-readable medium having instructions stored thereon, which when executed by one or more processors causes the one or more processors to:

ascertain a relative location of a target user with respect to a wireless device and a relative location of one or more external loudspeakers with respect to the wireless device, where the location of the target user relative to the wireless device is arbitrary, wherein ascertaining the relative location of the target user with respect to the wireless device comprises capturing audio signals from the target user, and wherein ascertaining the relative location of the one or more external loudspeakers with respect to the wireless device comprises obtaining information indicative of a location of one or more external devices relative to the wireless device, the information being based on an emitted audio signal that is emitted by the wireless device; and

wirelessly send speaker-specific audio signals from the wireless device to the one or more external loudspeakers, the speaker-specific audio signals being based on the relative location of the target user with respect to the wireless device and the relative location of the one or more external loudspeakers with respect to the wireless device, each speaker-specific audio signal adapted to be played back at a corresponding external loudspeaker to achieve a surround sound experience aimed at the target user.

20. The non-transitory processor-readable medium of claim 19, further having instructions stored thereon, which when executed by one or more processors causes the one or more processors to:

ascertain the location for the one or more external devices relative to the wireless device, the one or more external devices including one or more external microphones and/or the one or more external loudspeakers.

21. A wireless device, comprising:

a communication interface to communicate with one or more local external devices; and

a processing circuit coupled to the communication interface, the processing circuit adapted to;

ascertain the presence of one or more external loudspeakers;

ascertain a relative location of a target user with respect to the one or more external loudspeakers;

ascertain a relative location of the one or more external loudspeakers with respect to the wireless device;

steer first audio signals to the one or more external loudspeakers to achieve a surround sound experience aimed at the target user; and

instruct the one or more local external devices to cause one or more external microphones to capture second audio signals from the one or more external loudspeakers to ascertain one or more corresponding directions of arrival for the captured second audio signals;

wherein ascertaining the relative location of the target user with respect to the one or more external loudspeakers is based at least in part on the one or more corresponding directions of arrival for the captured second audio signals; and

wherein ascertaining the relative location of the one or more external loudspeakers with respect to the wireless device comprises obtaining information indicative of a location of the one or more local external devices relative to the wireless device, the information being based on an emitted audio signal that is emitted by the wireless device.

22. The wireless device of claim 21, wherein the processing circuit is further adapted to:

ascertain the presence of the one or more external microphones; and

instruct the one or more local external devices to cause the one or more external microphones to capture a third audio signal from the target user to ascertain a direction of arrival for the captured third audio signal;

wherein ascertaining the relative location of the target user with respect to the one or more external loudspeakers is also based on the direction of arrival for the captured third audio signal.

23. The wireless device of claim 21, further comprising:

one or more internal microphones which are also used to locate the target user relative to the wireless device; and

one or more internal loudspeakers which are also used to steer the first audio signals aimed at the target user.

24. The wireless device of claim 21, wherein the processing circuit is further adapted to:

steer the first audio signals to one or more internal loudspeakers to achieve the surround sound experience aimed at the target user.

25. A wireless device, comprising:

means for ascertaining the presence of one or more external loudspeakers;

means for ascertaining a relative location of a target user with respect to the one or more external loudspeakers;

means for ascertaining a relative location of the one or more external loudspeakers with respect to the wireless device;

means for steering first audio signals to the one or more external loudspeakers to achieve a surround sound experience aimed at the target user; and

means for instructing one or more external devices to cause one or more external microphones to capture second audio signals from the one or more external loudspeakers to ascertain one or more corresponding directions of arrival for the captured second audio signals;

wherein ascertaining the relative location of the target user with respect to the one or more external loudspeakers is based at least in part on the one or more corresponding directions of arrival for the captured second audio signals; and

wherein the means for ascertaining the relative location of the one or more external loudspeakers with respect to the wireless device comprises means for obtaining information indicative of a location of the one or more external devices relative to the wireless device, the information being based on an emitted audio signal that is emitted by the wireless device.

26. The wireless device of claim 25, further comprising:

means for ascertaining the presence of the one or more external microphones; and

means for instructing the one or more external devices to cause the one or more external microphones to capture a third audio signal from the target user to ascertain a direction of arrival for the captured third audio signal;

wherein ascertaining the relative location of the target user with respect to the one or more external loudspeakers is also based on the direction of arrival for the captured third audio signal.

27. A non-transitory processor-readable medium having instructions stored thereon, which when executed by one or more processors causes the one or more processors to:

ascertain the presence of one or more external loudspeakers;

ascertain a relative location of a target use with respect to the one or more external loudspeakers;

ascertain a relative location of the one or more external loudspeakers with respect to a wireless device;

steer first audio signals to the one or more external loudspeakers to achieve a surround sound experience aimed at the target user; and

instruct one or more external devices to cause one or more external microphones to capture second audio signals from the one or more external loudspeakers to ascertain one or more corresponding directions of arrival for the captured second audio signals;

wherein ascertaining the relative location of the target user with respect to the one or more external loudspeakers is based at least in part on the one or more corresponding directions of arrival for the captured second audio signals; and

wherein ascertaining the relative location of the one or more external loudspeakers with respect to the wireless device comprises obtaining information indicative of a location of one or more external devices relative to the wireless device, the information being based on an emitted audio signal that is emitted by the wireless device.

28. The non-transitory processor-readable medium of claim 27, further having instructions stored thereon, which when executed by one or more processors causes the one or more processors to:

ascertain the presence of the one or more external microphones; and

instruct the one or more external devices to cause the one or more external microphones to capture a third audio signal from the target user to ascertain a direction of arrival for the captured third audio signal;

wherein ascertaining the relative location of the target user with respect to the one or more external loud speakers is also based on the direction of arrival for the captured third audio signal.