Systems and methods of user localization

• US 10,277,981 B1
• Filed: 10/02/2018
• Issued: 04/30/2019
• Est. Priority Date: 10/02/2018
• Status: Active Grant

• Current Assignee: Sonos, Inc.
• Original Assignee: Sonos, Inc.
• Inventors: Frank, Eric
• Primary Examiner(s): Anwah, Olisa
• Application Number:
  US16/149,992
• Time in Patent Office: 210 Days
• CPC Class Codes: G01S 15/06 Systems determining the position data of a target
• CPC Class Codes: G01S 15/88 Sonar systems specially adapted for specific applications seismic or acoustic prospecting or detecting G01V1/00
• CPC Class Codes: G01S 5/18 using ultrasonic, sonic, or infrasonic waves
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/28 Constructional details of speech recognition systems
• CPC Class Codes: H04R 1/406 microphones
• CPC Class Codes: H04R 2205/021 Aspects relating to docking-station type assemblies to obtain an acoustical effect, e.g. the type of connection to external loudspeakers or housings, frequency improvement
• CPC Class Codes: H04R 2227/003 Digital PA systems using, e.g. LAN or internet
• CPC Class Codes: H04R 2227/005 Audio distribution systems for home, i.e. multi-room use
• CPC Class Codes: H04R 2410/00 Microphones
• CPC Class Codes: H04R 2430/23 Direction finding using a sum-delay beam-former
• CPC Class Codes: H04R 27/00 Public address systems circuits for preventing acoustic reaction H04R3/02; circuits for distributing signals to loudspeakers H04R3/12; monitoring or testing arrangements for public address systems H04R29/007; amplifiers H03F
• CPC Class Codes: H04R 3/005 for combining the signals of two or more microphones specially adapted for hearing aids H04R25/407
• CPC Class Codes: H04R 5/04 Circuit arrangements, e.g. for selective connection of amplifier inputs/outputs to loudspeakers, for loudspeaker detection, or for adaptation of settings to personal preferences or hearing impairments combinations of amplifiers H03F3/68; stereophonic systems H04S

Abstract

Systems and methods are disclosed in which a playback device transmits a first sound signal including a predetermined waveform. In one example, the playback device receives a second sound signal including at least one reflection of the first sound signal. The second sound signal is processed to determine a location of a person relative to the playback device, and a characteristic of audio reproduction by the playback device is selected, based on the determined location of the person.

Claims

1. A method comprising:

transmitting, by an electroacoustic transducer of a playback device, a first sound signal comprising a predetermined waveform;

receiving, by a microphone in the playback device, a second sound signal comprising at least one reflection of the first sound signal;

processing the second sound signal to determine a location of a person relative to the playback device, wherein processing the second sound signal comprises disregarding a portion of the second sound signal corresponding to a self-response of the playback device; and

selecting a characteristic of audio reproduction by the playback device based on the determined location of the person.

2. The method of claim 1, wherein the first sound signal is ultrasonic.

3. The method of claim 1, comprising performing, by the playback device, audio reproduction from a media source simultaneously with transmitting the first sound signal.

4. The method of claim 1, wherein the characteristic is a volume of the audio reproduction.

5. The method of claim 1, wherein the characteristic is a particular one or more channels for audio reproduction by the playback device from a media source.

6. The method of claim 1, wherein the determined location is a first determined location, the method comprising:

transmitting, by the electroacoustic transducer of the playback device, a third sound signal comprising the predetermined waveform;

receiving, by the microphone in the playback device, a fourth sound signal comprising at least one reflection of the third sound signal;

processing the further fourth signal to determine a second location of a person relative to the playback device, wherein the second location is different from the first location; and

adjusting the characteristic of audio reproduction by the playback device based on the determined second location of the person.

7. The method of claim 1, comprising:

transmitting, by the transducer of the playback device when a person is at a known location relative to the playback device, a third sound signal comprising the predetermined waveform;

receiving, by the microphone in the playback device, a fourth sound signal comprising at least one reflection of the third sound signal; and

storing location data comprising data indicative of the fourth sound signal and an association between the fourth sound signal and the known location,

wherein processing the second sound signal to determine the location of the person relative to the playback device comprises;

determining, using the location data, that the location of a person relative to the playback device is the known location based on a correspondence between the second sound signal and the fourth sound signal.

8. The method of claim 7, comprising:

before transmitting the third sound signal, determining an orientation of the playback device;

wherein

in the storing location data, the fourth sound signal is associated with the determined orientation of the playback device.

9. The method of claim 1, wherein processing the second sound signal comprises identifying a reflection of the predetermined waveform in the second sound signal.

10. The method of claim 1, comprising:

receiving, by the microphone in the playback device, an audio signal including a spoken input; and

associating the spoken input with the determined location of the person.

11. The method of claim 10, wherein the spoken input includes a voice command to control a further device, the method comprising:

sending a control signal from the playback device to the further device dependent on the determined location of the person.

12. The method of claim 1, comprising receiving a fifth sound signal by a further microphone in the playback device, the fifth sound signal comprising at least one reflection of the first sound signal,

wherein processing the second sound signal comprises:

determining a delay between the second sound signal and the fifth sound signal; and

determining, using the determined delay, an angular component of the first location from the playback device.

13. The method of claim 1, wherein the playback device is a first playback device and is bonded with a second playback device so as to synchronously perform audio reproduction with the second playback device, the method comprising selecting a characteristic of audio reproduction by the second playback device based on the determined location of the person.

14. The method of claim 13, wherein the characteristic of audio reproduction by the second playback device is a delay of the audio reproduction by the second playback device relative to the audio reproduction by the first playback device.

15. A playback device comprising:

a transducer configured to generate audio signals;

a microphone;

a network interface; and

a processing system, wherein;

the transducer is arranged to transmit a first sound signal comprising a predetermined waveform;

the microphone is arranged to receive a second sound signal comprising at least one reflection of the first ultrasound signal; and

the processing system is arranged to;

transmit, using the network interface, data indicative of the second sound signal to a computing system using the network interface;

receive, using the network interface, data indicative of the location of a person relative to the playback device from the computing system, thereby to determine a location of a person relative to the playback device based on the second sound signal; and

set a characteristic of audio reproduction by the playback device based on the determined location of the person.

16. The playback device of claim 15, wherein:

the microphone is arranged to receive an audio signal including a spoken input; and

the processor is arranged to associate the spoken input with the determined location of the person.

17. The playback device of claim 15, comprising a microphone array, the microphone being part of the microphone array, and wherein the second sound signal comprises at least two second sound signals from respective microphones within the microphone array;

and wherein the processing system is arranged to determine an angular component of the determined location of a person based on the at least two second sound signals.

18. A non-transitory computer readable medium comprising computer program instructions which, when executed by a processing system, instruct the processing system to:

cause an electroacoustic transducer in a playback device to transmit a first sound signal comprising a predetermined waveform;

cause a microphone in the playback device to receive a second sound signal comprising at least one reflection of the first sound signal;

process the second sound signal to determine a location of a person relative to the playback device, wherein processing the second sound signal comprises disregarding a portion of the second sound signal corresponding to a self-response of the playback device; and

set a characteristic of audio reproduction by the playback device based on the determined location of the person.