Audio playback settings for voice interaction

• US 10,582,322 B2
• Filed: 04/05/2018
• Issued: 03/03/2020
• Est. Priority Date: 09/27/2016
• Status: Active Grant

• Current Assignee: Sonos, Inc.
• Original Assignee: Sonos, Inc.
• Inventors: Hartung, Klaus, Kadri, Romi
• Primary Examiner(s): Nguyen, Quynh H
• Application Number:
  US15/946,585
• Publication Number:
  US 20180332417A1
• Time in Patent Office: 698 Days
• Field of Search: 381110
• CPC Class Codes: G06F 3/165 Management of the audio stream, e.g. setting of volume, audio stream path
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/088 Word spotting
• CPC Class Codes: G10L 25/84 for discriminating voice from noise
• CPC Class Codes: H03G 3/32 the control being dependent upon ambient noise level or sound level
• CPC Class Codes: H03G 3/342 Muting when some special characteristic of the signal is sensed which distinguishes it from noise, e.g. using speech detector H03G3/344, H03G3/345 take precedence
• CPC Class Codes: H04R 2227/003 Digital PA systems using, e.g. LAN or internet
• CPC Class Codes: H04R 2227/005 Audio distribution systems for home, i.e. multi-room use
• CPC Class Codes: H04R 2420/07 Applications of wireless loudspeakers or wireless microphones
• CPC Class Codes: H04R 2430/01 Aspects of volume control, not necessarily automatic, in sound systems
• CPC Class Codes: H04R 27/00 Public address systems circuits for preventing acoustic reaction H04R3/02; circuits for distributing signals to loudspeakers H04R3/12; monitoring or testing arrangements for public address systems H04R29/007; amplifiers H03F
• CPC Class Codes: H04R 29/007 for public address systems public address systems per se H04R27/00

Abstract

Example techniques relate to voice interaction in an environment with a media playback system that is playing back audio content. In an example implementation, while playing back first audio in a given environment at a given loudness: a playback device (a) detects that an event is anticipated in the given environment, the event involving playback of second audio and (b) determines a loudness of background noise in the given environment, the background noise comprising ambient noise in the given environment. The playback device ducks the first audio in proportion to a difference between the given loudness of the first audio and the determined loudness of the background noise and plays back the ducked first audio concurrently with the second audio.

Claims

1. A playback device comprising:

a network interface;

one or more microphones;

an audio stage comprising an amplifier;

one or more speakers;

one or more processors;

a housing, the housing carrying at least the network interface, the one or more microphones, the audio stage, the one or more speakers, the one or more processors, and a computer-readable media having stored therein instructions executable by the one or more processors to cause the playback device to perform operations comprising;

while playing back first audio in a given environment at a given loudness via the audio stage and the one or more speakers;

(a) recording, via the one or more microphones, audio into a buffer;

(b) detecting, within the recorded audio, a wake word to invoke a voice assistant service;

(c) in response to detecting the wake word;

(i) ducking the first audio while recording, into the buffer, audio representing a voice input to the voice assistant service and (ii) sending, via the network interface to one or more servers of the voice assistant service, the recorded audio in the buffer representing the voice input to the voice assistant service

(d) receiving, via the network interface from the one or more servers of the voice assistant service in response to the voice input, second audio representing a spoken response to the voice input; and

in response to receiving the second audio representing the spoken response to the voice input, ducking the first audio while playing back the ducked first audio concurrently with the second audio representing the spoken response to the voice input via the audio stage and the one or more speakers.

2. The playback device of claim 1, wherein ducking the first audio while recording audio representing the voice input to the voice assistant service comprises ducking the first audio to a first volume level, and wherein ducking the first audio while playing back the ducked first audio concurrently with the second audio representing the spoken response to the voice input comprises ducking the first audio to a second volume level that is different from the first level.

3. The playback device of claim 2, wherein the operations further comprise determining a loudness of background noise in the given environment, and wherein ducking the first audio to the second volume level that is different from the first level comprises ducking the first audio to a particular volume level that is based on the loudness of background noise in the given environment.

4. The playback device of claim 3, wherein ducking the first audio to the particular volume level that is based on the loudness of background noise in the given environment comprises ducking the first audio in proportion to a difference between the given loudness of the first audio and the determined loudness of the background noise.

5. The playback device of claim 3, wherein playing back the ducked first audio concurrently with the second audio comprises:

determining that a difference between a loudness of the ducked first audio and a given dynamic range is (a) less than the determined loudness of the background noise or (b) greater than the determined loudness of the background noise;

when the determined difference between the loudness of the ducked first audio and the given dynamic range is less than the determined loudness of the background noise, compressing the first audio to a dynamic range that is louder than the determined loudness of the background noise and playing back the compressed first audio; and

when the determined difference between the loudness of the ducked first audio and the given dynamic range is greater than the determined loudness of the background noise, playing back the first audio without compression.

6. The playback device of claim 3, wherein determining the loudness of background noise in the given environment comprises measuring the loudness of the background noise in the given environment via one or more microphones.

7. The playback device of claim 6, wherein at least one of the one or more microphones is housed in a networked microphone device that is distinct from the playback device, and wherein measuring the loudness of the background noise in the given environment via one or more microphones comprises causing the networked microphone device to measure the loudness of the background noise in the given environment.

8. The playback device of claim 7, wherein measuring the loudness of the background noise in the given environment comprises offsetting the first audio being played back by the playback device from the measurement of the background noise in the given environment.

9. The playback device of claim 1, wherein playing back the ducked first audio concurrently with the second audio comprises:

detecting that a signal-to-noise ratio within the given environment is below a voice input threshold; and

responsively, filtering the first audio, wherein filtering the first audio comprises cutting the first audio in a frequency range corresponding to human speech.

10. The playback device of claim 9, wherein filtering the first audio further comprises boosting the first audio outside the frequency range corresponding to human speech.

11. The playback device of claim 1, wherein the playback device is a first playback device of a group of playback device that includes one or more second playback devices, and wherein playing back the ducked first audio concurrently with the second audio comprises playing back the ducked first audio concurrently with the second audio in synchrony with the one or more second playback devices.

12. The playback device of claim 1, wherein the operations further comprise detecting that the spoken response to the voice input has been played back, and wherein ducking the first audio comprises ducking the first audio until the spoken response to the voice input has been played back.

13. A tangible, non-transitory computer-readable medium having stored therein instructions executable by one or more processors to cause a playback device to perform a method, the playback device comprising a housing carrying at least a network interface, one or more microphones, an audio stage, one or more speakers, and the one or more processors, and the method comprising:

while playing back first audio in a given environment at a given loudness via the audio stage and the one or more speakers;

(a) recording, via the one or more microphones, audio into a buffer;

(b) detecting, within the recorded audio, a wake word to invoke a voice assistant service;

(c) in response to detecting the wake word;

(i) ducking the first audio while recording, into the buffer, audio representing a voice input to the voice assistant service and (ii) sending, via the network interface to one or more servers of the voice assistant service, the recorded audio in the buffer representing the voice input to the voice assistant service

(d) receiving, via the network interface from the one or more servers of the voice assistant service in response to the voice input, second audio representing a spoken response to the voice input; and

in response to receiving the second audio representing the spoken response to the voice input, ducking the first audio while playing back the ducked first audio concurrently with the second audio representing the spoken response to the voice input via the audio stage and the one or more speakers.

14. The tangible, non-transitory computer-readable medium of claim 13, wherein ducking the first audio while recording audio representing the voice input to the voice assistant service comprises ducking the first audio to a first volume level, and wherein ducking the first audio while playing back the ducked first audio concurrently with the second audio representing the spoken response to the voice input comprises ducking the first audio to a second volume level that is different from the first level.

15. The tangible, non-transitory computer-readable medium of claim 14, wherein the method further comprises determining a loudness of background noise in the given environment, and wherein ducking the first audio to the second volume level that is different from the first level comprises ducking the first audio to a particular volume level that is based on the loudness of background noise in the given environment.

16. The tangible, non-transitory computer-readable medium of claim 15, wherein ducking the first audio to the particular volume level that is based on the loudness of background noise in the given environment comprises ducking the first audio in proportion to a difference between the given loudness of the first audio and the determined loudness of the background noise.

17. The tangible, non-transitory computer-readable medium of claim 15, wherein playing back the ducked first audio concurrently with the second audio comprises:

determining that a difference between a loudness of the ducked first audio and a given dynamic range is (a) less than the determined loudness of the background noise or (b) greater than the determined loudness of the background noise;

when the determined difference between the loudness of the ducked first audio and the given dynamic range is less than the determined loudness of the background noise, compressing the first audio to a dynamic range that is louder than the determined loudness of the background noise and playing back the compressed first audio; and

when the determined difference between the loudness of the ducked first audio and the given dynamic range is greater than the determined loudness of the background noise, playing back the first audio without compression.

18. A method to be performed by a playback device comprising a housing carrying at least a network interface, one or more microphones, an audio stage, one or more speakers, the method comprising:

while playing back first audio in a given environment at a given loudness via the audio stage and the one or more speakers;

(a) recording, via the one or more microphones, audio into a buffer;

(b) detecting, within the recorded audio, a wake word to invoke a voice assistant service;

(c) in response to detecting the wake word;

(i) ducking the first audio while recording, into the buffer, audio representing a voice input to the voice assistant service and (ii) sending, via the network interface to one or more servers of the voice assistant service, the recorded audio in the buffer representing the voice input to the voice assistant service

(d) receiving, via the network interface from the one or more servers of the voice assistant service in response to the voice input, second audio representing a spoken response to the voice input; and

in response to receiving the second audio representing the spoken response to the voice input, ducking the first audio while playing back the ducked first audio concurrently with the second audio representing the spoken response to the voice input via the audio stage and the one or more speakers.

19. The method of claim 18, wherein ducking the first audio while recording audio representing the voice input to the voice assistant service comprises ducking the first audio to a first volume level, and wherein ducking the first audio while playing back the ducked first audio concurrently with the second audio representing the spoken response to the voice input comprises ducking the first audio to a second volume level that is different from the first level.

20. The method of claim 19, wherein the method further comprises determining a loudness of background noise in the given environment, and wherein ducking the first audio to the second volume level that is different from the first level comprises ducking the first audio comprises ducking the first audio in proportion to a difference between the given loudness of the first audio and the determined loudness of the background noise.