Contextualization of voice inputs

• US 10,134,399 B2
• Filed: 07/15/2016
• Issued: 11/20/2018
• Est. Priority Date: 07/15/2016
• Status: Active Grant

• Current Assignee: Sonos, Inc.
• Original Assignee: Sonos, Inc.
• Inventors: Lang, Jonathan P., Kadri, Romi, Butts, Christopher
• Primary Examiner(s): Guerra-Erazo, Edgar X
• Application Number:
  US15/211,689
• Publication Number:
  US 20180018967A1
• Time in Patent Office: 858 Days
• Field of Search: 704235, 704246, 704270, 7042701, 704275
• CPC Class Codes: G01S 5/18 using ultrasonic, sonic, or infrasonic waves
• CPC Class Codes: G06F 3/165 Management of the audio stream, e.g. setting of volume, audio stream path
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: G10L 17/22 Interactive procedures; Man-machine interfaces
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 2015/226 using non-speech characteristics
• CPC Class Codes: G10L 2015/228 of application context
• CPC Class Codes: H05B 47/165 following a pre-assigned programmed sequence; Logic control [LC]

Abstract

Disclosed herein are example techniques to provide contextual information corresponding to a voice command. An example implementation may involve receiving voice data indicating a voice command, receiving contextual information indicating a characteristic of the voice command, and determining a device operation corresponding to the voice command. Determining the device operation corresponding to the voice command may include identifying, among multiple zones of a media playback system, a zone that corresponds to the characteristic of the voice command, and determining that the voice command corresponds to one or more particular devices that are associated with the identified zone. The example implementation may further involve causing the one or more particular devices to perform the device operation.

Claims

1. Tangible, non-transitory, computer-readable media having instructions encoded therein, wherein the instructions, when executed by one or more processors, cause a first networked microphone device (NMD) device to perform a method comprising:

recording, via a microphone array of the first NMD, audio data indicating a voice command;

identifying, based on the recorded audio data, a first characteristic of the voice command, the first characteristic comprising a sound pressure level of the voice command as detected by the microphone array of the first NMD, wherein the first NMD is associated with a first zone of a media playback system, the first zone comprising a first playback device;

receiving, via a network interface of the first NMD from one or more second NMDs, contextual information indicating second characteristics of the voice command, the second characteristics comprising respective sound pressure levels of the voice command as detected by respective microphone arrays of the one or more second NMDs, wherein the one or more second NMDs are associated with one or more second zones of the media playback system, each second zone comprising a second playback device;

based on the sound pressure level of the voice command as detected by the microphone array of the first NMD being greater than the sound pressure levels of the voice command as detected by the respective microphone arrays of the one or more second NMDs, determining that the voice command was uttered in the first zone;

in response to determining that the voice command was uttered in the first zone associated with the first NMD, querying, via the network interface, one or more servers of a voice assistant service with the voice command;

receiving, via the network interface in response to the query, a playback command corresponding to the voice command; and

instructing the first playback device to play back audio content according to the playback command via one or more amplifiers configured to drive one or more speakers.

9. The tangible, non-transitory, computer-readable media of claim 1, wherein the first NMD comprises the first playback device.

10. The tangible, non-transitory, computer-readable media of claim 9, wherein the first zone is configured into a zone group with a second zone, and wherein the method further comprises playing back the audio content according to the playback command in synchrony with one or more playback devices of the second zone.

11. The tangible, non-transitory, computer-readable media of claim 1, wherein the first NMD is connected to the first playback device via a local area network, and wherein instructing the first playback device to play back audio content according to the playback command via one or more amplifiers configured to drive one or more speakers comprises sending, via the network interface, an instruction to play back audio content according to the playback command.

12. The tangible, non-transitory, computer-readable media of claim 1, wherein the method further comprises:

determining that the sound pressure level of the voice command as detected by the microphone array of the first NMD has a higher magnitude than the sound pressure levels of the voice command as detected by the respective microphone arrays of the one or more second NMDs.

13. The tangible, non-transitory, computer-readable media of claim 1, wherein the playback command comprises a command to play back particular audio content in the first zone and a particular second zone, and wherein the method further comprises:

instructing a particular second playback device of the particular second zone to play back the audio content according to the playback command in synchrony with playback of the audio content by the first playback device of the first zone.

14. The tangible, non-transitory, computer-readable media of claim 1, wherein the method further comprises:

detecting, within the recorded audio data, a wake-word preceding the voice command; and

identifying a portion of the recorded audio data following the wake-word as the voice command.

2. A first networked microphone device (NMD), the first NMD comprising:

a microphone array;

a network interface;

one or more processors; and

computer-readable media having instructions encoded therein, wherein the instructions, when executed by the one or more processors, cause the first NMD device to perform functions comprising;

recording, via the microphone array, audio data indicating a voice command;

identifying, based on the recorded audio data, a first characteristic of the voice command, the first characteristic comprising a sound pressure level of the voice command as detected by the microphone array of the first NMD, wherein the first NMD is associated with a first zone of a media playback system, the first zone comprising a first playback device;

receiving, via a network interface of the first NMD from one or more second NMDs, contextual information indicating a second characteristic of the voice command, the second characteristics comprising respective sound pressure levels of the voice command as detected by respective microphone arrays of the one or more second NMDs, wherein the one or more second NMDs are associated with one or more second zones of the media playback system, each second zone comprising a second playback device;

based on the sound pressure level of the voice command as detected by the microphone array of the first NMD being greater than the sound pressure levels of the voice command as detected by the respective microphone arrays of the one or more second NMDs, determining that the voice command was uttered in the first zone;

in response to determining that the voice command was uttered in the first zone associated with the first NMD, querying one or more servers of a voice assistant service with the voice command;

receiving, via the network interface in response to the query, a playback command corresponding to the voice command; and

instructing the first playback device to play back audio content according to the playback command via one or more amplifiers configured to drive one or more speakers.

15. The first NMD of claim 2, wherein the first NMD comprises the first playback device.

16. The first NMD of claim 15, wherein the first zone is configured into a zone group with a second zone, and wherein the functions further comprise playing back the audio content according to the playback command in synchrony with one or more playback devices of the second zone.

17. The first NMD of claim 2, wherein the first NMD is connected to the first playback device via a local area network, and wherein instructing the first playback device to play back audio content according to the playback command via one or more amplifiers configured to drive one or more speakers comprises sending, via the network interface, an instruction to play back audio content according to the playback command.

18. The first NMD of claim 2, wherein the functions further comprise:

determining that the sound pressure level of the voice command as detected by the microphone array of the first NMD has a higher magnitude than the sound pressure levels of the voice command as detected by the respective microphone arrays of the one or more second NMDs.

19. The first NMD of claim 2, wherein the playback command comprises a command to play back particular audio content in the first zone and a particular second zone, and wherein the functions further comprise:

instructing a particular second playback device of the particular second zone to play back the audio content according to the playback command in synchrony with playback of the audio content by the first playback device of the first zone.

20. The first NMD of claim 2, wherein the functions further comprise:

detecting, within the recorded audio data, a wake-word preceding the voice command; and

identifying a portion of the recorded audio data following the wake-word as the voice command.

3. A method comprising:

recording, via a microphone array of a first networked microphone device (NMD), audio data indicating a voice command;

identifying, based on the recorded audio data, a first characteristic of the voice command, the first characteristic comprising a sound pressure level of the voice command as detected by the microphone array of the first NMD, wherein the first NMD is associated with a first zone of a media playback system, the first zone comprising a first playback device;

receiving, via a network interface of the first NMD from one or more second NMDs, contextual information indicating a second characteristic of the voice command, the second characteristics comprising respective sound pressure levels of the voice command as detected by respective microphone arrays of the one or more second NMDs, wherein the one or more second NMDs are associated with one or more second zones of the media playback system, each second zone comprising a second playback device;

based on the sound pressure level of the voice command as detected by the microphone array of the first NMD being greater than the sound pressure levels of the voice command as detected by the respective microphone arrays of the one or more second NMDs, determining that the voice command was uttered in the first zone;

in response to determining that the voice command was uttered in the first zone associated with the first NMD, querying, via the network interface, one or more servers of a voice assistant service with the voice command;

receiving, via the network interface in response to the query, a playback command corresponding to the voice command; and

instructing the first playback device to play back audio content according to the playback command via one or more amplifiers configured to drive one or more speakers.

4. The method of claim 3, wherein the first NMD comprises the first playback device.

5. The method of claim 4, wherein the first zone is configured into a zone group with a second zone, and wherein the method further comprises playing back the audio content according to the playback command in synchrony with one or more playback devices of the second zone.

6. The method of claim 3, wherein the first NMD is connected to the first playback device via a local area network, and wherein instructing the first playback device to play back audio content according to the playback command via one or more amplifiers configured to drive one or more speakers comprises sending, via the network interface, an instruction to play back audio content according to the playback command.

7. The method of claim 3, wherein the method further comprises:

determining that the sound pressure level of the voice command as detected by the microphone array of the first NMD has a higher magnitude than the sound pressure levels of the voice command as detected by the respective microphone arrays of the one or more second NMDs.

8. The method of claim 3, wherein the playback command comprises a command to play back particular audio content in the first zone and a particular second zone, and wherein the method further comprises:

instructing a particular second playback device of the particular second zone to play back the audio content according to the playback command in synchrony with playback of the audio content by the first playback device of the first zone.