Dynamic player selection for audio signal processing

• US 10,714,115 B2
• Filed: 06/18/2019
• Issued: 07/14/2020
• Est. Priority Date: 06/09/2016
• Status: Active Grant

• Current Assignee: Sonos, Inc.
• Original Assignee: Sonos, Inc.
• Inventors: Shih, Shao-Fu
• Primary Examiner(s): Kurr, Jason R
• Application Number:
  US16/444,975
• Publication Number:
  US 20190304481A1
• Time in Patent Office: 392 Days
• CPC Class Codes: G06F 3/162 Interface to dedicated audio devices, e.g. audio drivers, interface to CODECs
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 2021/02082 the noise being echo, reverberation of the speech
• CPC Class Codes: G10L 2021/02166 Microphone arrays; Beamforming
• CPC Class Codes: G10L 21/0208 Noise filtering
• CPC Class Codes: G10L 21/0216 characterised by the method used for estimating noise
• CPC Class Codes: G10L 21/0264 characterised by the type of parameter measurement, e.g. correlation techniques, zero crossing techniques or predictive techniques
• CPC Class Codes: H04R 2227/003 Digital PA systems using, e.g. LAN or internet
• CPC Class Codes: H04R 2227/005 Audio distribution systems for home, i.e. multi-room use
• CPC Class Codes: H04R 2227/009 Signal processing in [PA] systems to enhance the speech intelligibility
• CPC Class Codes: H04R 27/00 Public address systems circuits for preventing acoustic reaction H04R3/02; circuits for distributing signals to loudspeakers H04R3/12; monitoring or testing arrangements for public address systems H04R29/007; amplifiers H03F
• CPC Class Codes: H04R 3/005 for combining the signals of two or more microphones specially adapted for hearing aids H04R25/407
• CPC Class Codes: H04W 24/02 Arrangements for optimising operational condition

Abstract

A set of signal measures is sent, wherein each signal measure in the set of signal measures corresponds to a respective audio signal received by a playback device in a media playback system and is processed based on a first set of audio processing algorithms. A plurality of signal measures is identified in the set of signal measures. Audio signals corresponding to the identified plurality of signal measures are processed by one or more devices in the media playback system to improve a signal measure of each of the audio signals. The audio signals are processed based on a second set of audio processing algorithms. The processed audio signals are combined into a combined audio signal.

Claims

1. A media system comprising a first playback device and a second playback device, the media system comprising:

the first playback device comprising;

one or more microphones;

a network interface;

a processor;

a non-transitory computer-readable medium;

program instructions stored on the non-transitory computer-readable medium that, when executed by the processor, cause the first playback device to perform functions comprising;

receiving an indication of an available amount of computational power of the second playback device; and

based on the received indication of the available amount of computational power of the second playback device, sending, to the second playback device, a set of audio signals received by the one or more microphones of the first playback device, wherein the set of audio signals includes at least one voice input;

the second playback device comprising;

a network interface;

a processor;

a non-transitory computer-readable medium;

program instructions stored on the non-transitory computer-readable medium that, when executed by the processor, cause the second playback device to perform functions comprising;

receiving, from the first playback device, the set of audio signals received by the one or more microphones of the first playback device;

processing the set of audio signals using a first set of audio processing algorithms to determine a set of signal measures corresponding to the set of audio signals;

based on the set of signal measures, identifying, from the set of audio signals, at least two audio signals that are to be re-processed using a second set of audio processing algorithms so as to improve the respective signal measures of the at least two audio signals;

re-processing the at least two audio signals using the second set of audio processing algorithms;

combining the re-processed at least two audio signals into a combined audio signal; and

sending the combined audio signal to a network device.

2. The media system of claim 1, wherein the network device comprises a server, the server comprising:

a network interface;

a processor;

a non-transitory computer-readable medium;

program instructions stored on the non-transitory computer-readable medium that are executable by the processor to cause the server to perform functions comprising;

determining the available amount of computational power of the second playback device; and

sending, to the first playback device, the indication of the available amount of computational power of the second playback device.

3. The media system of claim 1, wherein the second playback device further comprises program instructions stored on the non-transitory computer-readable medium that are executable by the processor to cause the second playback device to perform functions comprising:

determining the available amount of computational power of the second playback device; and

sending, to the first playback device, the indication of the available amount of computational power of the second playback device.

4. The media system of claim 1, the second playback device further comprising one or more microphones.

5. The media system of claim 4, wherein at least a portion of the set of audio signals received by the one or more microphones of the first playback device is additionally received by the one or more microphones of the second playback device.

6. The media system of claim 1, the media system further comprising a third playback device, and wherein the first playback device further comprises program instructions stored on the non-transitory computer-readable medium that are executable by the processor to cause the first playback device to perform functions comprising:

receiving an indication of an available amount of computational power of the third playback device;

comparing the available amount of computational power of the second playback device with the available amount of computational power of the third playback device; and

based on comparing the available amount of computational power of the second playback device with the available amount of computational power of the third playback device, deciding to send the set of audio signals received by the one or more microphones of the first playback device to the second playback device.

7. The media system of claim 1, wherein each of the first set of audio processing algorithms and the second set of audio processing algorithms comprises at least one of an echo cancellation algorithm or a beamforming algorithm.

8. The media system of claim 1, wherein the first playback device further comprises program instructions stored on the non-transitory computer-readable medium that are executable by the processor to cause the first playback device to perform functions comprising:

based on the received indication of the available amount of computational power of the second playback device, determining that the available amount of computational power of the second playback device is above a threshold amount of computational power.

9. The media system of claim 1, wherein the at least one voice input comprises at least one command, and wherein the network device comprises a server, the server comprising:

a network interface;

a processor;

a non-transitory computer-readable medium;

program instructions stored on the non-transitory computer-readable medium that are executable by the processor to cause the server to perform functions comprising;

receiving the combined audio signal from the second playback device;

based on the combined audio signal, recognizing the at least one command; and

transmitting an indication of the recognized at least one command to the first playback device.

10. The media system of claim 1, wherein the at least two audio signals comprise audio signals having signal to noise ratios meeting a threshold amount.

11. The media system of claim 1, wherein a resolution of the first set of audio processing algorithms is lower than a resolution of the second set of audio processing algorithms.

12. The media system of claim 1, wherein a signal measure of the combined audio signal is higher than a respective signal measure of any of the set of audio signals or any of the re-processed at least two audio signals.

13. The media system of claim 1, wherein the signal measure is a signal to noise ratio.

14. The media system of claim 1, wherein the network device is one of the first playback device or a server.

15. A method comprising:

receiving, by a first playback device from a second playback device, an indication of an available amount of computational power of the second playback device; and

based on the received indication of the available amount of computational power of the second playback device, sending, from the first playback device to the second playback device, a set of audio signals received by one or more microphones of the first playback device wherein the set of audio signals includes at least one voice input;

receiving, by the second playback device from the first playback device, the set of audio signals received by the one or more microphones of the first playback device;

processing, by the second playback device, the set of audio signals using a first set of audio processing algorithms to determine a set of signal measures corresponding to the set of audio signals;

based on the set of signal measures, identifying, by the second playback device from the set of audio signals, at least two audio signals that are to be re-processed using a second set of audio processing algorithms so as to improve the respective signal measures of the at least two audio signals;

re-processing, by the second playback device, the at least two audio signals using the second set of audio processing algorithms;

combining, by the second playback device, the re-processed at least two audio signals into a combined audio signal; and

sending, by the second playback device, the combined audio signal to a network device.

16. The method of claim 15, wherein the network device is one of the first playback device or a server.

17. The method of claim 15, the second playback device further comprising one or more microphones.

18. The method of claim 15, further comprising:

based on the received indication of the available amount of computational power of the second playback device, determining, by the first playback device, that the available amount of computational power of the second playback device is above a threshold amount of computational power.

19. The method of claim 15, wherein the at least two audio signals comprise audio signals having signal to noise ratios meeting a threshold amount.

20. The method of claim 15, wherein the signal measure is a signal to noise ratio.