Compensation for speaker nonlinearities

• US 10,097,939 B2
• Filed: 02/21/2017
• Issued: 10/09/2018
• Est. Priority Date: 02/22/2016
• Status: Active Grant

• Current Assignee: Sonos, Inc.
• Original Assignee: Sonos, Inc.
• Inventors: Sheen, Timothy W., Jarvis, Simon, Kadri, Romi, Chen, Yean-Nian Willy
• Primary Examiner(s): GAY, SONIA L
• Application Number:
  US15/438,741
• Publication Number:
  US 20170245079A1
• Time in Patent Office: 595 Days
• CPC Class Codes: H04R 2227/003 Digital PA systems using, e.g. LAN or internet
• CPC Class Codes: H04R 2227/005 Audio distribution systems for home, i.e. multi-room use
• CPC Class Codes: H04R 2227/007 Electronic adaptation of audio signals to reverberation of the listening space for PA
• CPC Class Codes: H04R 2499/11 Transducers incorporated or for use in hand-held devices, e.g. mobile phones, PDA's, camera's
• CPC Class Codes: H04R 27/00 Public address systems circuits for preventing acoustic reaction H04R3/02; circuits for distributing signals to loudspeakers H04R3/12; monitoring or testing arrangements for public address systems H04R29/007; amplifiers H03F
• CPC Class Codes: H04R 29/003 of the moving-coil type
• CPC Class Codes: H04R 29/007 for public address systems public address systems per se H04R27/00
• CPC Class Codes: H04R 3/007 Protection circuits for transducers
• CPC Class Codes: H04R 3/04 for correcting frequency response
• CPC Class Codes: H04R 9/046 Construction
• CPC Class Codes: H04S 7/305 Electronic adaptation of stereophonic audio signals to reverberation of the listening space H04S7/301 takes precedence
• CPC Class Codes: H04S 7/307 Frequency adjustment, e.g. tone control H04S7/301 takes precedence

Abstract

A first signal may be received indicative of audio to be played by a speaker. A second signal may be received which comprises (i) a voice input received by a microphone and (ii) at least a portion of the audio played by the speaker at a same time that the microphone receives the voice input. Based on the first signal, nonlinearities output by the speaker which played the audio may be determined. At least the nonlinearities from the second signal may be removed to output a third signal comprising substantially the voice input received at the microphone.

Claims

1. An audio system comprising:

a playback device comprising a speaker, the playback device disposed at a first location; and

a network microphone device disposed at a second location, the network microphone device being displaceable relative to the playback device, the network microphone device comprising;

a microphone;

a processor; and

memory storing instructions executable by the processor to cause the processor to;

receive a first signal indicative of audio to be played back via the speaker of the playback device and a second signal that comprises (i) a voice input received via the microphone and (ii) at least a portion of the audio played by the speaker of the playback device at a same time that the microphone receives the voice input; and

perform self-sound suppression on at least one of the first signal and the second signal, wherein performing self-sound suppression comprises;

based on the first signal, determining nonlinearities output via the speaker of the playback device by inputting a representation of the first signal into a model configured to output an indication of a frequency response that changes over time, wherein at least a portion of the frequency response is indicative of nonlinear audio effects, and wherein the nonlinear audio effects comprise an intermodulation distortion; and

removing at least a portion of the determined nonlinearities from the second signal to output a third signal comprising substantially the voice input received at the microphone.

2. The audio system of claim 1, wherein the model is based on measurement of a position of a moving component of the speaker.

3. The audio system of claim 1, wherein removing at least the nonlinearities from the second signal to output a third signal comprises determining a compensated audio signal based on the first signal and the nonlinear audio effects output by the speaker of the playback device, wherein the compensated audio signal characterizes how the audio played by the speaker sounds at the microphone.

4. The audio system of claim 1, wherein removing at least the nonlinearities from the second signal to output a third signal comprises applying a transfer function to the first audio signal wherein the transfer function is a relative frequency response between a fourth signal indicative of second audio to be played by the speaker of the playback device and a fifth audio signal received at the microphone when the second audio is played.

5. The audio system of claim 1, wherein the microphone is located within a given distance from speaker of the playback device, wherein at the given distance the microphone detects the audio played by the speaker of the playback device.

6. The audio system of claim 1, further comprising computer instructions for converting the voice input in the third signal into text.

7. The audio system of claim 1, wherein the first signal is tapped from a signal processing pathway associated with the speaker of the playback device after a time varying filter is applied to the first signal.

8. A method comprising:

receiving a first signal indicative of audio to be played back via a speaker of a playback device disposed at a first location and a second signal that comprises (i) a voice input received via a microphone of a network microphone device disposed at a second location, the network microphone device being displaceable relative to the playback device, and (ii) at least a portion of the audio played by the speaker at a same time that the microphone receives the voice input; and

performing self-sound suppression on at least one of the first signal and the second signal, wherein performing self-sound suppression comprises;

based on the first signal, determining nonlinearities output via the speaker of the playback device by inputting a representation of the first signal into a model configured to output an indication of a frequency response that changes over time, wherein at least a portion of the frequency response is indicative of nonlinear audio effects, and wherein the nonlinear audio effects comprise an intermodulation distortion; and

removing at least a portion of the determined nonlinearities from the second signal to output a third signal comprising substantially the voice input received at the microphone of the network microphone device.

9. The method of claim 8, wherein the model is based on measurement of a position of a moving component of the speaker.

10. The method of claim 8, wherein removing at least the nonlinearities from the second signal to output a third signal comprises determining a compensated audio signal based on the first signal and the nonlinear audio effects output by the speaker, wherein the compensated audio signal characterizes how the audio played by the speaker sounds at the microphone.

11. The method of claim 8, wherein removing at least the nonlinearities from the second signal to output a third signal comprises applying a transfer function to the first audio signal wherein the transfer function is a relative frequency response between a fourth signal indicative of second audio to be played by the speaker and a fifth audio signal received at the microphone when the second audio is played.

12. The method of claim 8, wherein the microphone is acoustically proximate to the speaker.

13. The method of claim 8, further comprising converting the voice input in the third signal into text.

14. The method of claim 8, wherein the first signal is tapped from a signal processing pathway associated with the speaker after a time varying filter is applied to the first signal.

15. A tangible non-transitory computer readable storage medium including instructions for execution by a processor, the instructions, when executed, cause the processor to implement a method comprising:

receiving a first signal indicative of audio to be played back via a speaker of a playback device disposed at a first location and a second signal that comprises (i) a voice input received via a microphone of a network microphone device disposed at a second location, the network microphone device being displaceable relative to the playback device, and (ii) at least a portion of the audio played by the speaker at a same time that the microphone receives the voice input; and

performing self-sound suppression on at least one of the first signal and the second signal, wherein performing self-sound suppression comprises;

based on the first signal, determining nonlinearities output via the speaker of the playback device by inputting a representation of the first signal into a model configured to output an indication of a frequency response that changes over time, wherein at least a portion of the frequency response is indicative of nonlinear audio effects, and wherein the nonlinear audio effects comprise an intermodulation distortion; and

removing at least a portion of the determined nonlinearities from the second signal to output a third signal comprising substantially the voice input received at the microphone of the network microphone device.

16. The tangible non-transitory computer readable storage medium of claim 15, further comprising computer instructions to obtain acoustics of an environment in which the speaker is located;

and apply the acoustics to the third signal comprising substantially the voice input received at the microphone.