Device impairment detection

• US 10,418,050 B2
• Filed: 09/28/2017
• Issued: 09/17/2019
• Est. Priority Date: 09/17/2015
• Status: Active Grant

• Current Assignee: Sonos, Inc.
• Original Assignee: Sonos, Inc.
• Inventors: Hartung, Klaus, Bright, Greg
• Primary Examiner(s): Teshale, Akelaw
• Application Number:
  US15/718,579
• Publication Number:
  US 20180018991A1
• Time in Patent Office: 719 Days
• CPC Class Codes: G06N 3/02 Neural networks
• CPC Class Codes: G06N 3/084 Backpropagation, e.g. using gradient descent
• CPC Class Codes: G10L 25/30 using neural networks
• CPC Class Codes: H04R 2227/005 Audio distribution systems for home, i.e. multi-room use
• CPC Class Codes: H04R 27/00 Public address systems circuits for preventing acoustic reaction H04R3/02; circuits for distributing signals to loudspeakers H04R3/12; monitoring or testing arrangements for public address systems H04R29/007; amplifiers H03F
• CPC Class Codes: H04R 29/007 for public address systems public address systems per se H04R27/00
• CPC Class Codes: H04R 3/04 for correcting frequency response
• CPC Class Codes: H04S 7/301 Automatic calibration of stereophonic sound system, e.g. with test microphone

Abstract

Examples described herein involve detecting known impairments or other known conditions using a neural network. An example implementation receives a response matrix that represents, in respective dimensions, responses of a given playback device under respective iterations of a sound captured by a recording device, the iterations including first iterations with respective impairments to the recording device and second iterations without the respective impairments to the recording device. The implementation determines principle components representing the axes of greatest variance in the response matrix, a principle-component matrix that represents a given set of the principle components, and a teaching matrix by projecting the principle-component onto the response matrix. The implementation trains a neural network that includes an output layer comprising a neuron for each of the respective impairments by iteratively providing vectors of the teaching matrix to the neural network and stores the trained neural network on a computing system.

Claims

1. Tangible, non-transitory computer-readable medium having stored thereon instructions executable by a computing device to perform functions comprising:

receiving, by the computing device, a response matrix that represents, in respective dimensions, responses of a given playback device under respective iterations of a sound captured by a recording device, the iterations including first iterations with respective impairments to the recording device and second iterations without the respective impairments to the recording device;

determining, by the computing device, principle components representing the axes of greatest variance in the response matrix, the principle components comprising respective eigenvectors that include a component for each of the respective iterations;

determining a principle-component matrix that represents a given set of the principle components;

determining a teaching matrix by projecting the principle-component onto the response matrix;

training a neural network that includes an output layer comprising a neuron for each of the respective impairments by iteratively providing vectors of the teaching matrix to the neural network, wherein training the neural network comprises;

determining error between the teaching matrix and output of the neural network; and

adjusting respective transfer function factors of the neurons to offset the determined error; and

storing the trained neural network on a computing system.

2. The tangible, non-transitory computer-readable medium of claim 1, wherein receiving the response matrix that represents, in respective dimensions, responses of the given playback device under respective iterations comprises:

receiving data indicating vectors for each of the respective iterations; and

combining the vectors into the response matrix that includes a dimension for each iteration.

3. The tangible, non-transitory computer-readable medium of claim 1, wherein the given playback device is a first type of playback device, and wherein the functions further comprise:

determining a difference between the response of the given playback device and a response corresponding to a second type of playback device; and

determining the response matrix by determining a product of an inverse of the determined difference and the response of the second type of playback device to offset the different types of playback devices.

4. The tangible, non-transitory computer-readable medium of claim 1, wherein the functions further comprise:

receiving data indicating a response of one or more playback devices captured by a given microphone;

determining an input vector by projecting a response vector that represents the response of the one or more playback devices onto a principle component matrix representing variance caused by one or more calibration impairments;

providing the determined input vector to the trained neural network;

detecting that the input vector caused one or more neurons of the neural network to fire such that the trained neural network indicates that one or more particular calibration impairments are affecting the given microphone; and

causing a calibration of the one or more playback devices to be adjusted to offset the one or more particular calibration impairments.

5. The tangible, non-transitory computer-readable medium of claim 4, wherein causing the calibration of the one or more playback devices to be adjusted to offset the one or more particular calibration impairments comprises:

applying one or more respective correction curves corresponding to the one or more particular calibration impairments to a calibration profile that offsets acoustic characteristics of a given environment to calibrate the one or more playback devices to a calibration equalization.

6. The tangible, non-transitory computer-readable medium of claim 4, wherein detecting that the input vector caused one or more neurons of the trained neural network to fire comprises detecting that the input vector caused a neuron corresponding to a particular type of recording-device case to fire.

7. A computing device comprising:

one or more processors; and

tangible, non-transitory computer-readable medium having stored thereon instructions executable by the one or more processors to cause the computing device to perform functions comprising;

receiving, by the computing device, a response matrix that represents, in respective dimensions, responses of a given playback device under respective iterations of a sound captured by a recording device, the iterations including first iterations with respective impairments to the recording device and second iterations without the respective impairments to the recording device;

determining, by the computing device, principle components representing the axes of greatest variance in the response matrix, the principle components comprising respective eigenvectors that include a component for each of the respective iterations;

determining a principle-component matrix that represents a given set of the principle components;

determining a teaching matrix by projecting the principle-component onto the response matrix;

training a neural network that includes an output layer comprising a neuron for each of the respective impairments by iteratively providing vectors of the teaching matrix to the neural network, wherein training the neural network comprises;

determining error between the teaching matrix and output of the neural network; and

adjusting respective transfer function factors of the neurons to offset the determined error; and

storing the trained neural network on a computing system.

8. The computing device of claim 7, wherein receiving the response matrix that represents, in respective dimensions, responses of the given playback device under respective iterations comprises:

receiving data indicating vectors for each of the respective iterations; and

combining the vectors into the response matrix that includes a dimension for each iteration.

9. The computing device of claim 7, wherein the given playback device is a first type of playback device, wherein the functions further comprise:

determining a difference between the response of the given playback device and a response corresponding to a second type of playback device

determining the response matrix by determining a product of an inverse of the determined difference and the response of the second type of playback device to offset the different types of playback devices.

10. The computing device of claim 7, wherein the functions further comprise:

receiving data indicating a response of one or more playback devices captured by a given microphone;

determining an input vector by projecting a response vector that represents the response of the one or more playback devices onto a principle component matrix representing variance caused by one or more calibration impairments;

providing the determined input vector to the trained neural network;

detecting that the input vector caused one or more neurons of the neural network to fire such that the trained neural network indicates that one or more particular calibration impairments are affecting the given microphone; and

causing a calibration of the one or more playback devices to be adjusted to offset the one or more particular calibration impairments.

11. The computing device of claim 10, wherein causing the calibration of the one or more playback devices to be adjusted to offset the one or more particular calibration impairments comprises:

transmitting to a network device corresponding to the given microphone, data indicating the one or more particular calibration impairments.

12. The computing device of claim 10, wherein detecting that the input vector caused one or more neurons of the trained neural network to fire comprises detecting that the input vector caused a neuron corresponding to a particular type of room to fire.

13. The computing device of claim 10, wherein detecting that the input vector caused one or more neurons of the trained neural network to fire comprises detecting that the input vector caused a neuron corresponding to a particular type of recording-device case to fire.

14. A method of training a neural network to identify impairments, the method comprising:

receiving, by a computing device, a response matrix that represents, in respective dimensions, responses of a given playback device under respective iterations of a sound captured by a recording device, the iterations including first iterations with respective impairments to the recording device and second iterations without the respective impairments to the recording device;

determining, by the computing device, principle components representing the axes of greatest variance in the response matrix, the principle components comprising respective eigenvectors that include a component for each of the respective iterations;

determining a principle-component matrix that represents a given set of the principle components;

determining a teaching matrix by projecting the principle-component onto the response matrix;

training a neural network that includes an output layer comprising a neuron for each of the respective impairments by iteratively providing vectors of the teaching matrix to the neural network, wherein training the neural network comprises;

determining error between the teaching matrix and output of the neural network; and

adjusting respective transfer function factors of the neurons to offset the determined error; and

storing the trained neural network on a computing system.

15. The method of claim 14, wherein receiving the response matrix that represents, in respective dimensions, responses of the given playback device under respective iterations comprises:

receiving data indicating vectors for each of the respective iterations; and

combining the vectors into the response matrix that includes a dimension for each iteration.

16. The method of claim 14, wherein the given playback device is a first type of playback device, the method further comprising:

determining a difference between the response of the given playback device and a response corresponding to a second type of playback device

determining the response matrix by determining a product of an inverse of the determined difference and the response of the second type of playback device so as to offset the different types of playback devices.

17. The method of claim 14, further comprising:

receiving data indicating a response of one or more playback devices captured by a given microphone;

determining an input vector by projecting a response vector that represents the response of the one or more playback devices onto a principle component matrix representing variance caused by one or more calibration impairments;

providing the determined input vector to the trained neural network;

detecting that the input vector caused one or more neurons of the neural network to fire such that the trained neural network indicates that one or more particular calibration impairments are affecting the given microphone; and

causing a calibration of the one or more playback devices to be adjusted to offset the one or more particular calibration impairments.

18. The method of claim 17, wherein detecting that the input vector caused one or more neurons of the trained neural network to fire comprises determining that output values of neuron transfer functions corresponding to respective neurons exceed a firing threshold.

19. The method of claim 17, wherein detecting that the input vector caused one or more neurons of the trained neural network to fire comprises detecting that the input vector caused a neuron corresponding to a particular type of recording-device case to fire.

20. The method of claim 17, wherein detecting that the input vector caused one or more neurons of the trained neural network to fire comprises detecting that the input vector caused a neuron corresponding to a particular type of room to fire.