Spatial audio correction

• US 9,794,710 B1
• Filed: 07/15/2016
• Issued: 10/17/2017
• Est. Priority Date: 07/15/2016
• Status: Active Grant

• Current Assignee: Sonos, Inc.
• Original Assignee: Sonos, Inc.
• Inventors: Sheen, Timothy
• Primary Examiner(s): Fischer, Mark
• Application Number:
  US15/211,822
• Time in Patent Office: 459 Days
• CPC Class Codes: G06F 3/162 Interface to dedicated audio devices, e.g. audio drivers, interface to CODECs
• CPC Class Codes: G06F 3/165 Management of the audio stream, e.g. setting of volume, audio stream path
• CPC Class Codes: H04R 2227/003 Digital PA systems using, e.g. LAN or internet
• CPC Class Codes: H04R 2227/005 Audio distribution systems for home, i.e. multi-room use
• CPC Class Codes: H04R 2227/007 Electronic adaptation of audio signals to reverberation of the listening space for PA
• CPC Class Codes: H04R 2430/20 Processing of the output signals of the acoustic transducers of an array for obtaining a desired directivity characteristic H04R2203/12 takes precedence
• CPC Class Codes: H04R 2499/13 Acoustic transducers and sound field adaptation in vehicles
• CPC Class Codes: H04R 29/002 Loudspeaker arrays
• CPC Class Codes: H04R 29/007 for public address systems public address systems per se H04R27/00
• CPC Class Codes: H04R 3/12 for distributing signals to two or more loudspeakers specially adapted for hearing aids H04R25/407
• CPC Class Codes: H04S 2400/15 Aspects of sound capture and related signal processing for recording or reproduction
• CPC Class Codes: H04S 7/301 Automatic calibration of stereophonic sound system, e.g. with test microphone
• CPC Class Codes: H04S 7/305 Electronic adaptation of stereophonic audio signals to reverberation of the listening space H04S7/301 takes precedence

Abstract

Example techniques may involve performing aspects of a spatial calibration. An example implementation may include detecting a trigger condition that initiates calibration of a media playback system including multiple audio drivers that form multiple sound axes, each sound axis corresponding to a respective channel of multi-channel audio content The implementation may also include causing the multiple audio drivers to emit calibration audio that is divided into constituent frames, the multiple sound axes emitting calibration audio during respective slots of each constituent frame. The implementation may further include recording the emitted calibration audio. The implementation may include causing delays for each sound axis of the multiple sound axes to be determined, the determined delay for each sound axis based on the slots of recorded calibration audio corresponding to the sound axes and causing the multiple sound axes to be calibrated.

Claims

1. A tangible, non-transitory computer-readable medium having stored therein instructions executable by one or more processors to cause a networked microphone device (NMD) to perform a method comprising:

detecting a trigger condition that initiates calibration of a media playback system comprising multiple audio drivers that form multiple sound axes, each sound axis corresponding to a respective channel of multi-channel audio content;

causing, via a network interface, the multiple audio drivers to emit calibration audio that is divided into constituent frames, the multiple sound axes emitting calibration audio during respective slots of each constituent frame;

recording, via a microphone, the emitted calibration audio;

causing respective delays for each sound axis of the multiple sound axes to be determined based on the slots of recorded calibration audio corresponding to the sound axis; and

causing the multiple sound axes to be calibrated, wherein calibrating the multiple sound axes comprises causing audio output of the multiple sound axes to be delayed according to the respective determined delays.

2. The tangible, non-transitory computer-readable medium of claim 1, wherein causing respective delays for each sound axis of the multiple sound axes to be determined comprises:

causing a processing device to determine respective times-of-arrival at the microphone for each sound axis of the multiple sound axes from the slots of recorded calibration audio corresponding to each sound axis; and

causing the delays for each sound axis of the multiple sound axes to be determined, each determined delay corresponding to the determined time-of-arrival of a respective sound axis.

3. The tangible, non-transitory computer-readable medium of claim 2, wherein causing audio output of the multiple sound axes to be delayed according to the respective determined delays comprises causing respective filters to delay audio output of the multiple audio drivers according to the respective determined delays for the multiple sound axes.

4. The tangible, non-transitory computer-readable medium of claim 2, wherein the NMD comprises the processing device, and wherein causing the processing device to determine respective times-of-arrival at the microphone for each sound axis of the multiple sound axes comprises:

dividing the recorded calibration audio into the constituent frames and each constituent frame into the respective slots for each sound axis;

determining respective impulse responses for the sound axes from the respective slots corresponding to the each sound axis;

aligning the respective impulse responses to a first reference point;

identifying respective second reference points in each impulse response; and

determining the respective times-of-arrival at the microphone based on respective differences from the first reference point to the second reference points in each impulse response.

5. The tangible, non-transitory computer-readable medium of claim 4, wherein the sound axes consist of a reference sound axis and one or more other sound axes, and wherein identifying respective second reference points in each impulse response comprises:

identifying, as a given second reference point, a peak value in the impulse response of the reference sound axis; and

identifying, as the other second reference points, respective peak values of the impulse responses of the one or more other sound axes in a time window subsequent to the given second reference point.

6. The tangible, non-transitory computer-readable medium of claim 2, wherein the processing device is connected to the NMD via one or more networks, and wherein causing the processing device to determine respective times-of-arrival at the microphone for each sound axis of the multiple sound axes comprises:

sending, via the network interface to the processing device, (i) the recorded calibration audio, and (ii) an instruction to determine respective times-of-arrival at the microphone for each sound axis of the multiple sound axes; and

receiving, via the network interface, the determine respective times-of-arrival.

7. The tangible, non-transitory computer-readable medium of claim 1, wherein the each sound axis of the multiple sound axes corresponds to a respective channel of surround sound audio content.

8. The tangible, non-transitory computer-readable medium of claim 7, wherein the media playback system comprises multiple playback devices each comprising a subset of the multiple audio drivers.

9. The tangible, non-transitory computer-readable medium of claim 8, wherein the multiple playback devices comprise a given playback device comprising a particular subset of the multiple audio drivers, wherein the particular subset of the multiple audio drivers forms three sound axes, the three sound axes corresponding respectively to a left channel of the surround sound audio content, a right channel of the surround sound audio content, and a center channel of the audio content.

10. The tangible, non-transitory computer-readable medium of claim 1, wherein detecting the trigger condition that initiates calibration of a media playback system comprises detecting, via a user interface, input data indicating a command to initiate calibration of the media playback system.

11. The tangible, non-transitory computer-readable medium of claim 1, wherein detecting the trigger condition that initiates calibration of the media playback system comprises detecting configuration of the media playback system into a particular axis configuration, wherein the multiple audio drivers form a particular set of sound axes.

12. The tangible, non-transitory computer-readable medium of claim 1, wherein causing delays for each sound axis of the multiple sound axes to be determined comprises:

determining that time-of-arrival of a given sound axis exceeds a maximum delay threshold; and

causing delay of the given sound axis to be set at the maximum delay threshold when the media playback system is playing back audio content that is paired to video content.

13. A method comprising:

detecting a trigger condition that initiates calibration of a media playback system comprising multiple audio drivers that form multiple sound axes, each sound axis corresponding to a respective channel of multi-channel audio content;

causing, via a network interface, the multiple audio drivers to emit calibration audio that is divided into constituent frames, the multiple sound axes emitting calibration audio during respective slots of each constituent frame;

recording, via a microphone of a networked microphone device (NMD), the emitted calibration audio;

causing a processing device to determine respective times-of-arrival at the microphone for each sound axis of the multiple sound axes from the slots of recorded calibration audio corresponding to the sound axes;

causing delays for each sound axis of the multiple sound axes to be determined, each determined delay corresponding to the determined time-of-arrival of a respective sound axis; and

causing the multiple sound axes to be calibrated, wherein calibrating the multiple sound axes comprises causing audio output of the multiple sound axes to be delayed according to the respective determined delays.

14. The method of claim 13, wherein the NMD comprises the processing device, and wherein causing the processing device to determine respective times-of-arrival at the microphone for each sound axis of the multiple sound axes comprises:

dividing the recorded calibration audio into the constituent frames and each constituent frame into the respective slots for each sound axis;

determining respective impulse responses for the sound axes from the respective slots corresponding to the each sound axis;

aligning the respective impulse responses to a first reference point;

identifying respective second reference points in each impulse response; and

determining the respective times-of-arrival at the microphone based on respective differences from the first reference point to the second reference points in each impulse response.

15. The method of claim 14, wherein the sound axes consist of a reference sound axis and one or more other sound axes, and wherein identifying respective second reference points in each impulse response comprises:

identifying, as a given second reference point, a peak value in the impulse response of the reference sound axis; and

identifying, as the other second reference points, respective peak values of the impulse responses of the one or more other sound axes in a time window subsequent to the given second reference point.

16. The method of claim 13, wherein the processing device is connected to the NMD via one or more networks, and wherein causing the processing device to determine respective times-of-arrival at the microphone for each sound axis of the multiple sound axes comprises:

sending, via the network interface to the processing device, (i) the recorded calibration audio, and (ii) an instruction to determine respective times-of-arrival at the microphone for each sound axis of the multiple sound axes; and

receiving, via the network interface, the determine respective times-of-arrival.

17. The method of claim 13, wherein the each sound axis of the multiple sound axes corresponds to a respective channel of surround sound audio content, and wherein the media playback system comprises multiple playback devices each comprising a subset of the multiple audio drivers.

18. The method of claim 17, wherein the multiple playback devices comprise a given playback device comprising a particular subset of the multiple audio drivers, wherein the particular subset of the multiple audio drivers forms three sound axes, the three sound axes corresponding respectively to a left channel of the surround sound audio content, a right channel of the surround sound audio content, and a center channel of the audio content.

19. The method of claim 13, wherein detecting the trigger condition that initiates calibration of a media playback system comprises one of:

(a) detecting, via a user interface, input data indicating a command to initiate calibration of the media playback system, or (b) detecting configuration of the media playback system into a particular axis configuration, wherein the multiple audio drivers form a particular set of sound axes.

20. A media playback system comprising:

one or more playback devices comprising multiple audio drivers that form multiple sound axes, each sound axis corresponding to a respective channel of multi-channel audio content;

a networked microphone device comprising a microphone;

a processor; and

a computer-readable medium having stored therein instructions executable by one or more processors to cause the media playback system to perform a method comprising;

detecting a trigger condition that initiates calibration of a media playback system comprising causing, via a network interface, the multiple audio drivers to emit calibration audio that is divided into constituent frames, the multiple sound axes emitting calibration audio during respective slots of each constituent frame;

recording, via the microphone, the emitted calibration audio;

causing a processing device to determine respective times-of-arrival at the microphone for each sound axis of the multiple sound axes from the slots of recorded calibration audio corresponding to the sound axes;

causing delays for each sound axis of the multiple sound axes to be determined, each determined delay corresponding to the determined time-of-arrival of a respective sound axis; and

causing the multiple sound axes to be calibrated, wherein calibrating the multiple sound axes comprises causing audio output of the multiple sound axes to be delayed according to the respective determined delays.