Spatial audio correction

• US 10,129,678 B2
• Filed: 10/16/2017
• Issued: 11/13/2018
• Est. Priority Date: 07/15/2016
• Status: Active Grant

• Current Assignee: Sonos, Inc.
• Original Assignee: Sonos, Inc.
• Inventors: Sheen, Timothy
• Primary Examiner(s): Fischer, Mark
• Application Number:
  US15/785,088
• Publication Number:
  US 20180054687A1
• Time in Patent Office: 393 Days
• CPC Class Codes: G06F 3/162 Interface to dedicated audio devices, e.g. audio drivers, interface to CODECs
• CPC Class Codes: G06F 3/165 Management of the audio stream, e.g. setting of volume, audio stream path
• CPC Class Codes: H04R 2227/003 Digital PA systems using, e.g. LAN or internet
• CPC Class Codes: H04R 2227/005 Audio distribution systems for home, i.e. multi-room use
• CPC Class Codes: H04R 2227/007 Electronic adaptation of audio signals to reverberation of the listening space for PA
• CPC Class Codes: H04R 2430/20 Processing of the output signals of the acoustic transducers of an array for obtaining a desired directivity characteristic H04R2203/12 takes precedence
• CPC Class Codes: H04R 2499/13 Acoustic transducers and sound field adaptation in vehicles
• CPC Class Codes: H04R 29/002 Loudspeaker arrays
• CPC Class Codes: H04R 29/007 for public address systems public address systems per se H04R27/00
• CPC Class Codes: H04R 3/12 for distributing signals to two or more loudspeakers specially adapted for hearing aids H04R25/407
• CPC Class Codes: H04S 2400/15 Aspects of sound capture and related signal processing for recording or reproduction
• CPC Class Codes: H04S 7/301 Automatic calibration of stereophonic sound system, e.g. with test microphone
• CPC Class Codes: H04S 7/305 Electronic adaptation of stereophonic audio signals to reverberation of the listening space H04S7/301 takes precedence

Abstract

Example techniques may involve performing aspects of a spatial calibration. An example implementation may include detecting a trigger condition that initiates calibration of a media playback system including multiple audio drivers that form multiple sound axes, each sound axis corresponding to a respective channel of multi-channel audio content The implementation may also include causing the multiple audio drivers to emit calibration audio that is divided into constituent frames, the multiple sound axes emitting calibration audio during respective slots of each constituent frame. The implementation may further include recording the emitted calibration audio. The implementation may include causing delays for each sound axis of the multiple sound axes to be determined, the determined delay for each sound axis based on the slots of recorded calibration audio corresponding to the sound axes and causing the multiple sound axes to be calibrated.

Claims

1. A tangible, non-transitory computer-readable medium storing instructions executable by one or more processors to cause a network microphone device (NMD) to perform operations comprising:

recording, via a microphone of the NMD, calibration audio emitted via at least one of a plurality of audio drivers of one or more playback devices, wherein the plurality of audio drivers are configured to form a plurality of sound axes including a first sound axis and at least a second sound axis, and wherein individual sound axes correspond to a respective channel of multi-channel audio content, wherein the NMD is communicatively coupled with a media playback system that includes the one or more playback devices;

arranging the recorded calibration audio into constituent frames including a first frame having a first slot and at least a second slot, wherein the first and second slots comprise portions of the recorded calibration audio corresponding to the respective first and second axes;

determining a first arrival time and a second arrival time of the portions of the recorded calibration audio in the respective first and second slots;

determining a first delay for the first sound axis and a second delay for the second axis, wherein the first and second delays are based on the respective first and second arrival times, wherein determining the first delay comprises determining that the first delay exceeds a maximum delay threshold; and

causing audio output of the first and second sound axes to be delayed according to the respective first and second delays, wherein causing the audio output of the first and second sound axes to be delayed comprises causing audio output of the first axis to be delayed according to the maximum delay threshold when the media playback system is playing back audio content that is paired with video content.

2. The tangible, non-transitory computer-readable medium of claim 1, wherein causing audio output of the first and second sound axes to be delayed further comprises:

causing respective filters to delay audio output of the plurality of audio drivers according to the respective determined delays for the plurality of sound axes.

3. The tangible, non-transitory computer-readable medium of claim 1, wherein the NMD comprises the one or more processors, and wherein causing the one or more processors to determining the first and second arrival times comprises:

determining first and second impulse responses corresponding to the respective first and second slots;

aligning the first and second impulse responses to a respective first reference point;

identifying respective second reference points in the first and second impulse responses;

determining a first difference between the first and second reference points of the first impulse response;

determining a second difference between the first and second reference points of the second impulse response; and

determining first and second arrival times based on the respective first and second differences.

4. The tangible, non-transitory computer-readable medium of claim 1, wherein a network interface communicatively couples the NMD and the one or more processors, and wherein causing the one or more processors to determine respective first and second arrival times comprises:

sending, via the network interface to the one or more processors, (i) the recorded calibration audio, and (ii) an instruction to determine the first and second arrival times; and

receiving, via the network interface, the first and second arrival times.

5. The tangible, non-transitory computer-readable medium of claim 1, wherein the NMD is communicatively coupled to a media playback system, and wherein the instructions further include instructions for performing operations comprising:

detecting, via a user interface, input data indicating a command to initiate calibration of a media playback system.

6. The tangible, non-transitory computer-readable medium of claim 1, wherein the NMD is communicatively coupled with a media playback system, and wherein the instructions further include instructions for performing operations comprising:

detecting a trigger condition that initiates calibration of the media playback system.

7. The tangible, non-transitory computer-readable medium of claim 6, wherein the media playback system comprises one or more playback devices comprising a first audio driver and a second audio driver, wherein detecting the trigger condition comprises detecting a configuration of the media playback system in which the first and second audio drivers correspond to the first and second sound axes.

8. A method of operating a network microphone device (NMD), the method comprising:

recording, via a microphone of the NMD, calibration audio emitted via at least one of a plurality of audio drivers of one or more playback devices, wherein the plurality of audio drivers are configured to form a plurality of sound axes including a first sound axis and at least a second sound axis, and wherein individual sound axes correspond to a respective channel of multi-channel audio content, wherein the NMD is communicatively coupled with a media playback system that includes the one or more playback devices;

arranging, via one or more processors, the recorded calibration audio into constituent frames including a first frame having a first slot and at least a second slot, wherein the first and second slots comprise portions of the recorded calibration audio corresponding to the respective first and second axes;

determining, via one or more processors, a first arrival time and a second arrival time of the portions of the recorded calibration audio in the respective first and second slots;

determining, via one or more processors, a first delay for the first sound axis and a second delay for the second axis, wherein the first and second delays are based on the respective first and second arrival times, wherein determining the first delay comprises determining that the first delay exceeds a maximum delay threshold; and

causing audio output of the first and second sound axes to be delayed according to the respective first and second delays, wherein causing the audio output of the first and second sound axes to be delayed comprises causing audio output of the first axis to be delayed according to the maximum delay threshold when the media playback system is playing back audio content that is paired with video content.

9. The method of claim 8, wherein causing audio output of the first and second sound axes to be delayed further comprises:

causing respective filters to delay audio output of the plurality of audio drivers according to the respective determined delays for the plurality of sound axes.

10. The method of claim 8, wherein the NMD comprises the one or more processors, and wherein determining the first and second arrival times comprises:

determining first and second impulse responses corresponding to the respective first and second slots;

aligning the first and second impulse responses to a respective first reference point;

identifying respective second reference points in the first and second impulse responses;

determining a first difference between the first and second reference points of the first impulse response;

determining a second difference between the first and second reference points of the second impulse response; and

determining first and second arrival times based on the respective first and second differences.

11. The method of claim 8, wherein a network interface communicatively couples the NMD and the one or more processors, and wherein determining the respective first and second arrival times comprises:

sending, via the network interface to the one or more processors, (i) the recorded calibration audio, and (ii) an instruction to determine the first and second arrival times; and

receiving, via the network interface, the first and second arrival times.

12. The method of claim 8, wherein the NMD is communicatively coupled to a media playback system, the method further comprising:

detecting, via a user interface, input data indicating a command to initiate calibration of the media playback system.

13. The method of claim 8, wherein the NMD is communicatively coupled with a media playback system, the method further comprising:

detecting a trigger condition that initiates calibration of the media playback system.

14. The method of claim 13, wherein the media playback system comprises one or more playback devices comprising a first audio driver and a second audio driver, wherein detecting the trigger condition comprises detecting a configuration of the media playback system in which the first and second audio drivers correspond to the first and second sound axes.

15. A media playback system comprising:

one or more playback devices comprising a plurality of audio drivers configured to form a plurality of sound axes including a first sound axis and at least a second sound axis, wherein individual sound axes correspond to a respective channel of multi-channel audio content;

a network microphone device (NMD) comprising a microphone;

one or more processors; and

a computer-readable medium storing instructions, that when executed by the one or more processors, cause the media playback system to perform operations comprising;

recording, via the microphone, calibration audio emitted via at least one of the plurality of audio drivers, wherein the NMD is communicatively coupled with the media playback system;

arranging the recorded calibration audio into constituent frames including a first frame having a first slot and at least a second slot, wherein the first and second slots comprise portions of the recorded calibration audio corresponding to the respective first and second axes;

determining a first arrival time and a second arrival time of the portions of the recorded calibration audio in the respective first and second slots;

determining a first delay for the first sound axis and a second delay for the second axis, wherein the first and second delays are based on the respective first and second arrival times, wherein determining the first delay comprises determining that the first delay exceeds a maximum delay threshold; and

causing audio output of the first and second sound axes to be delayed according to the respective first and second delays, wherein causing the audio output of the first and second sound axes to be delayed comprises causing audio output of the first axis to be delayed according to the maximum delay threshold when the media playback system is playing back audio content that is paired with video content.

16. The media playback system of claim 15, wherein the computer readable medium further includes instructions for performing operations comprising:

detecting, via a user interface, input data indicating a command to initiate calibration of the media playback system.

17. The media playback system of claim 15, wherein the computer readable medium further includes instructions for performing operations comprising:

detecting a trigger condition that initiates calibration of the media playback system.

18. The media playback system of claim 17, wherein the plurality of audio drivers comprise a first audio driver and a second audio driver, and wherein detecting the trigger condition comprises detecting a configuration of the media playback system in which the first and second audio drivers correspond to the first and second sound axes.

19. The media playback system of claim 15, wherein causing audio output of the first and second sound axes to be delayed further comprises:

causing respective filters to delay audio output of the plurality of audio drivers according to the respective determined delays for the plurality of sound axes.

20. The media playback system of claim 15, wherein determining the first and second arrival times comprises:

determining first and second impulse responses corresponding to the respective first and second slots;

aligning the first and second impulse responses to a respective first reference point;

identifying respective second reference points in the first and second impulse responses;

determining a first difference between the first and second reference points of the first impulse response;

determining a second difference between the first and second reference points of the second impulse response; and

determining first and second arrival times based on the respective first and second differences.