Audio Playback System Monitoring

• US 20140119551A1
• Filed: 06/27/2012
• Published: 05/01/2014
• Est. Priority Date: 07/01/2011
• Status: Active Grant

• Current Assignee: Dolby Laboratories Licensing Corporation (Dolby Laboratories Incorporated)
• Original Assignee: Dolby Laboratories Licensing Corporation (Dolby Laboratories Incorporated)
• Inventors: Bharitkar, Sunil, Crockett, Brett G., Fielder, Louis D., Rockwell, Michael
• Granted Patent: US 9,462,399 B2
• Time in Patent Office: Days
• US Class Current: 381/59
• CPC Class Codes: H04H 60/04 Studio equipment; Interconnection of studios
• CPC Class Codes: H04H 60/33 Arrangements for monitoring the users' behaviour or opinions
• CPC Class Codes: H04R 2430/03 Synergistic effects of band splitting and sub-band processing
• CPC Class Codes: H04R 29/001 for loudspeakers H04R29/007 takes precedence
• CPC Class Codes: H04R 29/002 Loudspeaker arrays
• CPC Class Codes: H04R 3/12 for distributing signals to two or more loudspeakers specially adapted for hearing aids H04R25/407

Abstract

In some embodiments, a method for monitoring speakers within an audio playback system (e.g., movie theater) environment. In typical embodiments, the monitoring method assumes that initial characteristics of the speakers (e.g., a room response for each of the speakers) have been determined at an initial time, and relies on one or more microphones positioned in the environment to perform a status check on each of the speakers to identify whether a change to at least one characteristic of any of the speakers has occurred since the initial time. In other embodiments, the method processes data indicative of output of a microphone to monitor audience reaction to an audiovisual program. Other aspects include a system configured (e.g., programmed) to perform any embodiment of the inventive method, and a computer readable medium (e.g., a disc) which stores code for implementing any embodiment of the inventive method.

Claims

1. A method for monitoring status of a set of N speakers in a playback environment, where N is a positive integer, said method including steps of:

(a) playing back an audiovisual program whose soundtrack has N channels, including by emitting sound, determined by the program, from the speakers in response to driving each of the speakers with a speaker feed for a different one of the channels of the soundtrack;

(b) obtaining audio data indicative of a status signal captured by each microphone of a set of M microphones in the playback environment during emission of the sound in step (a), where M is a positive integer; and

(c) processing the audio data to perform a status check on each speaker of the set of N speakers, including by comparing, for each said speaker and each of at least one microphone in the set of M microphones, the status signal captured by the microphone and a template signal, wherein the template signal is indicative of response of a template microphone to playback by the speaker, in the playback environment at an initial time, of a channel of the soundtrack corresponding to said speaker.

2. The method of claim 1, wherein the audiovisual program is a movie trailer.

3. The method of claim 2, wherein the playback environment is a movie theater, and step (a) includes the step of playing back the trailer in the presence of an audience in the movie theater.

4. The method of claim 1, wherein the template microphone is positioned, at the initial time, at at least substantially the same position in the environment as is a corresponding microphone of the set during step (b).

5. The method of claim 1, wherein M=1, the audio data obtained in step (a) is indicative of a status signal captured by a microphone in the playback environment during emission of the sound in step (a), and the template microphone is said microphone.

6. The method of claim 1, wherein step (c) includes a step of determining, for each speaker-microphone pair consisting of one of the speakers and one said microphone, a cross-correlation of the template signal for said speaker and microphone with the status signal for said microphone.

7. The method of claim 6, wherein step (c) also includes a step of identifying, for each said speaker-microphone pair, a difference between the template signal for the speaker and the microphone of the pair and the status signal for said microphone, from a frequency domain representation of the cross-correlation for said speaker-microphone pair.

8. The method of claim 6, wherein step (c) also includes the steps of:

determining a cross-correlation power spectrum for each said speaker-microphone pair, from the cross-correlation for said speaker-microphone pair;

determining a smoothed cross-correlation power spectrum for each said speaker-microphone pair from the cross-correlation power spectrum for said speaker-microphone pair; and

analyzing the smoothed cross-correlation power spectrum for at least one said speaker-microphone pair to determine status of the speaker of said pair.

9. The method of claim 1, wherein step (c) includes steps of:

for each speaker-microphone pair consisting of one of the speakers and one said microphone, applying a bandpass filter to the template signal for said speaker and microphone, and to the status signal for said microphone, thereby determining a bandpass filtered template signal and a bandpass filtered status signal; and

determining, for each said speaker-microphone pair, a cross-correlation of the bandpass filtered template signal for said speaker and microphone with the bandpass filtered status signal for said microphone.

10. The method of claim 9, wherein step (c) also includes a step of identifying, for each speaker-microphone pair, a difference between the bandpass filtered template signal for the speaker and the microphone of the pair and the bandpass filtered status signal for said microphone, from a frequency domain representation of the cross-correlation for said speaker-microphone pair.

11. The method of claim 9, wherein step (c) also includes the steps of:

determining a cross-correlation power spectrum for each said speaker-microphone pair, from the cross-correlation for said speaker-microphone pair;

determining a smoothed cross-correlation power spectrum for each said speaker-microphone pair from the cross-correlation power spectrum for said speaker-microphone pair; and

analyzing the smoothed cross-correlation power spectrum for at least one said speaker-microphone pair to determine status of the speaker of said pair.

12. The method of claim 1, wherein step (c) includes steps of:

determining, for each speaker-microphone pair consisting of one of the speakers and one said microphone, a sequence of cross-correlations of the template signal for said speaker and microphone with the status signal for said microphone, wherein each of the cross-correlations is a cross-correlation of a segment of the template signal for said speaker and microphone with a corresponding segment of the status signal for said microphone; and

identifying a difference between the template signal for said speaker and microphone, and the status signal for said microphone, from an average of the cross-correlations.

13. The method of claim 1, wherein step (c) includes steps of:

for each speaker-microphone pair consisting of one of the speakers and one said microphone, applying a bandpass filter to the template signal for said speaker and microphone, and to the status signal for said microphone, thereby determining a bandpass filtered template signal and a bandpass filtered status signal;

determining, for each said speaker-microphone pair, a sequence of cross-correlations of the bandpass filtered template signal for said speaker and microphone with the bandpass filtered status signal for said microphone, wherein each of the cross-correlations is a cross-correlation of a segment of the bandpass filtered template signal for said speaker and microphone with a corresponding segment of the bandpass filtered status signal for said microphone; and

identifying a difference between the bandpass filtered template signal for said speaker and microphone, and the bandpass filtered status signal for said microphone, from an average of the cross-correlations.

14. The method of claim 1, wherein M=1, the audio data obtained in step (a) is indicative of a status signal captured by a microphone in the playback environment during emission of the sound in step (a), the template microphone is said microphone, and step (c) includes a step of determining, for each speaker of the set of M speakers, a cross-correlation of the template signal for said speaker with the status signal.

15. The method of claim 14, wherein step (c) also includes a step of identifying, for each speaker of the set of N speakers, a difference between the template signal for said speaker and the status signal, from a frequency domain representation of the cross-correlation for said speaker.

16. The method of claim 1, wherein M=1, the audio data obtained in step (a) is indicative of a status signal captured by a microphone in the playback environment during emission of the sound in step (a), the template microphone is said microphone, and step (c) includes steps of:

for each speaker of the set of N speakers, applying a bandpass filter to the template signal for said speaker and to the status signal, thereby determining a bandpass filtered template signal and a bandpass filtered status signal; and

determining, for each said speaker, a cross-correlation of the bandpass filtered template signal for said speaker with the bandpass filtered status signal.

17. The method of claim 16, wherein step (c) also includes a step of identifying, for each speaker of the set of N speakers, a difference between the bandpass filtered template signal for said speaker and the bandpass filtered status signal, from a frequency domain representation of the cross-correlation for said speaker.

18. The method of claim 1, wherein M=1, the audio data obtained in step (a) is indicative of a status signal captured by a microphone in the playback environment during emission of the sound in step (a), the template microphone is said microphone, and step (c) includes steps of:

determining, for each speaker of the set of N speakers, a sequence of cross-correlations of the template signal for said speaker with the status signal, wherein each of the cross-correlations is a cross-correlation of a segment of the template signal for said speaker with a corresponding segment of the status signal; and

identifying a difference between the template signal for said speaker and the status signal, from an average of the cross-correlations.

19. The method of claim 1, wherein M=1, the audio data obtained in step (a) is indicative of a status signal captured by a microphone in the playback environment during emission of the sound in step (a), the template microphone is said microphone, and step (c) includes steps of:

for each speaker of the set of N speakers, applying a bandpass filter to the template signal for said speaker and to the status signal, thereby determining a bandpass filtered template signal and a bandpass filtered status signal;

determining, for said each speaker, a sequence of cross-correlations of the bandpass filtered template signal for said speaker with the bandpass filtered status signal, wherein each of the cross-correlations is a cross-correlation of a segment of the bandpass filtered template signal for said speaker with a corresponding segment of the bandpass filtered status signal; and

identifying a difference between the bandpass filtered template signal for said speaker and the bandpass filtered status signal, from an average of the cross-correlations.

20. The method of claim 1, said method also including the steps of:

for each speaker-microphone pair consisting of one of the speakers and one template microphone of a set of M template microphones in the playback environment, determining an impulse response of the speaker by measuring sound emitted from said speaker at the initial time with the template microphone; and

for each of the channels, determining the convolution of the speaker feed for the channel with the impulse response of the speaker which is driven with said speaker feed in step (a), wherein said convolution determines the template signal employed in step (c) for the speaker-microphone pair employed to determine said convolution.

21. The method of claim 1, said method also including a step of:

for each speaker-microphone pair consisting of one of the speakers and one template microphone of a set of M template microphones in the playback environment, driving the speaker at the initial time with the speaker feed which drives said speaker in step (a), and measuring the sound emitted from said speaker in response to said speaker feed with the template microphone, wherein the measured sound determines the template signal employed in step (c) for said speaker-microphone pair.

22. The method of claim 1, said method also including the steps of:

(d) for each speaker-microphone pair consisting of one of the speakers and one template microphone of a set of M template microphones in the playback environment, determining an impulse response of the speaker by measuring sound emitted from said speaker at the initial time with the template microphone;

(e) for each of the channels, determining the convolution of the speaker feed for the channel with the impulse response of the speaker which is driven with said speaker feed in step (a); and

(f) for each of the channels, determining a bandpass filtered convolution by applying a bandpass filter to the convolution determined in step (e) for the channel, wherein said bandpass filtered convolution determines the template signal employed in step (c) for the speaker-microphone pair employed to determine said bandpass filtered convolution.

23. The method of claim 1, said method also including the steps of:

(d) for each speaker-microphone pair consisting of one of the speakers and one template microphone of a set of M template microphones in the playback environment, driving the speaker at the initial time with the speaker feed which drives said speaker in step (a), and employing the template microphone to generate a microphone output signal indicative of the sound emitted from said speaker in response to said speaker feed; and

(e) for each speaker-microphone pair, determining a bandpass filtered microphone output signal by applying a bandpass filter to the microphone output signal generated in step (d), wherein said bandpass filtered microphone output signal determines the template signal employed in step (c) for the speaker-microphone pair employed to determine said bandpass filtered microphone output signal.

24. The method of claim 1, wherein step (c) includes, for each speaker-microphone pair consisting of one of the speakers and one said microphone, the steps of:

(d) determining cross-correlation power spectra for the speaker-microphone pair, where each of the cross-correlation power spectra is indicative of a cross-correlation of the speaker feed for the speaker of said speaker-microphone pair and the speaker feed for another one of the set of N speakers;

(e) determining an auto-correlation power spectrum indicative of an auto-correlation of the speaker feed for the speaker of said speaker-microphone pair;

(f) filtering each of the cross-correlation power spectra and the auto-correlation power spectrum with a transfer function indicative of a room response for the speaker-microphone pair, thereby determining filtered cross-correlation power spectra and a filtered auto-correlation power spectrum;

(g) comparing the filtered auto-correlation power spectrum to a root mean square sum of all the filtered cross-correlation power spectra; and

(h) temporarily halting or slowing down the status check for the speaker of the speaker-microphone pair in response to determining that the root mean square sum is comparable to or greater than the filtered auto-correlation power spectrum.

25. The method of claim 24, wherein step (g) includes a step of comparing the filtered auto-correlation power spectrum and the root mean square sum on a frequency band-by-band basis, and step (h) includes a step of temporarily halting or slowing down the status check for the speaker of the speaker-microphone pair in each frequency band in which the root mean square sum is comparable to or greater than the filtered auto-correlation power spectrum.

26. A method for monitoring audience reaction to an audiovisual program played back by a playback system including a set of N speakers in a playback environment, where M is a positive integer, wherein the program has a soundtrack comprising M channels, said method including steps of:

(a) playing back the audiovisual program in the presence of an audience in the playback environment, including by emitting sound, determined by the program, from the speakers of the playback system in response to driving each of the speakers with a speaker feed for a different one of the channels of the soundtrack;

(b) obtaining audio data indicative of at least one microphone signal generated by at least microphone in the playback environment during emission of the sound in step (a); and

(c) processing the audio data to extract audience data from said audio data, and analyzing the audience data to determine audience reaction to the program, wherein the audience data are indicative of audience content indicated by the microphone signal, and the audience content comprises sound produced by the audience during playback of the program.

27. The method of claim 26, wherein the step of analyzing the audience data includes a step of performing pattern classification.

28. The method of claim 26, wherein the playback environment is a movie theater, and step (a) includes the step of playing back the program in the presence of the audience in the movie theater.

29. The method of claim 26, wherein step (c) includes a step of performing a spectral subtraction to remove, from the audio data, program data indicative of program content indicated by the microphone signal, wherein the program content consists of sound emitted from the speakers during playback of the program.

30. The method of claim 29, wherein the spectral subtraction includes a step of determining a difference between the microphone signal and a sum of filtered versions of speaker feed signals asserted to the speakers during step (a).

31. The method of claim 30, wherein the filtered versions of speaker feed signals are generated by applying filters to the speaker feeds, and each of the filters is an equalized room response of a different one of the speakers measured at the microphone.

32. A system for monitoring status of a set of M speakers in a playback environment, where M is a positive integer, said system including:

a set of M microphones positioned in the playback environment, where M is a positive integer; and

a processor coupled to each of the microphones in the set, wherein the processor is configured to process audio data to perform a status check on each speaker of the set of speakers, including by comparing, for each said speaker and each of at least one microphone in the set of microphones, a status signal captured by the microphone and a template signal, wherein the template signal is indicative of response of a template microphone to playback by the speaker, in the playback environment at an initial time, of a channel of the soundtrack corresponding to said speaker, and

wherein the audio data are indicative of a status signal captured by each microphone of the set of microphones during playback of an audiovisual program whose soundtrack has M channels, wherein said playback of the program includes emission of sound determined by the program from the speakers in response to driving each of the speakers with a speaker feed for a different one of the channels of the soundtrack.

33. The system of claim 32, wherein the audiovisual program is a movie trailer, and the playback environment is a movie theater.

34. The system of claim 32, wherein the audio data are indicative of a status signal captured by a microphone in the playback environment during playback of the program, and the template microphone is said microphone.

35. The system of claim 32, wherein the processor is configured to determine, for each speaker-microphone pair consisting of one of the speakers and one said microphone, a cross-correlation of the template signal for said speaker and microphone with the status signal for said microphone.

36. The system of claim 35, wherein the processor is configured to identify, for each said speaker-microphone pair, a difference between the template signal for the speaker and the microphone of the pair and the status signal for said microphone, from a frequency domain representation of the cross-correlation for said speaker-microphone pair.

37. The system of claim 35, wherein the processor is configured to:

determine a cross-correlation power spectrum for each said speaker-microphone pair, from the cross-correlation for said speaker-microphone pair;

determine a smoothed cross-correlation power spectrum for each said speaker-microphone pair from the cross-correlation power spectrum for said speaker-microphone pair; and

analyze the smoothed cross-correlation power spectrum for at least one said speaker-microphone pair to determine status of the speaker of said pair.

38. The system of claim 32, wherein the processor is configured to:

for each speaker-microphone pair consisting of one of the speakers and one said microphone, apply a bandpass filter to the template signal for said speaker and microphone, and to the status signal for said microphone, thereby determining a bandpass filtered template signal and a bandpass filtered status signal; and

determine, for each said speaker-microphone pair, a cross-correlation of the bandpass filtered template signal for said speaker and microphone with the bandpass filtered status signal for said microphone.

39. The system of claim 38, wherein the processor is configured to identify, for each speaker-microphone pair, a difference between the bandpass filtered template signal for the speaker and the microphone of the pair and the bandpass filtered status signal for said microphone, from a frequency domain representation of the cross-correlation for said speaker-microphone pair.

40. The system of claim 38, wherein the processor is configured to:

determine a cross-correlation power spectrum for each said speaker-microphone pair, from the cross-correlation for said speaker-microphone pair;

determine a smoothed cross-correlation power spectrum for each said speaker-microphone pair from the cross-correlation power spectrum for said speaker-microphone pair; and

analyze the smoothed cross-correlation power spectrum for at least one said speaker-microphone pair to determine status of the speaker of said pair.

41. The system of claim 32, wherein the processor is configured to:

determine, for each speaker-microphone pair consisting of one of the speakers and one said microphone, a sequence of cross-correlations of the template signal for said speaker and microphone with the status signal for said microphone, wherein each of the cross-correlations is a cross-correlation of a segment of the template signal for said speaker and microphone with a corresponding segment of the status signal for said microphone; and

identify a difference between the template signal for said speaker and microphone, and the status signal for said microphone, from an average of the cross-correlations.

42. The system of claim 32, wherein the processor is configured to:

for each speaker-microphone pair consisting of one of the speakers and one said microphone, apply a bandpass filter to the template signal for said speaker and microphone, and to the status signal for said microphone, thereby determining a bandpass filtered template signal and a bandpass filtered status signal;

determine, for each said speaker-microphone pair, a sequence of cross-correlations of the bandpass filtered template signal for said speaker and microphone with the bandpass filtered status signal for said microphone, wherein each of the cross-correlations is a cross-correlation of a segment of the bandpass filtered template signal for said speaker and microphone with a corresponding segment of the bandpass filtered status signal for said microphone; and

identify a difference between the bandpass filtered template signal for said speaker and microphone, and the bandpass filtered status signal for said microphone, from an average of the cross-correlations.

43. The system of claim 32, wherein M=1, the audio data are indicative of a status signal captured by a microphone in the playback environment during playback of the program, the template microphone is said microphone, and the processor is configured to determine, for each speaker of the set of M speakers, a cross-correlation of the template signal for said speaker with the status signal.

44. The system of claim 43, wherein the processor is configured to identify, for each speaker of the set of M speakers, a difference between the template signal for said speaker and the status signal, from a frequency domain representation of the cross-correlation for said speaker.

45. The system of claim 32, wherein M=1, the audio data are indicative of a status signal captured by a microphone in the playback environment during playback of the program, the template microphone is said microphone, and the processor is configured to:

for each speaker of the set of M speakers, apply a bandpass filter to the template signal for said speaker and to the status signal, thereby determining a bandpass filtered template signal and a bandpass filtered status signal; and

determine, for each said speaker, a cross-correlation of the bandpass filtered template signal for said speaker with the bandpass filtered status signal.

46. The system of claim 45, wherein the processor is configured to identify, for each speaker of the set of M speakers, a difference between the bandpass filtered template signal for said speaker and the bandpass filtered status signal, from a frequency domain representation of the cross-correlation for said speaker.

47. The system of claim 32, wherein M=1, the audio data are indicative of a status signal captured by a microphone in the playback environment during playback of the program, the template microphone is said microphone, and the processor is configured to:

determine, for each speaker of the set of M speakers, a sequence of cross-correlations of the template signal for said speaker with the status signal, wherein each of the cross-correlations is a cross-correlation of a segment of the template signal for said speaker with a corresponding segment of the status signal; and

identify a difference between the template signal for said speaker and the status signal, from an average of the cross-correlations.

48. The system of claim 32, wherein M=1, the audio data are indicative of a status signal captured by a microphone in the playback environment during playback of the program, the template microphone is said microphone, and the processor is configured to:

for each speaker of the set of M speakers, apply a bandpass filter to the template signal for said speaker and to the status signal, thereby determining a bandpass filtered template signal and a bandpass filtered status signal;

determine, for said each speaker, a sequence of cross-correlations of the bandpass filtered template signal for said speaker with the bandpass filtered status signal, wherein each of the cross-correlations is a cross-correlation of a segment of the bandpass filtered template signal for said speaker with a corresponding segment of the bandpass filtered status signal; and

identify a difference between the bandpass filtered template signal for said speaker and the bandpass filtered status signal, from an average of the cross-correlations.

49. The system of claim 32, wherein the processor is configured to:

for each speaker-microphone pair consisting of one of the speakers and one template microphone of a set of M template microphones in the playback environment, determine an impulse response of the speaker by measuring sound emitted from said speaker at the initial time with the template microphone; and

for each of the channels, determine the convolution of the speaker feed for the channel with the impulse response of the speaker which is driven with said speaker feed during capture of the status signal, wherein said convolution determines the template signal for the speaker-microphone pair employed to determine said convolution.

50. The system of claim 32, wherein the processor is configured to:

determine, for each speaker-microphone pair consisting of one of the speakers and one template microphone of a set of M template microphones in the playback environment, an impulse response of the speaker in response to sound measured from said speaker at the initial time with the template microphone;

determine, for each of the channels, the convolution of the speaker feed for the channel with the impulse response of the speaker which is driven with said speaker feed during capture of the status signal; and

determine, for each of the channels, a bandpass filtered convolution by applying a bandpass filter to the convolution determined for the channel, wherein said bandpass filtered convolution determines the template signal for the speaker-microphone pair employed to determine said bandpass filtered convolution.

51. A system for monitoring audience reaction to an audiovisual program played back by a playback system including a set of M speakers in a playback environment, where M is a positive integer, wherein the program has a soundtrack comprising M channels, said system including:

a set of M microphones positioned in the playback environment, where M is a positive integer; and

a processor coupled to at least one of the microphones in the set, wherein the processor is configured to process audio data to extract audience data from said audio data, and to analyze the audience data to determine audience reaction to the program,

wherein the audio data are indicative of at least one microphone signal generated by said at least one of the microphones during playback of an audiovisual program in the presence of an audience in the playback environment, said playback of the program including emission of sound determined by the program from the speakers of the playback system in response to driving each of the speakers with a speaker feed for a different one of the channels of the soundtrack, and wherein the audience data are indicative of audience content indicated by the microphone signal, and the audience content comprises sound produced by the audience during playback of the program.

52. The system of claim 51, wherein the processor is configured to analyze the audience data including by performing pattern classification.

53. The system of claim 51, wherein the processor is configured to perform a spectral subtraction to remove, from the audio data, program data indicative of program content indicated by the microphone signal, wherein the program content consists of sound emitted from the speakers during playback of the program.

54. The system of claim 53, wherein the processor is configured to perform the spectral subtraction such that said spectral subtraction includes a step of determining a difference between the microphone signal and a sum of filtered versions of speaker feed signals asserted to the speakers.

55. The system of claim 54, wherein the processor is configured to generate the filtered versions of the speaker feed signals by applying filters to the speaker feeds, and wherein each of the filters is an equalized room response of a different one of the speakers measured at the microphone.