Audio signature extraction and correlation

• US 7,672,843 B2
• Filed: 06/02/2005
• Issued: 03/02/2010
• Est. Priority Date: 10/27/1999
• Status: Expired due to Term

• Current Assignee: Nielsen Company (US), LLC
• Original Assignee: The Nielsen Company LLC (MOOD Media Corporation)
• Inventors: Deng, Keqiang, Srinivasan, Venugopal, Lu, Daozheng
• Primary Examiner(s): Han; Qi
• Application Number:
  US11/143,808
• Publication Number:
  US 20050232411A1
• Time in Patent Office: 1,734 Days
• Field of Search: 704/231, 704/246, 704/243, 725/18
• US Class Current: 704/231
• CPC Class Codes: G10L 25/48 specially adapted for particular use
• CPC Class Codes: H04H 60/29 Arrangements for monitoring broadcast services or broadcast-related services
• CPC Class Codes: H04H 60/58 of audio determination or detection of speech characteristics in general G10L25/00; speech recognition in general G10L15/00

Abstract

A signature is extracted from the audio of a program received by a tunable receiver such that the signature characterizes the program. In order to extract the signature, blocks of the audio are converted to corresponding spectral moments. At least one of the spectral moments is then converted to the signature. Also, a test audio signal from a receiver is correlated to a reference audio signal by converting the test audio signal and the reference audio signal to corresponding test and reference spectra, determining test slopes corresponding to coefficients of the test spectrum and reference slopes corresponding to coefficients of the reference spectrum, and comparing the test slopes to the reference slopes in order to determine a match between the test audio signal and the reference audio signal.

Claims

1. A method of extracting a signature from audio of a program received by a receiver, wherein the signature characterizes the program, the method comprising:

converting the audio to corresponding spectral moments; and

converting at least one of the spectral moments to the signature by iteratively smoothing the spectral moments and converting the smoothed spectral moments to the signature, wherein iteratively smoothing the spectral moments comprises iteratively smoothing the spectral moments according to the following equation;

2. The method of claim 1 wherein the audio has a spectral power, and wherein the conversion of the audio to corresponding spectral moments comprises determining the spectral moments from the spectral power of the audio.

3. The method of claim 2 wherein determining the spectral moments from the spectral power of the audio comprises determining the spectral moments from the spectral power of the audio according to the following equation:

4. The method of claim 3 wherein T_k is based upon a FFT of the audio.

5. The method of claim 3 wherein T_k is based upon a MDCT of the audio.

6. The method of claim 3 wherein the signature is (A_n, D_n), wherein A_n is an amplitude of a peak of the spectral moments, and wherein D_n is a time duration between the peak of the spectral moments and a neighboring peak of the spectral moments.

7. The method of claim 1 wherein the signature is (A_n, D_n), wherein A_n is an amplitude of a peak of the smoothed spectral moments, and wherein D_n is a time duration between the peak of the smoothed spectral moments and a neighboring peak of the smoothed spectral moments.

8. The method of claim 1 wherein the audio has a spectral power, and wherein converting at least one of the spectral moments to the signature comprises determining the spectral moments from the spectral power of the audio according to the following equation:

9. The method of claim 1 wherein converting at least one of the spectral moments to the signature comprises converting blocks of the audio to corresponding spectral moments, and wherein each of the blocks contains a number of samples of the audio.

10. The method of claim 9 wherein each of the blocks contains N samples of the audio, and wherein each block contains N/2 old samples and N/2 new samples.

11. The method of claim 1 wherein the signature is a signature S, and wherein the method further comprises comparing the signature S to a reference signature R.

12. The method of claim 11 wherein the signature S is derived from a FFT, and wherein the reference signature R is derived from a FFT.

13. The method of claim 11 wherein the signature S is derived from a MDCT, and wherein the reference signature R is derived from a MDCT.

14. The method of claim 11 wherein one of the signature S and the reference signature R is derived from a FFT, and wherein the other of the signature S and the reference signature R is derived from a MDCT.

15. A method of extracting a signature from a digital program received by a digital receiver, wherein the signature characterizes the digital program, the method comprising:

obtaining an encoded audio stream from the digital receiver;

decoding the encoded audio stream to obtain modified discrete cosine transform (MDCT) coefficients representing audio blocks;

determining a spectral moment based on spectral power represented by the MDCT coefficients, wherein the spectral power represented by the MDCT coefficients is weighted by a frequency index when determining the spectral moment;

smoothing the spectral moment by iterative averaging of spectral moments across a plurality of audio blocks; and

converting the smoothed spectral moment into the signature based on amplitude maxima in the smoothed spectral moment.

16. The method of claim 15, wherein the plurality of audio blocks comprises 32 audio blocks.

17. The method of claim 16, wherein the smoothing is carried out according to the following equation:

18. The method of claim 17, wherein the signature comprises two bytes of data, wherein a first byte of data represents a maximum of a peak amplitude of the smoothed spectral moment.

19. The method of claim 15, wherein the spectral moment is based on spectral power represented by MDCT coefficients representing a frequency range from about 4.3 kilohertz (kHz) to about 6.5 kHz.

20. The method of claim 15, wherein determining a spectral moment based on spectral power represented by the MDCT coefficients is performed according to the following equation:

21. The method of claim 20, wherein k₁ and k₂ are frequency indices that are approximately 45 and approximately 70, respectively.