Audio signature extraction and correlation

• US 20050232411A1
• Filed: 06/02/2005
• Published: 10/20/2005
• Est. Priority Date: 10/27/1999
• Status: Active Grant

• Current Assignee: Nielsen Company (US), LLC
• Original Assignee: Nielsen Company (US), LLC
• Inventors: Srinivasan, Venugopal, Lu, Daozheng, Deng, Keqiang
• Granted Patent: US 7,672,843 B2
• Time in Patent Office: Days
• US Class Current: 379/413
• CPC Class Codes: G10L 25/48 specially adapted for particular use
• CPC Class Codes: H04H 60/29 Arrangements for monitoring broadcast services or broadcast-related services
• CPC Class Codes: H04H 60/58 of audio determination or detection of speech characteristics in general G10L25/00; speech recognition in general G10L15/00

Abstract

A signature is extracted from the audio of a program received by a tunable receiver such that the signature characterizes the program. In order to extract the signature, blocks of the audio are converted to corresponding spectral moments. At least one of the spectral moments is then converted to the signature. Also, a test audio signal from a receiver is correlated to a reference audio signal by converting the test audio signal and the reference audio signal to corresponding test and reference spectra, determining test slopes corresponding to coefficients of the test spectrum and reference slopes corresponding to coefficients of the reference spectrum, and comparing the test slopes to the reference slopes in order to determine a match between the test audio signal and the reference audio signal.

Claims

1. A method of extracting a signature from audio of a program received by a tunable receiver, wherein the signature characterizes the program, and wherein the method comprises the following steps:

a) converting the audio to corresponding spectral moments; and

,

b) converting at least one of the spectral moments to the signature.

2. The method of claim 1 wherein the audio has a spectral power, and wherein step a) comprises the step of determining the spectral moments from the spectral power of the audio.

3. The method of claim 2 wherein the step of determining the spectral moments from the spectral power of the audio comprises the step of determining the spectral moments from the spectral power of the audio according to the following equation;

4. The method of claim 3 wherein T_k is based upon a FFT of the audio.

5. The method of claim 3 wherein T_k is based upon a MDCT of the audio.

6. The method of claim 3 wherein the signature is (A_n, D_n), wherein A_n is an amplitude of a peak of the spectral moments, and wherein D_n is a time duration between the peak of the spectral moments and a neighboring peak of the spectral moments.

7. The method of claim 1 wherein step b) comprises the steps of iteratively smoothing the spectral moments resulting from step a) and converting the smoothed spectral moments to the signature.

8. The method of claim 7 wherein the step of iteratively smoothing the spectral moments resulting from step a) comprises the step of iteratively smoothing the spectral moments resulting from step a) according to the following equation:

9. The method of claim 7 wherein the signature is (A_n, D_n), wherein A_n is an amplitude of a peak of the smoothed spectral moments, and wherein D_n is a time duration between the peak of the smoothed spectral moments and a neighboring peak of the smoothed spectral moments.

10. The method of claim 7 wherein the audio has a spectral power, and wherein step a) comprises the step of determining the spectral moments from the spectral power of the audio according to the following equation;

11. The method of claim 1 wherein step a) comprises the step of converting blocks of the audio to corresponding spectral moments, and wherein each of the blocks contains a number of samples of the audio.

12. The method of claim 11 wherein each of the blocks contains N samples of the audio, and wherein each block contains N/2 old samples and N/2 new samples.

13. The method of claim 1 wherein the signature is a signature S, and wherein the method further comprises the step of comparing the signature S to a reference signature R.

14. The method of claim 13 wherein the signature S is derived from a FFT, and wherein the reference signature R is derived from a FFT.

15. The method of claim 13 wherein the signature S is derived from a MDCT, and wherein the reference signature R is derived from a MDCT.

16. The method of claim 13 wherein one of the signature S and the reference signature R is derived from a FFT, and wherein the other of the signature S and the reference signature R is derived from a MDCT.

17. A method of extracting a signature from a program received by a tunable receiver, wherein the signature characterizes the program, and wherein the method comprises the following steps:

a) converting the program to a corresponding frequency related spectrum; and

,

b) converting a frequency related component of the frequency related spectrum to the signature.

18. The method of claim 17 wherein the frequency related component has a spectral power, and wherein step b) comprises the step converting the spectral power of the frequency related component to the signature.

19. The method of claim 18 wherein the spectral power is based upon a FFT of the program.

20. The method of claim 18 wherein the spectral power is based upon a MDCT of the program.

21. The method of claim 17 wherein step a) comprises the step converting a plurality of blocks of the program to corresponding frequency related spectra, wherein each of the blocks contains a number of samples of the program, and wherein step b) comprises the step of converting frequency related components of the frequency related spectra to the signature.

22. The method of claim 21 wherein each of the frequency related components has a corresponding spectral power, and wherein step b) comprises the step converting the spectral powers of the frequency related components to the signature.

23. The method of claim 22 wherein the spectral powers are based upon a FFT of the program.

24. The method of claim 22 wherein the spectral powers are based upon a MDCT of the program.

25. The method of claim 21 wherein each of the blocks contains N samples of the audio, and wherein each block contains N/2 old samples and N/2 new samples.

26. The method of claim 17 wherein step a) comprises the step of converting blocks of the audio to corresponding spectral moments, and wherein each of the blocks contains a number of samples of the audio.

27. The method of claim 26 wherein each of the blocks contains N samples of the audio, and wherein each block contains N/2 old samples and N/2 new samples.

28. The method of claim 17 wherein the signature is a signature S, and wherein the method further comprises the step of comparing the signature S to a reference signature R.

29. The method of claim 28 wherein the signature S is derived from a FFT, and wherein the reference signature R is derived from a FFT.

30. The method of claim 28 wherein the signature S is derived from a MDCT, and wherein the reference signature R is derived from a MDCT.

31. The method of claim 28 wherein one of the signature S and the reference signature R is derived from a FFT, and wherein the other of the signature S and the reference signature R is derived from a MDCT.

32. A method of correlating a test audio signal derived from a receiver to a reference audio signal comprising the following steps:

a) converting the test audio signal to a corresponding frequency related test spectrum;

b) selecting segments between frequency related components of the frequency related test spectrum as test segments; and

,

c) comparing the test segments to reference segments derived from the reference audio signal in order to determine a match between the test audio signal and the reference audio signal.

33. The method of claim 32 wherein step c) comprises the following steps:

converting the reference audio signal to a corresponding frequency related reference spectrum; and

,

selecting segments between frequency related components of the frequency related reference spectrum as the reference segments.

34. The method of claim 33 wherein the test audio signal is converted to a corresponding frequency related test spectrum by a FFT, and wherein the reference audio signal is converted to a corresponding frequency related reference spectrum by a FFT.

35. The method of claim 33 wherein the test audio signal is converted to a corresponding frequency related test spectrum by a MDCT, and wherein the reference audio signal is converted to a corresponding frequency related reference spectrum by a MDCT.

36. The method of claim 33 wherein, in step c), only test segments associated with frequency components having a magnitude greater than a first minimum are compared to reference segments associated with frequency components having a magnitude greater than a second minimum in order to determine a match between the test audio signal and the reference audio signal.

37. The method of claim 36 wherein the first minimum is equal to the second minimum.

38. The method of claim 36 wherein a ratio of the number of matches between the test segments and the reference segments to the total number of reference segments must exceed a minimum in order to determine a match between the test audio signal and the reference audio signal.

39. The method of claim 33 wherein the test segments have corresponding slopes, wherein the reference segments have corresponding slopes, and wherein step c) comprises the step of comparing the slopes of the test segments to the slopes of the reference segments in order to determine a match between the test audio signal and the reference audio signal.

40. The method of claim 39 wherein, in step c), only slopes of test segments associated with frequency components having a magnitude greater than a first minimum are compared to slopes of reference segments associated with frequency components having a magnitude greater than a second minimum in order to determine a match between the test audio signal and the reference audio signal.

41. The method of claim 40 wherein the first minimum is equal to the second minimum.

42. The method of claim 40 wherein a ratio of the number of matches between the slopes of test segments and the slopes of the reference segments to the total number of reference segments must exceed a minimum in order to determine a match between the test audio signal and the reference audio signal.

43. The method of claim 32 wherein step a) comprises the step of converting blocks of the test audio signal to a corresponding frequency related test spectrum, and wherein each of the blocks contains a number of samples of the test audio signal.

44. The method of claim 43 wherein each of the blocks contains N samples of the test audio signal, and wherein each block contains N/2 old samples and N/2 new samples.

45. A method of correlating a test audio signal derived from a receiver to a reference audio signal comprising the following steps:

a) converting the test audio signal to a test spectrum;

b) determining test slopes corresponding to coefficients of the test spectrum;

c) converting the reference audio signal to a reference spectrum;

d) determining reference slopes corresponding to coefficients of the reference spectrum; and

,

e) comparing the test slopes to the reference slopes in order to determine a match between the test audio signal and the reference audio signal.

46. The method of claim 45 wherein the test audio signal is converted to the test spectrum by a FFT, and wherein the reference audio signal is converted to the reference spectrum by a FFT.

47. The method of claim 45 wherein the test audio signal is converted to the test spectrum by a MDCT, and wherein the reference audio signal is converted to the reference spectrum by a MDCT.

48. The method of claim 45 wherein, in step e), only test slopes associated with coefficients having a magnitude greater than a first minimum are compared to reference slopes associated with coefficients having a magnitude greater than a second minimum in order to determine a match between the test audio signal and the reference audio signal.

49. The method of claim 48 wherein the first minimum is equal to the second minimum.

50. The method of claim 48 wherein a ratio of the number of matches between the test slopes and the reference slopes to the total number of reference slopes must exceed a minimum in order to determine a match between the test audio signal and the reference audio signal.