Content identification system

• US 20050027766A1
• Filed: 07/29/2003
• Published: 02/03/2005
• Est. Priority Date: 07/29/2003
• Status: Active Grant

• Current Assignee: Alcatel-Lucent SA (Nokia Corporation)
• Original Assignee: Alcatel-Lucent SA (Nokia Corporation)
• Inventors: Mousavi, Madjid Sam, Nohl, Craig R., Ben, Jan I., Burges, Christopher J.
• Granted Patent: US 8,918,316 B2
• Time in Patent Office: Days
• US Class Current: 708/300
• CPC Class Codes: G10L 25/48 specially adapted for particular use
• CPC Class Codes: G11B 27/11 by using information not detectable on the record carrier
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence

Abstract

The content of a media program is recognized by analyzing its audio content to extract therefrom prescribed features, which are compared to a database of features associated with identified content. The identity of the content within the database that has features that most closely match the features of the media program being played is supplied as the identity of the program being played. The features are extracted from a frequency domain version of the media program by a) filtering the coefficients to reduce their number, e.g., using triangular filters; b) grouping a number of consecutive outputs of triangular filters into segments; and c) selecting those segments that meet prescribed criteria, such as those segments that have the largest minimum segment energy with prescribed constraints that prevent the segments from being too close to each other. The triangular filters may be log-spaced and their output may be normalized.

Claims

1. A method for use in recognizing the content of a media program, said method comprising the steps of:

filtering each first frequency domain representation of blocks of said media program using a plurality of filters to develop a respective second frequency domain representation of each of said blocks of said media program, said second frequency domain representation of each of said blocks having a reduced number of frequency coefficients with respect to said first frequency domain representation;

grouping frequency coefficients of said second frequency domain representation of said blocks to form segments; and

selecting a plurality of said segments.

2. The invention as defined in claim 1 wherein each grouping of frequency coefficients of said second frequency domain to form a segment represents blocks that are consecutive in time in said media program.

3. The invention as defined in claim 1 wherein said plurality of filters are arranged in a group that processes a block at a time, the portion of said second frequency domain representation produced by said group for each block forms a frame, and wherein at least two frames are grouped to form a segment.

4. The invention as defined in claim 1 wherein said selected segments correspond to portions of said media program that are not contiguous in time.

5. The invention as defined in claim 1 wherein said plurality of filters includes at least a set of triangular filters.

6. The invention as defined in claim 1 wherein said plurality of filters includes at least a set of log-spaced triangular filters.

7. The invention as defined in claim 1 wherein the segments selected in said selecting step are those that have largest minimum segment energy.

8. The invention as defined in claim 1 wherein the segments selected in said selecting step are selected in accordance with prescribed constraints such that said segments are prevented from being too close to each other.

9. The invention as defined in claim 1 wherein the segments selected in said selecting step are selected for portions of said media program that correspond in time to prescribed search windows that are separated by gaps.

10. The invention as defined in claim 1 wherein the segments selected in said selecting step are those that result in the selected segments having a maximum entropy over the selected segments.

11. The invention as defined in claim 1 further comprising the step of normalizing said frequency coefficients in said second frequency domain representation after performing said grouping step, said normalization being performed on a per-segment basis.

12. The invention as defined in claim 11 wherein said normalization step includes performing at least a preceding-time normalization.

13. The invention as defined in claim 11 wherein said normalization is step includes performing at least an L2 normalization.

14. The invention as defined in claim 1 further comprising the step of storing said selected segments in a database in association with an identifier of said media program.

15. The invention as defined in claim 14 further comprising the step of storing in said database information indicating timing of said selected segments.

16. The invention as defined in claim 1 wherein said first frequency domain representation of blocks of said media program is developed by the steps of:

digitizing an audio representation of said media program to be stored in said database;

dividing the digitized audio representation into blocks of a prescribed number of samples;

smoothing said blocks using a filter; and

converting said smoothed blocks into the frequency domain, wherein said smoothed blocks are represented by frequency coefficients.

17. The invention as defined in claim 16 wherein said filter used in said smoothing step is a Hamming window filter.

18. The invention as defined in claim 16 wherein each of said smoothed blocks are converted into the frequency domain in said converting step using a Fast Fourier Transform (FFT).

19. The invention as defined in claim 16 wherein each of said smoothed blocks are converted into the frequency domain in said converting step using a Discrete Cosine Transform (DCT).

20. A method for identifying the content of a media program, comprising the steps of:

comparing a digital representation of the content of said media program to be identified with digital representations of the content of a plurality of media programs stored in a database; and

identifying the content of said media program to be identified as the one of said media programs having a digital representation of its content stored in said database that most closely matches said digital representation of the content of said media program to be identified.

21. A method for use in recognizing the content of a media program, comprising the steps of:

filtering a first frequency domain representation of said media program using a plurality of filters to develop a second frequency domain representation of said media program having a reduced number of frequency coefficients in said second frequency domain representation with respect to said first frequency domain representation;

grouping ones of said second frequency domain representation to form segments; and

selecting a plurality of said segments.

22. Apparatus for use in recognizing the content of a media program, comprising:

a plurality of filters for filtering a first representation of said media program using frequency coefficients to develop a second representation of said media program that has a reduced number of frequency coefficients with respect to said first representation;

means for grouping ones of said coefficients of said second representation to form segments; and

means for selecting a plurality of said segments.

23. Apparatus for use in recognizing the content of a media program, comprising:

means for filtering a first frequency domain representation of said media program using a plurality of filters to develop a second frequency domain representation of said media program having a reduced number of frequency coefficients in said second frequency domain representation with respect to said first frequency domain representation;

means for grouping ones of said second frequency domain representation to form segments; and

means for selecting a plurality of said segments.

24. A method for use in recognizing the content of a media program, said method comprising the steps of:

filtering each first frequency domain representation of blocks of said media program using a plurality of filters to develop a respective second frequency domain representation of each of said blocks of said media program, said second frequency domain representation of each of said blocks having a reduced number of frequency coefficients with respect to said first frequency domain representation;

grouping frequency coefficients of said second frequency domain representation of said blocks to form segments; and

searching a database for substantially matching segments, said database having stored therein segments of media programs and respective corresponding program identifiers.

25. The invention as defined in claim 24 further comprising the step of indicating that said media program cannot be identified when substantially matching segments are not found in said database in said searching step.

26. The invention as defined in claim 24 wherein said data base includes information indicating timing of segments of each respective media program identified therein, and wherein a match may be found in said searching step only when the timing of said segments produced in said grouping step substantially matches the timing of said segments stored in said database.

27. The invention as defined in claim 24 wherein said matching between segments is based on the Euclidean distances between segments.

28. The invention as defined in claim 24 further comprising the step of identifying said media program as being the media program indicated by the identifier stored in said database having a best matching score when substantially matching segments are found in said database in said searching step.

29. The invention as defined in claim 28 further comprising the step of determining a speed differential between said media program and a media program identified in said identifying step.

30. The invention as defined in claim 28 wherein said matching score for a program P_i is determined by

$P_i=\frac{1}{z}$

∑

j = 1 z ⁢

f ⁡

( S j ′

- S j ⁡

( P i ) ) .

31. The invention as defined in claim 28 further comprising the steps of:

repeating said filtering, grouping, searching and identifying; and

determining, in the event of another match, whether said identified program is the same program determined prior to said repetition or a different program.

32. The invention as defined in claim 31 wherein said determining step is based on an overlap score.

33. The invention as defined in claim 32 wherein overlap score is calculated between said program determined prior to said repetition, P0, and said program determined during said repetition, P1, is calculated as

Overlap score=(t_end−

t_begin)/(end time of P1−

beginning time of P1)

where

t_end is min(end time of P0, P1); and

t_begin is max(beginning time of P0, P1).

34. A method for use in recognizing the content of a media program, said method comprising the steps of:

filtering a first frequency domain representation of said media program using a plurality of filters to develop a second frequency domain representation of said media program having a reduced number of frequency coefficients in said second frequency domain representation with respect to said first frequency domain representation;

grouping ones of said second frequency domain representation to form segments; and

searching a database for substantially matching segments, said database having stored therein segments of media programs and respective corresponding program identifiers.

35. Apparatus for use in recognizing the content of a media program, comprising:

means for filtering a first frequency domain representation of said media program using a plurality of filters to develop a second frequency domain representation of said media program having a reduced number of frequency coefficients in said second frequency domain representation with respect to said first frequency domain representation;

means for grouping ones of said second frequency domain representation to form segments; and

means for searching a database for substantially matching segments, said database having stored therein segments of media programs and respective corresponding program identifiers.

36. The invention as defined in claim 35 wherein said first frequency domain representation of said media program comprises a plurality of blocks of coefficients corresponding to respective time domain sections of said media program and said second frequency domain representation of said media program comprises a plurality of blocks of coefficients corresponding to respective time domain sections of said media program.

37. A computer readable storage arranged to store segments derived from, and representative of, various media programs, said segments of each respective one of said media programs being stored in said database so as to be associated with a respective media program identifier.

38. The invention as defined in claim 37 wherein each of said media program identifiers is unique.

39. The invention as defined in claim 37 wherein each of said segments is developed by

filtering a first frequency domain representation of said media program using a plurality of filters to develop a second frequency domain representation of said media program having a reduced number of frequency coefficients in said second frequency domain representation with respect to said first frequency domain representation;

grouping ones of said second frequency domain representation to form said segments

40. A method for identifying a media program to be identified, the method comprising the steps of

comparing segments of said media program to be identified with segments representative of various media programs that are stored in a database, said segments of each respective media program stored in said database being stored in association with a respective media program identifier;

and

identifying said media program to be identified with the media program identifier that is associated with the stored segments that most closely matches said segments of said media program to be identified.