System and method for media recognition

• US 8,768,495 B2
• Filed: 06/02/2011
• Issued: 07/01/2014
• Est. Priority Date: 06/09/2010
• Status: Active Grant

• Current Assignee: Soundmouse Limited
• Original Assignee: Adelphoi Ltd.
• Inventors: Owen, Mark St John, Selby, Alexander Paul
• Primary Examiner(s): Tsang, Fan
• Assistant Examiner(s): Zhao, Eugene
• Application Number:
  US13/151,365
• Publication Number:
  US 20110307085A1
• Time in Patent Office: 1,125 Days
• Field of Search: 711/94, 381/182, 707/999.102, 707/E17.009, 704/238, 700/94
• US Class Current: 700/94
• CPC Class Codes: G10L 25/18 the extracted parameters being spectral information of each sub-band
• CPC Class Codes: G10L 25/51 for comparison or discrimination

Abstract

Automatic recognition of sample media content is provided. A spectrogram is generated for successive time slices of audio signal. One or more sample hash vectors are generated for a time slice by calculating ratios of magnitudes of respective frequency bins from a column for the time slice. In a primary evaluation stage an exact match of bits of the sample hash vector is performed to entries in a look-up table to identify a group of one or more reference hash vectors. In a secondary evaluation stage a degree of similarity between the sample hash vector and each of the group of reference hash vectors is performed to identify any reference hash vectors that are candidates for matching the sample media content, each reference hash vector representing a time slice of reference media content.

Claims

1. An apparatus for providing automatic recognition of source media content from a source signal by comparison to reference media content, the apparatus including:

one or more computer systems configured to implement;

a spectrogram generator operable to generate a spectrogram from the source signal by applying a Fourier transform to the source signal, the spectrogram including a plurality of columns, each column being representative of a time slice and including a plurality of frequency bins each representative of a respective range of frequency components for the time slice of the source signal;

a vector generator operable to generate a plurality of source vectors including at least one source vector for each of respective time slices of the source signal, said at least one source vector for a said time slice of the source signal being generated by calculating ratios of magnitudes of selected frequency bins from the column for said time slice and quantizing the ratios to generate digits of said source vector, wherein a plurality of reference vectors represent the reference media content including at least one reference vector for each of respective time slices of the reference media content;

a primary evaluator operable to perform a primary evaluation by performing an exact match of digits of first vectors to entries in a look-up table, wherein each entry in the look-up table is associated with a group of second vectors, wherein the number of digits of the first vectors used to perform the exact match differs between entries in the look-up table, and wherein the first vectors are one of the source vectors and the reference vectors, and the second vectors are the other of the source vectors and the reference vectors;

a secondary evaluator operable to perform a secondary evaluation to determine a degree of similarity between the first vectors and each of the group of second vectors to identify any second vectors that are candidates for matching the source media content to the reference media content; and

a database comprising the look-up table and the second vectors.

2. The apparatus of claim 1, wherein, for generating said at least one source vector for a time slice, the vector generator is operable:

for at least one selected frequency bin of a time slice, to calculate ratios of that bin and an adjacent or a near adjacent frequency bin from the column for the time slice; and

to divide the ratios into ranges to generate at least one selected digit for each ratio.

3. The apparatus of claim 2, wherein for generating said at least one source vector for a time slice, the vector generator is operable:

for at least one selected frequency bin of a time slice, to calculate ratios of that bin and an adjacent or near adjacent frequency bin from the column for the time slice; and

to divide the ratios into ranges to generate two binary digits for each ratio.

4. The apparatus of claim 2, wherein:

the ranges differ between selected ratios to provide a substantially equal distribution of ratio values between ranges.

5. The apparatus of claim 2, wherein the vector generator is operable:

to generate a first source vector using frequency bins selected from a frequency band from 400 Hz to 1100 Hz and a second source vector using frequency bins selected from a frequency band from 1100 Hz to 3000 Hz.

6. The apparatus of claim 1, wherein, for generating a further source vector for a time slice:

the spectrogram generator is operable to generate a further spectrogram by applying a Fourier transform to the source signal, the further spectrogram including a plurality of columns, each column being representative of a time slice and including a plurality of frequency bins each representative of a respective range of frequency components for the time slice of the source signal and to apply a further Fourier transform to the respective frequency bins from the column for the time slice to generate a respective set of coefficients; and

the vector generator is operable to generate the further source vector such that, for a set of N coefficients in a column for a time slice, for each of elements 2 to N−

1 of the further source vector, an nth element is formed by the square of the nth coefficient divided by the product of the (n−

1)th coefficient and the (n+1)th coefficient; and

to quantise the elements of the resulting vector to generate at least one digit for each element.

7. The apparatus of claim 1, wherein the source signal is an audio signal and the frequencies of the spectrogram bins are allocated according to a logarithmic scale.

8. The apparatus of claim 1, wherein:

the look-up table is organised as a variable depth tree leading to leaves, the table being indexed by a first vector;

each leaf forms an entry in the look-up table associated with a respective group of second vectors;

the number of digits leading to each leaf is determined to provide substantially equally sized groups of second vectors for each leaf.

9. The apparatus of claim 8, wherein:

the number of digits leading to each leaf forms the number of digits of the first vector used to perform the exact match for a given leaf.

10. The apparatus of claim 8, wherein each leaf of the look-up table identifies a group of second vectors having d matching digits, wherein d corresponds to the depth of the tree to that leaf.

11. The apparatus of claim 1, wherein the secondary evaluator is operable to perform the secondary evaluation using a distance metric to determine the degree of similarity between the first vector and each of the group of second vectors.

12. The apparatus of claim 1, the one or more computer systems further configured to implement a tertiary evaluator for performing a tertiary evaluation for any second vector identified as a candidate, the tertiary evaluator being operable to determine a degree of similarity between one or more further first vectors and one or more further second vectors corresponding to the candidate second vector identified in the secondary evaluation.

13. The apparatus of claim 12, where the further first vectors and the further second vectors are separated in time from the first vector and the candidate second vector, respectively.

14. The apparatus of claim 1, wherein the source signal is a received programme signal.

15. The apparatus of claim 14, the one or more computer systems further configured to implement a record generator operable to generate a record of the matched media content of the programme signal.

16. The apparatus of claim 15, the one or more computer systems further configured to implement a cue sheet generator operable to generate a cue sheet identifying the matched media content.

17. The apparatus of claim 1, wherein the second vectors are the source vectors and the apparatus is configured to generate the database from the source vectors.

18. The apparatus of claim 1, wherein the one or more computer systems include at least one processor and storage and computer software operable to implement the spectrogram generator, the vector generator and the evaluators.

19. A computer-implemented recognition method for the automatic recognition of source media content from a source signal by comparison to reference media content, the method including:

generating a spectrogram from the source signal by applying a Fourier transform to the source signal, the spectrogram including a plurality of columns, each column being representative of a time slice and including a plurality of frequency bins each representative of a respective range of frequency components for the time slice of the source signal;

generating a plurality of source vectors including at least one source vector for each of respective time slices of the source signal, said at least one source vector for a said time slice of the source signal being generated by calculating ratios of magnitudes of selected frequency bins from the column for said time slice and quantizing the ratios to generate digits of said source vector, wherein a plurality of reference vectors represent the reference media content including at least one reference vector for each of respective time slices of the reference media content;

performing a primary evaluation by exact matching of digits of first vectors to entries in a look-up table, wherein each entry in the look-up table is associated with a group of second vectors, wherein the number of digits of the first vectors used to perform the exact match differs between entries in the look-up table, and wherein the first vectors are one of the source vectors and the reference vectors, and the second vectors are the other of the source vectors and the reference vectors; and

performing a secondary evaluation to determine a degree of similarity between the first vectors and each of the group of second vectors to identify any second vectors that are candidates for matching the source media content to the reference media content,

wherein a database stores the look-up table and the second vectors.

20. A non-transitory machine readable medium carrying program instructions configured to cause one or more processors of one or more computer systems to perform an automatic recognition method for the automatic recognition of source media content from a source signal by comparison to reference media content, the method including:

generating a spectrogram from the source signal by applying a Fourier transform to the source signal, the spectrogram including a plurality of columns, each column being representative of a time slice and including a plurality of frequency bins each representative of a respective range of frequency components for the time slice of the source signal;

generating a plurality of source vectors including at least one source vector for each of respective time slices of the source signal, said at least one source vector for a said time slice of the source signal being generated by calculating ratios of magnitudes of selected frequency bins from the column for said time slice and quantizing the ratios to generate digits of said source vector, wherein a plurality of reference vectors represent the reference media content including at least one reference vector for each of respective time slices of the reference media content;

performing a primary evaluation by exact matching of digits of first vectors to entries in a look-up table, wherein each entry in the look-up table is associated with a group of second vectors, wherein the number of digits of the first vectors used to perform the exact match differs between entries in the look-up table, and wherein the first vectors are one of the source vectors and the reference vectors, and the second vectors are the other of the source vectors and the reference vectors; and

performing a secondary evaluation to determine a degree of similarity between the first vectors and each of the group of second vectors to identify any second vectors that are candidates for matching the source media content to the reference media content,

wherein a database stores the look-up table and the second vectors.