Unordered matching of audio fingerprints

• US 9,460,201 B2
• Filed: 05/06/2013
• Issued: 10/04/2016
• Est. Priority Date: 05/06/2013
• Status: Active Grant

• Current Assignee: iHeartMedia Management Services, Inc. (iHeartMedia, Inc.)
• Original Assignee: iHeartMedia Management Services, Inc. (iHeartMedia, Inc.)
• Inventors: Anniballi, Dyon
• Primary Examiner(s): Tsang, Fan
• Assistant Examiner(s): Zhao, Eugene
• Application Number:
  US13/887,988
• Publication Number:
  US 20140330413A1
• Time in Patent Office: 1,247 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/638 Presentation of query results
• CPC Class Codes: G06F 16/683 using metadata automatically derived from the content

Abstract

A method includes receiving an audio fingerprint from a listening device. The method also includes, in response to determining that a portion of a stored audio fingerprint substantially matches a portion of the received audio fingerprint, identifying a longest unordered match between the received audio fingerprint and the stored audio fingerprint that satisfies a similarity threshold. The method further includes, in response to determining that the identified longest unordered match satisfies a length criterion, detecting a match between the received audio fingerprint and the stored audio fingerprint.

Claims

1. A method comprising:

receiving an audio fingerprint from a listening device;

in response to determining that a portion of a stored audio fingerprint substantially matches a portion of the received audio fingerprint;

for each portion of a first set of consecutive portions of the received audio fingerprint, generating multiple corresponding match scores based on comparing the portion of the first set of consecutive portions to multiple portions of a second set of portions of the stored audio fingerprint; and

identifying a longest unordered match as a number of consecutive portions of the first set that each have a corresponding highest match score that satisfies a similarity threshold; and

in response to determining that the identified longest unordered match satisfies a length criterion, detecting a match between the received audio fingerprint and the stored audio fingerprint.

2. The method of claim 1, wherein the portion of the stored audio fingerprint is determined to substantially match the portion of the received audio fingerprint when the portion of the stored audio fingerprint and the portion of the received audio fingerprint have a hamming distance that is less than a distance threshold.

3. The method of claim 1, wherein identifying the longest unordered match comprises identifying a highest match score for each portion of the first set of consecutive portions of the received audio fingerprint, and wherein detecting the match comprises comparing the longest unordered match to the length criterion.

4. The method of claim 1, further comprising receiving a second audio fingerprint from a field recorder and storing the second audio fingerprint as the stored audio fingerprint.

5. The method of claim 1, wherein the received audio fingerprint includes X portions of spectral data, wherein the stored audio fingerprint includes Y portions of spectral data, wherein each of the X portions is a non-overlapping portion, and wherein X and Y are integers greater than one.

6. The method of claim 5, wherein determining that the portion of the stored audio fingerprint substantially matches the portion of the received audio fingerprint comprises:

selecting a probe from the stored audio fingerprint; and

determining that the probe substantially matches the portion of the received audio fingerprint.

7. The method of claim 6, wherein the probe comprises sixteen audio samples that are each one tenth of a second in length.

8. The method of claim 1, wherein, for each portion of the first set of consecutive portions, the multiple corresponding match scores include a score for each portion of the second set of portions of the stored audio fingerprint, and further comprising:

for each of the first set of consecutive portions of the received audio fingerprint, identifying a highest match score; and

generating a list of runs based on the identified highest match score for each of the first set of consecutive portions, wherein the longest unordered match is identified based on the list of runs.

9. The method of claim 8, wherein identifying the highest score for a particular portion of the received audio fingerprint comprises excluding at least one match score that corresponds to a portion of the stored audio fingerprint that violates a disorder threshold.

10. The method of claim 8, further comprising applying a positional penalty to at least one of the highest scores in response to determining that the highest scores for two consecutive portions of the received audio fingerprint correspond to two non-consecutive portions of the stored audio fingerprint.

11. The method of claim 8, wherein:

the list of runs includes alternating below-threshold runs and above-threshold runs,

each below-threshold run corresponds to a sequence of one or more of the first set of consecutive portions that have a highest score that is less than the similarity threshold,

each above-threshold run corresponds to a sequence of one or more of the first set of consecutive portions that have a highest score that is greater than or equal to the similarity threshold, and

identifying the longest unordered match based on the list of runs comprises identifying a longest above-threshold run in the list of runs as the longest unordered match.

12. The method of claim 11, further comprising replacing a below-threshold run in the list of runs with an aggregate of the below-threshold run and neighboring above-threshold runs in response to determining that the aggregate has an average match score that is greater than or equal to the similarity threshold.

13. The method of claim 11, further comprising replacing an above-threshold run in the list of runs with an aggregate of the above-threshold run and neighboring below-threshold runs in response to determining that the aggregate has an average match score that is less than the similarity threshold.

14. The method of claim 1, further comprising, in response to detecting the match between the received audio fingerprint and the stored audio fingerprint, storing data indicating a station associated with the stored fingerprint, a timestamp corresponding to the received fingerprint, and a user identifier associated with the listening device.

15. The method of claim 1, wherein the length criterion is based at least in part on a programming element represented by the stored audio fingerprint, wherein the programming element comprises an advertisement programming element or a song programming element.

16. The method of claim 1, wherein the length criterion is based at least in part on a station associated with the stored audio fingerprint, wherein the station comprises a radio station, a television channel, or an Internet media source.

17. An apparatus comprising:

a network interface configured to receive an audio fingerprint from a listening device;

a processor; and

a memory storing instructions that, when executed by the processor, cause the processor to perform operations comprising;

in response to determining that a portion of a stored audio fingerprint substantially matches a portion of the received audio fingerprint;

for each portion of a first set of consecutive portions of the received audio fingerprint, generating multiple corresponding match scores based on comparing the portion of the first set of consecutive portions to multiple portions of a second set of portions of the stored audio fingerprint; and

identifying a longest unordered match as a number of a highest number of consecutive portions of the first set that each have a corresponding match score that satisfies a similarity threshold; and

in response to determining that the identified longest unordered match satisfies a length criterion, detecting a match between the received audio fingerprint and the stored audio fingerprint.

18. The apparatus of claim 17, wherein the memory stores audio fingerprints corresponding to eight days of audio transmissions associated with a plurality of stations, wherein the length criterion is user configurable, and wherein the listening device comprises a microphone, a mobile telephone, a personal media monitor, or any combination thereof.

19. A non-transitory computer-readable storage device storing instructions that, when executed by a computer, cause the computer to perform operations comprising:

receiving an audio fingerprint from a listening device;

in response to determining that a portion of a stored audio fingerprint substantially matches a portion of the received audio fingerprint;

for each portion of a first set of consecutive portions of the received audio fingerprint, generating multiple corresponding match scores based on comparing the portion of the first set of consecutive portions to multiple portions of a second set of portions of the stored audio fingerprint; and

identifying a longest unordered match as a number of a highest number of consecutive portions of the first set that each have a corresponding match score that satisfies a similarity threshold; and

in response to determining that the identified longest unordered match satisfies a length criterion, detecting a match between the received audio fingerprint and the stored audio fingerprint,

wherein identifying the longest unordered match comprises scaling at least one match score when consecutive portions of the received audio fingerprint are determined to be similar to non-consecutive portions of the stored audio fingerprint.

20. The non-transitory computer-readable storage device of claim 19, wherein identifying the longest unordered match further comprises excluding at least one match score that corresponds to a portion of the stored audio fingerprint that violates a disorder threshold.