Automatic identification of repeated material in audio signals

• US 20060277047A1
• Filed: 02/08/2006
• Published: 12/07/2006
• Est. Priority Date: 02/08/2005
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Landmark Digital Services LLC
• Inventors: Karliner, Michael, Cheong Tang, Richard W., Li-Chun Wang, Avery, Briggs, Darren P., DeBusk, David L.
• Granted Patent: US 8,090,579 B2
• Time in Patent Office: Days
• US Class Current: 704/273
• CPC Class Codes: G06F 16/685 using automatically derived transcript of audio data, e.g. lyrics speech recognition G10L15/00
• CPC Class Codes: G06F 2218/08 Feature extraction
• CPC Class Codes: G10L 19/018 Audio watermarking, i.e. embedding inaudible data in the audio signal
• CPC Class Codes: G10L 25/48 specially adapted for particular use
• CPC Class Codes: G10L 25/54 for retrieval
• CPC Class Codes: H04H 60/37 for identifying segments of broadcast information, e.g. scenes or extracting programme ID
• CPC Class Codes: H04H 60/58 of audio determination or detection of speech characteristics in general G10L25/00; speech recognition in general G10L15/00

Abstract

A system and method are described for recognizing repeated audio material within at least one media stream without prior knowledge of the nature of the repeated material. The system and method are able to create a screening database from the media stream or streams. An unknown sample audio fragment is taken from the media stream and compared against the screening database to find if there are matching fragments within the media streams by determining if the unknown sample matches any samples in the screening database.

Claims

1. A method of recognizing repeated audio material within at least one media stream without prior knowledge of the nature of the repeated material, comprising:

creating a screening database from the at least one media stream;

taking an unknown sample audio fragment from the media stream;

finding matching fragments within the media stream; and

determining if the unknown sample matches any samples in the screening database.

2. The method of claim 1 further comprising:

mapping the matching fragments into a candidate group; and

evaluating the candidate group to find a best exemplar suitable for publication whereby a best exemplar matching parallel chain may constitute recognized repeated material.

3. The method of claim 1 in which the unknown sample is delimited by time.

4. The method of claim 1 in which the unknown sample is delimited by segments of already recognized material.

5. The method of claim 2 in which evaluating the candidate group is based upon best mutual match to all the other samples in the candidate group.

6. The method of claim 1 further comprising:

periodically pruning the reference database by identifying reference material that no longer passes criteria for inclusion.

7. The method of claim 2 wherein mapping the matching fragments further comprises collating parallel chains of adjacent sample fragments, based on continuity of matching of adjacent sample fragments within each chain.

8. The method of claim 7 wherein collating parallel chains includes:

selecting a first chain of unknown probe sample fragments adjacent in time, each probe sample fragment having a time offset within a corresponding media stream;

for each probe sample fragment in the first chain, finding a set of matching fragments from the screening database, each matching sample having a time offset within a corresponding media stream; and

grouping the matching sample fragments in parallel to the probe sample fragments into chains according time adjacency of the matching sample fragments within corresponding media streams, thereby forming parallel chains.

9. The method of claim 8 wherein grouping the matching sample fragments further includes:

for each probe sample and each matching sample, determining a relative time offset between the probe sample and the matching sample;

generating a histogram of relative time offsets; and

for each peak in the histogram of relative time offsets, forming a chain of adjacent matching samples from the matching samples associated with each point in the histogram peak, whereby each chain is a member of the candidate group.

10. The method of claim 9 wherein the relative time offset is determined as the difference between the time offset of the probe sample and the time offset of the matching sample.

11. The method of claim 9 wherein for each probe sample and each matching sample a speed correction factor between the probe sample and the matching sample is obtained, and the relative time offset is determined as the difference between the time offset of the probe sample and the speed-corrected time offset of the matching sample.

12. The method of claim 7 wherein each parallel chain is extended as far as possible in time to form a maximal matching parallel chain.

13. A system for recognizing repeated segments of non-recognized media content in at least one source of non-recognized media content, the system comprising:

a candidate manager receiving the non-recognized media and associating an identifier with samples of the non-recognized media;

a fingerprint generator operable to create fingerprints for non-recognized media segments; and

an media search engine connected to the candidate manager and the fingerprint generator, the media search engine able to compare fingerprints of non-recognized media against a database of previously stored non-recognized media fingerprints to find repeated segments within the non-recognized media content.

14. The system of claim 13 in which the non-recognized media is delimited by time.

15. The system of claim 13 in which the non-recognized media is delimited by segments of already recognized material.

16. The system of claim 13 wherein the non-recognized media is mapped into fragment which are matched against fragments in a screening database associated with the fingerprint generator, and the matching fragments are grouped into a candidate group.

17. The system of claim 16 wherein the candidate group is evaluated to find a best exemplar suitable for publication whereby the best exemplar may constitute recognized repeated material.

18. The system of claim 16 in which evaluating the candidate group is based upon best mutual match to all the other samples in the candidate group.

19. The system of claim 13 further comprising a reference database holding non-recognized media segments received from the at least one source.

20. The system of claim 19 wherein the reference database is periodically pruned by identifying reference material that no longer passes criteria for inclusion.