System for monitoring audio content in a video broadcast

• US 20020082731A1
• Filed: 06/29/2001
• Published: 06/27/2002
• Est. Priority Date: 11/03/2000
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Pitman, Michael C., Fitch, Blake G., Germain, Robert S., Abrams, Steven
• Granted Patent: US 7,085,613 B2
• Time in Patent Office: Days
• US Class Current: 700/94
• CPC Class Codes: G06Q 30/0283 Price estimation or determination
• CPC Class Codes: G11B 20/00086 Circuits for prevention of unauthorised reproduction or copying, e.g. piracy indicating unauthorised use of record carriers in general G11B23/28; scrambling for television signal recording H04N5/913; network architectures or network protocols for network security H04L63/00; cryptographic mechanisms or cryptographic arrangements for secret or secure communication H04L9/00
• CPC Class Codes: G11B 20/10527 Audio or video recording; Data buffering arrangements G11B20/12 - G11B20/18 take precedence
• CPC Class Codes: G11B 2020/10546 specifically adapted for audio data
• CPC Class Codes: H04H 20/14 for monitoring programmes
• CPC Class Codes: H04N 21/4394 involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/4627 Rights management associated to the content security in data switching network management H04L41/28; security management or policies for network security H04L63/20; access security in wireless networks H04W12/08

Abstract

A method is provided for monitoring audio content in a video broadcast. According to the method, an audio datastream from the video broadcast is received, and audio identifying information is generated for audio content from the audio datastream. It is determined whether the audio identifying information generated for the received audio content matches audio identifying information in an audio content database. In one preferred embodiment, the audio identifying information is an audio feature signature that is based on audio content. Also provided is a system for monitoring audio content in a video broadcast.

Claims

1. A method for monitoring audio content in a video broadcast, said method comprising the steps of:

receiving an audio datastream from the video broadcast;

generating audio identifying information for audio content from the audio datastream; and

determining whether the audio identifying information generated for the received audio content matches audio identifying information in an audio content database.

2. The method according to claim 1, wherein the audio identifying information is an audio feature signature that is based on audio content.

3. The method according to claim 2, wherein the determining step includes the sub-step of comparing the audio feature signature generated for the received audio content with the audio feature signatures stored in the audio content database.

4. The method according to claim 1, further comprising the steps of:

generating audio identifying information for predetermined audio content; and

storing the audio identifying information for the predetermined audio content in the audio content database.

6. The method according to claim 1, further comprising the step of:

if the audio identifying information generated for the received audio content matches audio identifying information in the audio content database, recording information on a match between the received audio content and the audio content database.

7. The method according to claim 6, further comprising the steps of:

compiling cue sheet entries from the recorded match information, the cue sheet entries including identification of at least one piece of audio content in the video broadcast, a start time of each piece of audio content, and a duration of each piece of audio content; and

charging royalties based on the cue sheet entries.

8. The method according to claim 6, further comprising the steps of:

compiling cue sheet entries from the recorded match information, the cue sheet entries including identification of each piece of audio content in the video broadcast, a start time of each piece of audio content, and a duration of each piece of audio content; and

paying royalties based on the cue sheet entries.

9. The method according to claim 6, further comprising the step of:

compiling a cue sheet containing cue sheet entries from the recorded match information, the cue sheet entries including identification of at least one piece of audio content in the video broadcast, a start time of each piece of audio content, and a duration of each piece of audio content; and

charging a fee for the cue sheet.

10. The method according to claim 1, wherein the generating step includes the sub-steps of:

obtaining an audio signal characterized by a time dependent power spectrum;

analyzing the spectrum to obtain a plurality of time dependent frequency components;

detecting a plurality of events in the plurality of time dependent frequency components; and

producing the audio identifying information for the audio content from the audio datastream based on the plurality of events.

11. The method according to claim 10, wherein the sub-step of analyzing the spectrum includes:

sampling the audio signal to obtain a plurality of audio signal samples;

taking a plurality of subsets from the plurality of audio signal samples; and

performing a Fourier transform on each of the plurality of subsets to obtain a set of Fourier frequency components.

12. The method according to claim 10, wherein the sub-step of detecting a plurality of events includes:

taking a first running average with a first averaging period of each of a first subset of the plurality of time dependent frequency components to obtain a first sequence of average powers at a set of successive times for each of the plurality of time dependent frequency components;

taking a second running average with a second averaging period that is different from the first averaging period of each of the subset of the plurality of time dependent frequency components to obtain a second sequence of average powers at the set of successive times for each of the plurality of time dependent frequency components; and

identifying a plurality of average crossing events at a plurality of event times at which the first running average crosses the second running average.

13. The method according to claim 1, wherein the generating step includes the sub-steps of:

performing a Fourier transformation of the audio signals of the portion into a time series of audio power dissipated over a first plurality of frequencies;

grouping the frequencies into a smaller second plurality of bands that each include a range of neighboring frequencies;

detecting power dissipation events in each of the bands; and

grouping together the power dissipation events from mutually adjacent bands at a selected moment so as to form the identifying feature.

15. The computer-readable medium according to claim 14, wherein the audio identifying information is an audio feature signature that is based on audio content.

16. The computer-readable medium according to claim 15, wherein the determining step includes the sub-step of comparing the audio feature signature generated for the received audio content with the audio feature signatures stored in the audio content database.

17. The computer-readable medium according to claim 14, wherein said program further contains instructions for performing the steps of:

generating audio identifying information for predetermined audio content; and

storing the audio identifying information for the predetermined audio content in the audio content database.

18. The computer-readable medium according to claim 14, wherein said program further contains instructions for performing the step of:

if the audio identifying information generated for the received audio content matches audio identifying information in the audio content database, determining a duration of the audio content in the video broadcast and charging a royalty based on the duration.

19. The computer-readable medium according to claim 14, wherein said program further contains instructions for performing the steps of:

if the audio identifying information generated for the received audio content matches audio identifying information in the audio content database, recording information on a match between the received audio content and the audio content database;

compiling cue sheet entries from the recorded match information, the cue sheet entries including identification of at least one piece of audio content in the video broadcast, a start time of each piece of audio content, and a duration of each piece of audio content; and

charging royalties based on the cue sheet entries.

20. The computer-readable medium according to claim 14, wherein said program further contains instructions for performing the steps of:

if the audio identifying information generated for the received audio content matches audio identifying information in the audio content database, recording information on a match between the received audio content and the audio content database;

compiling cue sheet entries from the recorded match information, the cue sheet entries including identification of each piece of audio content in the video broadcast, a start time of each piece of audio content, and a duration of each piece of audio content; and

paying royalties based on the cue sheet entries.

21. The computer-readable medium according to claim 14, wherein the generating step includes the sub-steps of:

obtaining an audio signal characterized by a time dependent power spectrum;

analyzing the spectrum to obtain a plurality of time dependent frequency components;

detecting a plurality of events in the plurality of time dependent frequency components; and

producing the audio identifying information for the audio content from the audio datastream based on the plurality of events.

22. The computer-readable medium according to claim 21, wherein the sub-step of analyzing the spectrum includes:

sampling the audio signal to obtain a plurality of audio signal samples;

taking a plurality of subsets from the plurality of audio signal samples; and

performing a Fourier transform on each of the plurality of subsets to obtain a set of Fourier frequency components.

23. The computer-readable medium according to claim 21, wherein the sub-step of detecting a plurality of events includes:

taking a first running average with a first averaging period of each of a first subset of the plurality of time dependent frequency components to obtain a first sequence of average powers at a set of successive times for each of the plurality of time dependent frequency components;

taking a second running average with a second averaging period that is different from the first averaging period of each of the subset of the plurality of time dependent frequency components to obtain a second sequence of average powers at the set of successive times for each of the plurality of time dependent frequency components; and

identifying a plurality of average crossing events at a plurality of event times at which the first running average crosses the second running average.

24. The computer-readable medium according to claim 14, wherein the generating step includes the sub-steps of:

performing a Fourier transformation of the audio signals of the portion into a time series of audio power dissipated over a first plurality of frequencies;

grouping the frequencies into a smaller second plurality of bands that each include a range of neighboring frequencies;

detecting power dissipation events in each of the bands; and

grouping together the power dissipation events from mutually adjacent bands at a selected moment so as to form the identifying feature.

26. The system according to claim 25, wherein the audio identifying information is an audio feature signature that is based on audio content.

27. The system according to claim 25, wherein the audio content database stores audio identifying information for predetermined audio content.

28. The system according to claim 25, further comprising:

an invoicer for determining a duration of the audio content in the video broadcast and charging a royalty based on the duration, if the audio identifying information generated for the received audio content matches audio identifying information in the audio content database.

29. The system according to claim 25, further comprising:

an information collector for recording information on a match between the received audio content and the audio content database, if the audio identifying information generated for the received audio content matches audio identifying information in the audio content database;

a cue sheet generator for compiling cue sheet entries from the recorded match information, the cue sheet entries including identification of at least one piece of audio content in the video broadcast, a start time of each piece of audio content, and a duration of each piece of audio content; and

an invoicer for charging royalties based on the cue sheet entries.

30. The system according to claim 25, further comprising:

an information collector for recording information on a match between the received audio content and the audio content database, if the audio identifying information generated for the received audio content matches audio identifying information in the audio content database;

a cue sheet generator for compiling cue sheet entries from the recorded match information, the cue sheet entries including identification of each piece of audio content in the video broadcast, a start time of each piece of audio content, and a duration of each piece of audio content; and

a royalty calculator for calculating royalties to be paid based on the cue sheet entries.

5. A method for charging a royalty for usage of copyrighted audio content in a video broadcast, said method comprising the steps of:

receiving an audio datastream from the video broadcast;

generating audio identifying information for audio content from the audio datastream;

determining whether the audio identifying information generated for the received audio content matches audio identifying information in a copyrighted audio content database; and

if the audio identifying information generated for the received audio content matches audio identifying information in the copyrighted audio content database, determining a duration of the audio content in the video broadcast and charging a royalty based on the duration.

14. A computer-readable medium encoded with a program for monitoring audio content in a video broadcast, said program containing instructions for performing the steps of:

receiving an audio datastream from the video broadcast;

generating audio identifying information for audio content from the audio datastream; and

determining whether the audio identifying information generated for the received audio content matches audio identifying information in an audio content database.

25. A system for monitoring audio content in a video broadcast, said system comprising:

a receiver for receiving an audio datastream from the video broadcast;

an identifying information generator for generating audio identifying information for audio content from the audio datastream; and

a match detector for determining whether the audio identifying information generated for the received audio content matches audio identifying information in an audio content database.