Iterative, maximally probable, batch-mode commercial detection for audiovisual content

• US 8,630,536 B2
• Filed: 07/13/2010
• Issued: 01/14/2014
• Est. Priority Date: 11/18/1999
• Status: Active Grant

• Current Assignee: Interval Licensing LLC
• Original Assignee: Interval Licensing LLC
• Inventors: Harville, Michael L.
• Primary Examiner(s): HARVEY, DAVID E
• Application Number:
  US12/835,505
• Publication Number:
  US 20100281499A1
• Time in Patent Office: 1,281 Days
• Field of Search: 386249-251
• US Class Current: 396/249
• CPC Class Codes: G11B 27/031 Electronic editing of digitised analogue information signals, e.g. audio or video signals
• CPC Class Codes: H04N 21/438 Interfacing the downstream path of the transmission network originating from a server, e.g. retrieving encoded video stream packets from an IP network
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/812 involving advertisement data advertising per se G06Q30/02
• CPC Class Codes: H04N 5/76 Television signal recording
• CPC Class Codes: H04N 5/781 on disks or drums
• CPC Class Codes: H04N 5/782 on tape
• CPC Class Codes: H04N 5/85 on discs or drums
• CPC Class Codes: H04N 5/907 using static stores, e.g. storage tubes or semiconductor memories H04N5/91 takes precedence
• CPC Class Codes: H04N 7/163 by receiver means only
• CPC Class Codes: H04N 9/87 Regeneration of colour television signals H04N9/80 takes precedence
• CPC Class Codes: Y10S 358/908 Pause control, i.e. "commercial killers"

Abstract

Identification of starting and ending times of commercial breaks and commercials within those commercial breaks found in audiovisual content is disclosed. A solution to a “batch optimization” problem is used in which commercial locations within a set of audiovisual content are detected as a group by choosing a set of commercial locations which optimizes a cost function which can include considerations of, for example, 1) one or more cues, 2) relative locations of commercials within the audiovisual content, and/or 3) probability models based on statistics obtained regarding characteristics of typical commercial and commercial breaks. Optimization can be done over the total set of commercial location decisions, rather than on a per-commercial basis. Additionally, the cost function can be iteratively evaluated and many more types of cues and combinations of cues can be used in detection of commercials.

Claims

1. A method for detecting a commercial in a set of audiovisual content spanning a duration of time, the method comprising:

selecting a first candidate time identified within the duration of time spanned by the set of audiovisual content,

wherein the first candidate time is identified based on the detection of one or more predetermined cues, and

wherein the candidate time represents a possible boundary time of a commercial;

assigning a score to the first candidate time based on one or more predetermined cues; and

analyzing the first candidate time to determine a likelihood that the first candidate time is a commercial boundary time,

wherein the analysis is performed by processing circuitry and is based at least in part on an evaluation of one or more characteristics of audiovisual content occurring after the first candidate time and associated probabilities, and

wherein the score of the first candidate time is adjusted based on the evaluated characteristics.

2. The method as in claim 1, wherein the analysis is further based, in part, on one or more characteristics of audiovisual content occurring before the first candidate time.

3. The method as in claim 1, wherein the analysis is based on one or more characteristics of audiovisual content occurring throughout the entire duration of time.

4. A method for detecting a commercial in a set of audiovisual content spanning a duration of time, the method comprising:

selecting a first candidate time identified within the duration of time spanned by the set of audiovisual content, the first candidate time being identified based on the detection of one or more predetermined cues and the candidate time representing a possible boundary time of a commercial; and

assigning a score to the first candidate time based on one or more predetermined cues;

analyzing the first candidate time to determine a likelihood that the first candidate time is a commercial boundary time,

wherein the analysis is performed by processing circuitry and is based at least in part on a relationship of the first candidate time to one or more other candidate times occurring after the first candidate time, and

wherein the score of the first candidate time is adjusted based on the relationship to the one or more other candidate times.

5. The method as in claim 4, wherein the analysis is based on a relationship of the first candidate time to all other candidate times.

6. A non-transitory computer-readable medium storing computer-readable instructions to perform a method for detecting a commercial in a set of audiovisual content spanning a duration of time, the method comprising:

detecting the presence of a cue identifying an absence of a usually present network icon in the audiovisual content;

selecting a first candidate time within the duration of time spanned by the set of audiovisual content based on the identified cue, the first candidate time representing a possible boundary time of a commercial;

assigning a score to the first candidate time based, at least in part, on the identified cue; and

analyzing the first candidate time to determine a likelihood that the first candidate time is a commercial boundary time,

wherein the analysis is performed by processing circuitry and is based at least in part on an evaluation of one or more characteristics of audiovisual content occurring after the first candidate time and associated probabilities; and

wherein the score of the first candidate time is adjusted based on the evaluated characteristics.

7. The computer-readable medium as in claim 6, further comprising detecting the presence of a cue identifying a sequence of black frames in the visual content, wherein a candidate time is selected based on an evaluation of one or more network icon and black frame cues.

8. The computer-readable medium as in claim 6, further comprising detecting the presence of a cue identifying an audio pause in the audio content, wherein a candidate time is selected based on an evaluation of one or more network icon and audio pause cues.

9. The computer-readable medium as in claim 6, further comprising detecting the presence of a cue identifying a scene cut or fade in the visual content, wherein a candidate time is selected based on an evaluation of one or more network icon and scene cut/fade cues.

10. The computer-readable medium as in claim 6, further comprising detecting the presence of a cue identifying the occurrence of specified closed-captioning formatting signals or the absence of closed-captioning, wherein a candidate time is selected based on an evaluation of one or more network icon and closed-captioning cues.

11. A method for detecting a commercial in a set of audiovisual content spanning a duration of time, the method comprising:

detecting the presence of a cue identifying a presence of music in the audio content of the audiovisual content;

selecting a first candidate time within the duration of time spanned by the set of audiovisual content based on the identified cue, the first candidate time representing a possible boundary of a commercial;

assigning a score to the first candidate time based, at least in part, on the identified cue; and

analyzing the first candidate time to determine a likelihood that the first candidate time is a commercial boundary time,

wherein the analysis is performed by processing circuitry and is based at least in part on an evaluation of one or more characteristics of audiovisual content occurring after the first candidate time and associated probabilities; and

wherein the score of the first candidate time is adjusted based on the evaluated characteristics.

12. The method as in claim 11, further comprising detecting the presence of a cue identifying a sequence of black frames in the visual content of the audiovisual content, wherein a candidate time is selected based on an evaluation of one or more music and black frame cues.

13. The method as in claim 11, further comprising detecting the presence of a cue regarding an audio pause in the audio content of the audiovisual content, wherein a candidate time is selected based on an evaluation of one or more music and audio pause cues.

14. The method as in claim 11, further comprising detecting the presence of a cue identifying a scene cut or fade in the visual content of the audiovisual content, wherein a candidate time is selected based on an evaluation of one or more music and scene cut/fade cues.

15. The method as in claim 11, further comprising detecting the presence of a cue identifying the occurrence of specified closed-captioning formatting signals or the absence of closed-captioning in the audiovisual content, wherein a candidate time is selected based on an evaluation of one or more music and closed-captioning cues.

16. The method as in claim 11, further comprising detecting the presence of a cue regarding the absence of a usually present network icon in the audiovisual content, wherein a candidate time is selected based on an evaluation of one or more music and network icon cues.

17. A method for detecting a commercial in a set of audiovisual content spanning a duration of time, the method comprising:

detecting the presence of a cue identifying a density of scene cuts or fades in the visual content of the audiovisual content;

selecting a first candidate time within the duration of time spanned by the set of audiovisual content based on the identified cue, the candidate time representing a possible boundary time of a commercial;

assigning a score to the first candidate time based, at least in part, on the identified cue; and

analyzing the first candidate time to determine a likelihood that the first candidate time is a commercial boundary time,

wherein the analysis is performed by processing circuitry and is based at least in part on an evaluation of one or more characteristics of audiovisual content occurring after the first candidate time and associated probabilities; and

wherein the score of the first candidate time is adjusted based on the evaluated characteristics.

18. The method as in claim 17, further comprising detecting the presence of a cue identifying a sequence of black frames in the visual content of the audiovisual content, wherein a candidate time is selected based on an evaluation of one or more scene cut/fade density and black frame cues.

19. The method as in claim 17, further comprising detecting the presence of a cue identifying an audio pause in the audio content of the audiovisual content, wherein a candidate time is selected based on an evaluation of one or more scene cut/fade density and audio pause cues.

20. The method as in claim 17, further comprising detecting the presence of a cue identifying a scene cut or fade in the visual content of the audiovisual content, wherein a candidate time is selected based on an evaluation of one or more scene cut/fade density and scene cut/fade cues.

21. The method as in claim 17, further comprising detecting the presence of a cue identifying the occurrence of specified closed-captioning formatting signals or the absence of closed-captioning in the audiovisual content, wherein a candidate time is selected based on an evaluation of one or more scene cut/fade density and closed-captioning cues.