Apparatus and method for locating a commercial disposed within a video data stream

• US 6,100,941 A
• Filed: 07/28/1998
• Issued: 08/08/2000
• Est. Priority Date: 07/28/1998
• Status: Expired due to Term

• Current Assignee: Philips Electronics North America Corporation (Koninklijke Philips N.V.)
• Original Assignee: US Philips Corporation (Koninklijke Philips N.V.)
• Inventors: Berkowitz, David, Ramsey, Carolyn, Leyvi, Eugene, Dimitrova, Nevenka, Elenbaas, Jan Hermanus, Mc Gee, Thomas
• Primary Examiner(s): Kostak, Victor R.
• Application Number:
  US09/123,444
• Time in Patent Office: 742 Days
• Field of Search: 348/699, 348/700, 348/722, 348/907, 358/908, 382/107, 382/112, 386/52
• US Class Current: 348/700
• CPC Class Codes: G06F 16/7328 Query by example, e.g. a complete video frame or video sequence graphical querying G06F16/7335
• CPC Class Codes: G06F 16/7864 using domain-transform features, e.g. DCT or wavelet transform coefficients
• CPC Class Codes: G11B 2220/90 Tape-like record carriers
• CPC Class Codes: G11B 27/107 of operating tapes
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence
• CPC Class Codes: H04H 20/10 Arrangements for replacing or switching information during the broadcast or the distribution
• CPC Class Codes: H04H 60/375 Commercial
• CPC Class Codes: H04H 60/59 of video recognising characters or patterns in general G06F18/00, G06V20/00
• CPC Class Codes: H04N 21/4325 by playing back content from the storage medium reproduction of recorded television signals H04N5/76; reproduction of recorded television signals H04N9/79
• CPC Class Codes: H04N 21/4334 Recording operations recording of a television signal H04N5/76; arrangements for recording or accumulating broadcast information or broadcast-related information H04H60/27
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/812 involving advertisement data advertising per se G06Q30/02
• CPC Class Codes: H04N 5/147 Scene change detection
• CPC Class Codes: H04N 5/782 on tape
• CPC Class Codes: Y10S 358/908 Pause control, i.e. "commercial killers"

Abstract

A commercial detection apparatus includes a frame grab thread which acquires frames from an input data stream to be analyzed by a cut detector thread and a filter thread. A commercial detection thread determines whether a commercial has occurred from data created by the cut detector thread and the filter thread stored in a memory. A factor used by the commercial detection thread is whether a black frame has occurred. The input data stream is divided into a plurality of regions and then a maximum and minimum value for a section of these regions is determined and compared to one another. If the maximum and minimum values are close enough, and the maximum value is below a threshold, then the frame is deemed to be black. The commercial detection thread also looks at the average cut frame distance, cut rate, changes in the average cut frame distance, the absence of a logo, a commercial signature detection, brand name detection, a series of black frames preceding a high cut rate, similar frames located within a specified period of time before a frame being analyzed and character detection. During playback, the detected commercials are either skipped or substituted with alternate content.

Claims

1. A home information appliance for detecting at least a portion of a commercial within a video data stream divided into a plurality of frames comprising:

at least one of an average cut frame distance detector, a cut frame distance trend detector, a brand name detector, a cut rate and black frame detector, a frame similarity detector, a character detector, and a static image detector, wherein

said average cut frame distance detector performs the steps of

a. calculating a quotient which includes a numerator representing the number of frames between a first frame with a cut and a previous frame with a cut and a denominator representing the number of cuts between said previous and first frames,

b. comparing said quotient to a cut frame distance threshold, and

c. identifying the occurrence of a portion of a commercial based upon said step of comparing;

said average cut frame distance trend detector determines a change in the number of cuts in a plurality of frames and detects a portion of a commercial based on said change;

said brand name detector performs the steps of

a. processing said data stream through at least one of an audio processor, a video processor, and a closed captioning processor to produce text,

b. comparing said text to a bank of product and trade names, and

c. identifying a portion of a commercial based upon said step of comparing;

said cut rate and black frame detector identifies a portion of a commercial based on whether a series of black frames has occurred within a cut rate and black frame time period before an analyzed cut, said analyzed cut occurring within a sequence of cuts, said sequence occurring within a cut rate time period;

said frame similarity detector determines whether a present frame is similar to a previous frame and detects a portion of a commercial when a frame within a time period is similar to the present frame;

said character detector performs the steps of

a. finding letters within an analyzed frame of said frames,

b. creating a box around said letters,

c. determining dimensions of said box,

d. determining a location of said box within said analyzed frame,

e. analyzing said dimensions and said location of said analyzed frame with subsequent analyzed frames, and

f. detecting a portion of a commercial based upon said step of analyzing; and

said static image detector detects a portion of a commercial when a number of static frames has occurred within a time period.

2. The home information appliance as claimed in claim 1 further comprising:

a frame grabber for acquiring said frames;

a first detector for identifying within said data stream at least one of cuts, similar frames located within a first period of time, and black frames;

a second detector for identifying within said data stream at least one of frames which are similar within a second period of time and frames which are substantially the same color; and

a third detector thread for identifying said commercials using information produced by said first and second detectors.

3. The home information appliance as claimed in claim 2 wherein said second period of time is greater than said first period of time.

4. The home information appliance as claimed in claim 1 wherein at least one of the following is predetermined:

said number of cuts in said average cut frame distance detector, said cut rate and black frame time period, said series, said sequence, said cut rate time period, said time period in said frame similarity detector, and said time period and said number of static frames in said static image detector.

5. The home information appliance as claimed in claim 1 wherein each of said detectors uses DCT coefficients of said data stream.

6. The home information appliance as in claim 1 further comprising a storage device for storing said data stream and for substituting alternate content in place of a detected commercial.

7. The home information appliance as claimed in claim 1 further comprising a means for skipping a detected commercial.

8. A method for identifying commercials within a video data stream divided into a plurality of frames through detection of at least one of average cut frame distance, average cut frame distance trend, brand name, black frame, cut and black frame, frame similarity, character detection, and static image, wherein:

average cut frame distance detection includes;

a. calculating a quotient which includes a numerator representing the number of frames between a first frame with a cut and a previous frame with a cut and a denominator representing the number of cuts between said previous and first frames,

b. comparing said quotient to a cut frame distance threshold, and

c. identifying the occurrence of a portion of a commercial based upon said step of comparing;

average cut frame distance trend detection is a determination of a change in the number of cuts in a plurality of frames, detection of a portion of a commercial being based on said change;

brand name detection includes the steps of

a. processing said data stream through at least one of an audio processor, a video processor, and a closed captioning processor to produce text,

b. comparing said text to a bank of product and trade names, and

c. identifying a portion of a commercial based upon said step of comparing;

black frame detection includes the steps of

a. dividing an analyzed frame of said frames into a plurality of regions;

b. calculating an average maximum luminance value for said regions;

c. calculating an average minimum luminance value for said regions;

d. comparing said average maximum luminance value and said average minimum luminance value with a black frame threshold, and

e. identifying the occurrence of beginning or an ending of a commercial based on said step of comparing;

cut and black frame detection identifies a portion of a commercial based on whether a series of black frames has occurred within a cut and black frame time period before an analyzed cut, said analyzed cut occurring within a sequence of cuts, said sequence occurring within a cut rate time period;

frame similarity detection is a determination of whether a present frame is similar to a previous frame, a portion of a commercial being when a frame within a time period is similar to the present frame;

character detection includes the steps of

a. finding letters within an analyzed frame of said frames,

b. creating a box around said letters,

c. determining dimensions of said box,

d. determining a location of said box within said analyzed frame,

e. analyzing said dimensions and said location of said analyzed frame with subsequent analyzed frames, and

f. detecting a portion of a commercial based upon said step of analyzing; and

static image detection identifies a portion of a commercial when a number of static frames has occurred within a time period.

9. The method as in claim 8 where in said black frame detection:

said step of dividing includes the step of dividing said analyzed frame into four quadrants; and

said step of comparing includes comparing said average minimum and average maximum luminance values for one quadrant at a time.

10. The method as in claim 8 where said black frame detection further comprises the steps of:

determining the smallest average maximum luminance value; and

comparing said maximum luminance value with said smallest average maximum luminance value.

11. The method as in claim 8 where said detectors use DCT coefficients of said data stream.

12. The method as in claim 8 further comprising a storage device for storing said data stream and for substituting alternate content in place of said commercial.

13. The method as claimed in claim 8 further comprising a means for skipping a detected commercial.

14. The method as claimed in claim 8 further comprising:

a frame grabber for acquiring said frames;

a first detector for identifying within said data stream at least one of cuts, similar frames located within a first period of time, and black frames;

a second detector for identifying within said data stream at least one of frames which are similar within a second period of time and frames which are substantially the same color; and

a third detector thread for identifying said commercials using information produced by said first and second detectors.

15. The method as claimed in claim 14 wherein said second period of time is greater than said first period of time.

16. The method as claimed in claim 8 wherein at least one of the following is predetermined:

said number of cuts in said average cut frame distance detector, said cut rate and black frame time period, said series, said sequence, said cut rate time period, said time period in said frame similarity detector, and said time period and said number of static frames in said static image detector.

17. The method as in claim 8 where in said black frame detection said steps of comparing are performed on every other one of said regions.

18. The method as in claim 8 where in said black frame detection said step of dividing comprises excluding outermost portions of said analyzed frame.

19. The method as in claim 8 where in said black frame detection said steps of calculating, comparing, and identifying are performed on one of said regions at a time.