Automatic signature-based spotting, learning and extracting of commercials and other video content

• US 6,469,749 B1
• Filed: 10/13/1999
• Issued: 10/22/2002
• Est. Priority Date: 10/13/1999
• Status: Expired due to Fees

• Current Assignee: Koninklijke Philips Electronics N.V. (Koninklijke Philips N.V.)
• Original Assignee: Koninklijke Philips Electronics N.V. (Koninklijke Philips N.V.)
• Inventors: McGee, Thomas, Dimitrova, Nevenka, Agnihotri, Lalitha
• Primary Examiner(s): Miller, John
• Assistant Examiner(s): NATNAEL, PAULOS M
• Application Number:
  US09/417,288
• Time in Patent Office: 1,105 Days
• Field of Search: 348/722, 348/907, 348/700, 348/701, 348/465, 725/14, 725/22, 358/908, 358/84
• US Class Current: 348/722
• CPC Class Codes: G06F 16/7864 using domain-transform features, e.g. DCT or wavelet transform coefficients
• CPC Class Codes: G06V 20/41 Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items segmenting video sequences G06V20/49
• CPC Class Codes: H04H 60/37 for identifying segments of broadcast information, e.g. scenes or extracting programme ID
• CPC Class Codes: H04H 60/59 of video recognising characters or patterns in general G06F18/00, G06V20/00
• CPC Class Codes: H04N 21/235 Processing of additional data, e.g. scrambling of additional data or processing content descriptors
• CPC Class Codes: H04N 21/26603 for automatically generating descriptors from content, e.g. when it is not made available by its provider, using content analysis techniques
• CPC Class Codes: H04N 21/435 Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
• CPC Class Codes: H04N 21/454 Content or additional data filtering, e.g. blocking advertisements
• CPC Class Codes: H04N 21/458 Scheduling content for creating a personalised stream, e.g. by combining a locally stored advertisement with an incoming stream; Updating operations, e.g. for OS modules ; time-related management operations arrangements for replacing or switching information during the broadcast or during the distribution H04H20/10
• CPC Class Codes: H04N 21/812 involving advertisement data advertising per se G06Q30/02
• CPC Class Codes: H04N 7/002 Special television systems not provided for by H04N7/007 - H04N7/18 still pictures via a television channel H04N1/00098

Abstract

A video signal is processed to identify segments that are likely to be associated with a commercial or other particular type of video content. A signature is extracted from each of the segments so identified, and the extracted signatures are used, possibly in conjunction with additional temporal and contextual information, to determine which of the identified segments are in fact associated with the particular video content. One or more of the extracted signatures may be, e.g., a visual frame signature based at least in part on a visual characteristic of a frame of the video segment, as determined using information based on DC and motion coefficients of the frame, or DC and AC coefficients of the frame. A given extracted signature may alternatively be an audio signature based at least in part on a characteristic of an audio signal associated with a portion of the video segment. Other types of signatures can also be used. Advantageously, the invention allows the identification and extraction of particular video content to be implemented with significantly reduced amounts of memory and computational resources.

Claims

1. A method for processing a video signal, the method comprising the steps of:

identifying a plurality of segments of the video signal that are likely to be associated with a particular type of video content; and

extracting a signature from each of at least a subset of the one or more identified segments, wherein the extracted signatures are utilized to determine which of the identified segments are associated with the particular type of video content,

wherein the signature extracted from a given one of the video segments comprises a visual frame signature based at least in part on a visual characteristic of a frame of the video segment, and

wherein the visual frame signature is extracted using information based on DC and motion coefficients of the frame.

2. A method for processing a video signal, the method comprising the steps of:

identifying a plurality of segments of the video signal that are likely to be associated with a particular type of video content; and

extracting a signature from each of at least a subset of the one or more identified segments, wherein the extracted signatures are utilized to determine which of the identified segments are associated with the particular type of video content,

wherein the signature extracted from a given one of the video segments comprises a visual frame signature based at least in part on a visual characteristic of a frame of the video segment, and

wherein the visual frame signature is extracted using information based on DC and AC coefficients of the frame.

3. The method of claim 2 wherein the identifying step comprises the steps of determining if a given one of the segments includes at least one of a black frame combined with silence in its associated audio signal, a cut rate exceeding a specified value, a specified change in a color characteristic, a specified change in a characteristic of the associated audio, a specified type of text, a specified type of speech, an absence or particular type of closed caption information, and an absence of a superimposed video logo.

4. The method of claim 2 wherein the particular type of video content comprises a commercial in a broadcast program.

5. The method of claim 2 wherein the signature extracted from a given one of the video segments comprises an audio signature based at least in part on a characteristic of an audio signal associated with at least a portion of the video segment.

6. The method of claim 2 wherein the signature extracted from a given one of the video segments comprises at least one of a histogram, closed caption text, a company name, a product or service name, a telephone number, a uniform resource locator, a frame number in conjunction with text box position, color or character code information, and a frame number in conjunction with face or other object position information.

7. The method of claim 2 further including the step of utilizing temporal information relating to the identified segments to determine which of the identified segments are associated with the particular type of video content.

8. The method of claim 2 further including the step of maintaining a set of lists of signatures, the set of lists including one or more of at least one probable list, at least one candidate list and at least one found list, with each entry in a given one of the lists corresponding to a signature associated with a particular video segment.

9. The method of claim 8 wherein a given one of the signatures of a segment identified as likely to be associated with the particular video content is initially placed on a given probable list if it does not match any signature already on one or more probable lists.

10. The method of claim 9 wherein a given one of the signatures of a segment identified as likely to be associated with the particular video content is placed on a given candidate list if it matches a signature already on one or more of the probable lists.

11. The method of claim 10 wherein a given one of the signatures on the given candidate list is moved to a given found list if it matches a signature already on one or more of the candidate lists.

12. The method of claim 8 wherein a given one of the signatures is removed from at least one of the lists if it is determined that the signature has not been repeated within a specified time period.

13. The method of claim 2 further including the step of automatically extracting a signature from a particular segment of the video signal in response to a user command identifying the particular segment as associated with the particular type of video content.

14. The method of claim 2 further including the step of automatically extracting a signature from a particular segment of the video signal in response to detection of a user action relating to the particular segment.

15. The method of claim 14 wherein user action comprises fast forwarding through at least a portion of the particular segment.

16. A method for processing a video signal, the method comprising the steps of:

identifying a plurality of segments of the video signal that are likely to be associated with a particular type of video content;

extracting a signature from each of at least a subset of the one or more identified segments, wherein the extracted signatures are utilized to determine which of the identified segments are associated with the particular type of video content; and

utilizing program identification information from an electronic programming guide in conjunction with the extracted signatures to determine which of the identified segments are associated with the particular type of video content.

17. The method of claim 16 wherein the program identification information comprises at least one of a program name, a program channel, a program time slot and a program rating.