Video content detection method and system leveraging data-compression constructs

• US 20020186768A1
• Filed: 05/14/2001
• Published: 12/12/2002
• Est. Priority Date: 05/14/2001
• Status: Active Grant

• Current Assignee: Koninklijke Philips Electronics N.V. (Koninklijke Philips N.V.)
• Original Assignee: Koninklijke Philips Electronics N.V. (Koninklijke Philips N.V.)
• Inventors: McGee, Thomas, Dimitrova, Nevenka, Mekenkamp, Gerhardus, Salomons, Edwin
• Granted Patent: US 6,714,594 B2
• Time in Patent Office: Days
• US Class Current: 375/240.12
• CPC Class Codes: H04N 19/137 Motion inside a coding unit, e.g. average field, frame or block difference
• CPC Class Codes: H04N 19/142 Detection of scene cut or scene change
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 21/235 Processing of additional data, e.g. scrambling of additional data or processing content descriptors
• CPC Class Codes: H04N 21/2353 specifically adapted to content descriptors, e.g. coding, compressing or processing of metadata
• CPC Class Codes: H04N 21/435 Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/812 involving advertisement data advertising per se G06Q30/02

Abstract

The process of compressing video requires the calculation of a variety data that are used in the process of compression. The invention exploits some or all of these data for purposes of content detection. For example, these data may be leveraged for purposes of commercial detection. The luminance, motion vector field, residual values, quantizer, bit rate, etc. may all be used either directly or in combination, as signatures of content. A process for content detection may employ one or more features as indicators of the start and/or end of a sequence containing a particular type of content and other features as verifiers of the type of content bounded by these start/end indicators. The features may be combined and/or refined to produce higher-level feature data with good computational economy and content-classification utility.

Claims

1. A content editor, comprising:

a video compression encoder that generates first and second feature data from a video sequence as part of a compression process resulting in a compressed version of video data;

said first and second feature data being separate from said compressed version of video data;

an analysis engine programmed to receive said first and second feature data and calculate at least a third feature datum from at least one of said first and second feature data;

a playback selector programmed to edit said compressed version of video data responsively to said at least a third feature datum.

2. A content editor as in claim 1, wherein said playback selector is programmed to edit said compressed version of video data responsively to at least one of said first and second data.

3. A content editor as in claim 1, wherein said third data includes an identifier of a presence of a sequence of unicolor frames.

4. A content editor as in claim 1, wherein said third data includes an identifier of a transition between letterbox format and non-letterbox format.

5. A content editor as in claim 1, wherein said third data includes an identifier of a transition between interlaced and progressive video.

6. A content editor as in claim 1, wherein said third data includes an identifier of a frequency of scene cuts.

7. A content editor as in claim 1, wherein said third data includes a color histogram representing a frame.

8. A content editor as in claim 1, wherein said first and second data includes audio features of said video sequence.

9. A content editor as in claim 1, wherein said playback selector is programmed to edit said compressed version of video data responsively to at least one of said first, second, and third data includes at least one of an average of motion vectors, a current bit rate, a variation of luminance within a frame, variation of color within a frame, a total luminance of a frame, a total color of a frame, change in luminance between frames, a mean absolute difference, and a quantizer scale.

10. A video content detector, comprising:

a video compression encoder capable of receiving uncompressed video data and generating compressed video data;

said analysis engine being connected to receive first data from the video compression encoder, said first data being separate from said compressed video data;

said first data being generated as a result of a compression process;

said analysis engine being programmed to generate an identifier of a beginning of a type of content in said compressed video responsively to said first data.

11. A content detector as in claim 10, wherein said first data includes at least one of a quantizer scale, motion vector data, bit rate data, a variation of luminance within a frame, variation of color within a frame, a total luminance of a frame, a total color of a frame, change in luminance between frames, a mean absolute difference, and a quantizer scale.

12. A content detector as in claim 10, wherein said analysis engine is programmed to calculate a derivative feature from at least one of said first data and to generate said identifier responsively also to said derivative data.

13. A content detector as in claim 10, wherein said analysis engine is programmed to identify, responsively to said first data, the presence or absence of a letterbox in said uncompressed video data and to generate an identifier of a location in a sequence of said compressed video data coinciding with said presence or absence.

14. A content detector as in claim 10, wherein said analysis engine is programmed to identify, responsively to said first data, the presence of interlaced or progressive video format in said uncompressed video data and to generate an identifier of a location in a sequence of said compressed video data coinciding with said interlaced or progressive video format.

15. A content detector as in claim 10, wherein said analysis engine is programmed to identify, responsively to said first data, the presence of unicolor frames in said uncompressed video data and to generate an identifier of a location in a sequence of said compressed video data coinciding with said unicolor frames.

16. A content detector as in claim 10, wherein said analysis engine is programmed to identify, responsively to said first data, an indicator or a frequency of scene cuts in said uncompressed video data and to generate an identifier of a location in a sequence of said compressed video data coinciding with said frequency of scene cuts.

18. A method as in claim 17, wherein said step of verifying includes calculating at least one of a scene cut rate, a unicolor frame sequence, a letterbox border of a video frame, and whether the video format is progressive or interlaced.

20. A method as in claim 19, wherein said step of classifying includes programming a classification engine based on examples of said predefined content.

21. A method as in claim 19, wherein said step of classifying includes training a classifier and using said classifier to classify said predefined content.

22. A method as in claim 21, wherein said classifier includes at least one of a Bayesian classifier, a neural network, and a hidden Markov model classifier.

17. A method for detecting commercials in a compressed video stream, comprising the steps of:

compressing video data and generating compressed video data and first data as a byproduct of said step of compressing;

identifying first events in said first data indicating a potential start of a commercial sequence;

verifying that a content of video following said potential start is characteristic of a commercial sequence responsively to said first data;

indicating a presence of a commercial responsively to results of said steps of identifying and verifying.

19. A method for detecting content in video data, comprising the steps of:

compressing video data and generating compressed video data and compression feature data as a byproduct of said step of compressing;

classifying content portions of said video data based on said compression feature data in combination with non-compression feature data;

indicating content identified in said step of classifying.