Detecting blocks of commercial content in video data

• US 20080127244A1
• Filed: 06/30/2006
• Published: 05/29/2008
• Est. Priority Date: 06/30/2006
• Status: Active Grant

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Zhang, Tong
• Granted Patent: US 7,982,797 B2
• Time in Patent Office: Days
• US Class Current: 725/32
• CPC Class Codes: H04N 21/426 Internal components of the client ; Characteristics thereof H04N5/44 takes precedence
• CPC Class Codes: H04N 21/434 Disassembling of a multiplex stream, e.g. demultiplexing audio and video streams, extraction of additional data from a video stream; Remultiplexing of multiplex streams; Extraction or processing of SI; Disassembling of packetised elementary stream demultiplexing of data packets for data networks, e.g. RTP/UDP H04L65/00
• CPC Class Codes: H04N 21/435 Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/478 Supplemental services, e.g. displaying phone caller identification, shopping application
• CPC Class Codes: H04N 21/812 involving advertisement data advertising per se G06Q30/02
• CPC Class Codes: H04N 5/147 Scene change detection

Abstract

Systems and methods of detecting blocks of commercial content in video data are described. In one aspect, visual parameter values and audio parameter values are extracted from the video data. Visual boundary events, audio boundary events, and commercial content transition events are detected in the video data. Candidate commercial blocks of consecutive commercial content are detected in the video data from a combination of the detected visual boundary events and the detected audio boundary events. Each of the candidate commercial blocks is defined between respective candidate starting and ending boundary points. Ones of the starting and ending boundary points of the candidate commercial blocks are adjusted based on occurrence of one or more of the detected commercial content transition events within temporal windows respectively encompassing the starting and ending points.

Claims

1. A video data processing method, comprising:

extracting visual parameter values and audio parameter values from the video data;

detecting in the video data visual boundary events associated with respective ones of the extracted visual parameter values that satisfy at least one visual boundary condition, audio boundary events associated with respective ones of the extracted audio parameter values that satisfy at least one audio boundary condition, and commercial content transition events associated with respective ones of the extracted visual and audio parameter values that satisfy at least one condition that distinguishes commercial type content from program type content;

determining candidate commercial blocks of consecutive commercial content in the video data from a combination of the detected visual boundary events and the detected audio boundary events, each of the candidate commercial blocks being defined between respective candidate starting and ending boundary points; and

adjusting ones of the starting and ending boundary points of the candidate commercial blocks based on occurrence of one or more of the detected commercial content transition events within temporal windows respectively encompassing the starting and ending points.

2. The method of claim 1, wherein the detecting comprises detecting visual boundary events corresponding to blank video frames in the video data.

3. The method of claim 2, wherein the detecting of blank frames comprises detecting black video frames in the video data.

4. The method of claim 2, wherein the detecting comprises detecting audio boundary events corresponding to silence in the video data.

5. The method of claim 4, wherein the determining comprises labeling as commercial boundary points ones of the detected visual boundary events corresponding to blank video frames that coincide with respective ones of the detected audio boundary events corresponding to silence.

6. The method of claim 4, the determining comprises labeling as commercial boundary points three or more consecutive ones of the detected visual boundary events corresponding to two or more adjacent pairs of blank video frames, wherein the blank video frames in each pair are separated by a respective period approximately an integer multiple of fifteen seconds in length.

7. The method of claim 1, wherein the determining comprises identifying commercial boundary points in the video data from the combination of the detected visual boundary events and the detected audio boundary events, and clustering ones of the identified commercial boundary points into the candidate commercial blocks.

8. The method of claim 7, wherein the clustering comprises merging into respective ones of the candidate commercial blocks periods in the video data between adjacent ones of the identified commercial boundary points based on at least one temporal clustering threshold.

9. The method of claim 1, wherein the adjusting comprises adjusting ones of the starting and ending points of the candidate commercial blocks to respective ones of the detected commercial content transition events within corresponding ones of the temporal windows.

10. The method of claim 9, wherein the detecting comprises detecting commercial content transition events corresponding to still frames in the video data, and the adjusting comprises adjusting ones of the starting and ending points of the candidate commercial blocks to respective ones of the detected commercial content transition events corresponding to still frames within corresponding ones of the temporal windows.

11. The method of claim 9, wherein the detecting comprises detecting commercial content transition events corresponding to shot cut rate transitions in the video data, and the adjusting comprises adjusting ones of the starting and ending points of the candidate commercial blocks to respective ones of the detected commercial content transition events corresponding to shot cut rate transitions within corresponding ones of the temporal windows.

12. The method of claim 9, wherein the detecting comprises detecting commercial content transition events corresponding to transitions in audio content continuity in the video data, and the adjusting comprises adjusting ones of the starting and ending points of the candidate commercial blocks to respective ones of the detected commercial content transition events corresponding to transitions in audio content continuity within corresponding ones of the temporal windows.

13. The method of claim 9, wherein the detecting comprises detecting commercial content transition events corresponding to transitions in speech speed in the video data, and the adjusting comprises adjusting ones of the starting and ending points of the candidate commercial blocks to respective ones of the detected commercial content transition events corresponding to transitions in speech speed within corresponding ones of the temporal windows.

14. A video data processing system, comprising:

a detection module operable to

extract visual parameter values and audio parameter values from the video data;

detect in the video data visual boundary events associated with respective ones of the extracted visual parameter values that satisfy at least one visual boundary condition, audio boundary events associated with respective ones of the extracted audio parameter values that satisfy at least one audio boundary condition, and commercial content transition events associated with respective ones of the extracted visual and audio parameter values that satisfy at least one condition that distinguishes commercial type content from program type content; and

a commercial block identification module operable to

determine candidate commercial blocks of consecutive commercial content in the video data from a combination of the detected visual boundary events and the detected audio boundary events, each of the candidate commercial blocks being defined between respective candidate starting and ending boundary points, and

adjust ones of the starting and ending boundary points of the candidate commercial blocks based on occurrence of one or more of the detected commercial content transition events within temporal windows respectively encompassing the starting and ending points.

15. The system of claim 14, wherein the detection module is operable to detect visual boundary events corresponding to blank video frames in the video data.

16. The system of claim 15, wherein the detection is operable to detect audio boundary events corresponding to silence in the video data.

17. The system of claim 14, wherein the commercial block identification module is operable to identify commercial boundary points in the video data from the combination of the detected visual boundary events and the detected audio boundary events, and clustering ones of the identified commercial boundary points into the candidate commercial blocks.

18. The system of claim 14, wherein the commercial block identification module is operable to adjust ones of the starting and ending points of the candidate commercial blocks to respective ones of the detected commercial content transition events within corresponding ones of the temporal windows.

19. The system of claim 18, wherein:

the detection is operable to detect in the video data commercial content transition events corresponding to at least one of still frames, shot cut rate transitions, transitions in audio content continuity, and transitions in speech speed, and

the commercial block identification module is operable to adjust ones of the starting and ending points of the candidate commercial blocks to respective ones of the detected content characterizing events corresponding to one or more of still frames, high shot cut rate transitions, sudden audio content changes, and high speech speed transitions within corresponding ones of the temporal windows.

20. A video data processing system, comprising:

means for extracting visual parameter values and audio parameter values from the video data;

means for detecting in the video data visual boundary events associated with respective ones of the extracted visual parameter values that satisfy at least one visual boundary condition, audio boundary events associated with respective ones of the extracted audio parameter values that satisfy at least one audio boundary condition, and commercial content transition events associated with respective ones of the extracted visual and audio parameter values that satisfy at least one condition that distinguishes commercial type content from program type content;

means for determining candidate commercial blocks of consecutive commercial content in the video data from a combination of the detected visual boundary events and the detected audio boundary events, each of the candidate commercial blocks being defined between respective candidate starting and ending boundary points; and

means for adjusting ones of the starting and ending boundary points of the candidate commercial blocks based on occurrence of one or more of the detected commercial content transition events within temporal windows respectively encompassing the starting and ending points s encompassing one or more of the detected commercial content transition events.