SYSTEMS AND METHODS FOR REAL-TIME TELEVISION AD DETECTION USING AN AUTOMATED CONTENT RECOGNITION DATABASE

• US 20140282673A1
• Filed: 03/17/2014
• Published: 09/18/2014
• Est. Priority Date: 05/27/2010
• Status: Active Grant

• Current Assignee: Inscape Data, Inc. (VIZIO, Inc.)
• Original Assignee: Cognitive Networks, Inc. (VIZIO, Inc.)
• Inventors: Neumeier, Zeev, Reed, Brian
• Granted Patent: US 8,930,980 B2
• Time in Patent Office: Days
• US Class Current: 725/19
• CPC Class Codes: H04N 21/4316 for displaying supplemental content in a region of the screen, e.g. an advertisement in a separate window
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/442 Monitoring of processes or resources, e.g. detecting the failure of a recording device, monitoring the downstream bandwidth, the number of times a movie has been viewed, the storage space available from the internal hard disk arrangements for monitoring broadcast services or broadcast-related services H04H60/29; arrangements for identifying or recognising characteristics with a direct linkage to broadcast information H04H60/35; monitoring of user activities for profile generation for accessing a video database G06F16/739; monitoring in wireless networks H04W24/00
• CPC Class Codes: H04N 21/44204 Monitoring of content usage, e.g. the number of times a movie has been viewed, copied or the amount which has been watched monitoring of user activities for profile generation for accessing a video database G06F16/739; protecting generic digital content where the protection is independent of the precise nature of the content G06F21/10; arrangements for monitoring the use made of the broadcast services in broadcast systems H04H60/31
• CPC Class Codes: H04N 21/4532 involving end-user characteristics, e.g. viewer profile, preferences monitoring of user activities for profile generation for accessing a video database G06F16/739; user profiles in network data switching protocols H04L67/306; processing of user preferences or user profiles in wireless networks H04W8/18
• CPC Class Codes: H04N 21/462 Content or additional data management, e.g. creating a master electronic program guide from data received from the Internet and a Head-end, controlling the complexity of a video stream by scaling the resolution or bit-rate based on the client capabilities
• CPC Class Codes: H04N 21/4667 Processing of monitored end-user data, e.g. trend analysis based on the log file of viewer selections
• CPC Class Codes: H04N 21/47 End-user applications
• CPC Class Codes: H04N 21/478 Supplemental services, e.g. displaying phone caller identification, shopping application
• CPC Class Codes: H04N 21/812 involving advertisement data advertising per se G06Q30/02
• CPC Class Codes: H04N 21/8126 involving additional data, e.g. news, sports, stocks, weather forecasts

Abstract

A system and method for the automated real-time detection and processing of commercial messages, public service announcements or similar short-duration, repeated TV programming segments occurring in one or more broadcast video steams is described. A process is utilized that identifies possible commercial segments by identifying discrete video segments that have specific short durations among other attributes. Video segments that appear to have these characteristics are considered likely to be television commercials, promotions, or public service announcements and are stored in a TV Ad database which is separate from the primary television content database. Incoming video from a plurality of television programming sources is process into fingerprints and placed in a master TV content database. At the same time, said incoming programming is tested for matches in a TV ad database. If a match is found, then the presumed TV ad is removed from the master content database.

Claims

1. A method, comprising:

ingesting at least one audiovisual stream, including at least storing at least some data associated with one or more frames of the audiovisual stream into a data store of program content;

determining a candidate segment, the candidate segment including at least one or more sequential frames from the at least one ingested audiovisual stream; and

testing the determined candidate segment for at least one match in a data store of known advertisements and, if testing the candidate segment returns at least one match in the data store of known advertisements, at least removing at least some data associated with the candidate segment from the data store of program content.

2. The method of claim 1, further comprising:

testing the determined candidate segment for at least one match in the data store of program content and, if testing the candidate segment returns at least one match in the data store of program content, at least removing at least some data associated with the candidate segment from the data store of program content and storing at least some data associated with the candidate segment in the data store of known advertisements.

3. The method of claim 1, wherein determining a candidate segment, the candidate segment including at least one or more sequential frames from the at least one ingested audiovisual stream comprises:

receiving one or more indications of a frame of the ingested audiovisual data stream;

analyzing the one or more indications of a frame, including at least determining whether the one or more indications could be a starting frame of a candidate segment;

if a starting frame is determined, analyzing one or more frames received subsequent to the starting frame, including at least determining whether one of the one or more frames received subsequent to the starting frame could be an ending frame; and

if an ending frame is determined, establishing (i) the starting frame, (ii) the one or more frames received subsequent to the starting frame and previous to the ending frame, and (iii) the ending frame as the candidate segment.

4. The method of claim 3, wherein analyzing the one or more indications of a frame, including at least determining whether the one or more indications could be a starting frame of a candidate segment and analyzing one or more frames received subsequent to the starting frame, including at least determining whether a frame received subsequent to the starting frame could be an ending frame comprise:

determining whether a frame is at least one of a monochromatic frame or a frame having one or more substantially different fingerprint values from an immediately-previous frame and, if a frame is at least one of a monochromatic frame or a frame having one or more substantially different fingerprint values from an immediately-previous frame, establishing the frame as at least one of a starting frame or an ending frame.

5. The method of claim 1, wherein determining a candidate segment, the candidate segment including at least one or more sequential frames from the at least one ingested audiovisual stream comprises:

receiving one or more indications of a frame of the ingested audiovisual data stream;

comparing data associated with a contiguous sequence of ingested frames ending with the most recently received frame with data associated with one or more contiguous sequences of frames previously stored in the data store of program content; and

providing one or more indications of one or more contiguous sequences of frames previously stored in the data store of program content based at least partially on the comparing data.

6. The method of claim 5, further comprising:

for each of the one or more indicated contiguous sequences of frames, determining whether the contiguous sequence of frames is an advertisement.

7. The method of claim 6, wherein determining whether the contiguous sequence of frames is an advertisement comprises:

evaluating at least one of a length in seconds or a count in frames associated with the contiguous sequence of frames and, if the at least one of a length in seconds or a count in frames is substantially similar to a standard advertisement length, establishing the contiguous sequence of frames as a candidate segment.

8. The method of claim 7, further comprising:

adjusting at least one threshold associated with matching, the adjusted at least one threshold associated with a lower returned number of suspects;

matching the candidate segment with the one or more contiguous sequences of frames previously stored in the data store based at least partially on the adjusted at least one threshold; and

if the candidate segment matches the one or more contiguous sequences of frames previously stored in the data store based at least partially on the adjusted at least one threshold, determining starting and ending points of the candidate segment.

9. The method of claim 8, wherein adjusting at least one threshold associated with matching, the adjusted at least one threshold associated with a lower returned number of suspects comprises:

adjusting at least one radius associated with a path pursuit algorithm, the adjusted at least one radius associated with a lower returned number of suspects.

10. The method of claim 8, wherein adjusting at least one threshold associated with matching, the adjusted at least one threshold associated with a lower returned number of suspects comprises:

adjusting at least one duration related to at least one bin associated with a path pursuit algorithm, the adjusted at least one radius associated with a lower returned number of suspects.

11. The method of claim 8, wherein determining starting and ending points of the candidate segment comprises:

adjusting at least one threshold associated with matching, the adjusted at least one threshold associated with a higher returned number of suspects;

comparing one or more indications associated with each frame in the candidate segment with one or more indications associated with a corresponding frame in the one or more contiguous sequences of frames previously stored in the data store, the comparing based at least partially on the adjusted at least one threshold; and

testing for inconsistencies between each frame in the candidate segment and a corresponding frame in the one or more contiguous sequences of frames previously stored in the data store, based at least partially on the comparing.

12. The method of claim 11, wherein testing for inconsistencies between each frame in the candidate segment and a corresponding frame in the one or more contiguous sequences of frames previously stored in the data store, based at least partially on the comparing comprises:

if any frame of the candidate segment does not have a corresponding match in the one or more contiguous sequences of frames previously stored in the data store, dropping the frame of the candidate segment without a corresponding match from the one or more contiguous sequences of frames previously stored in the data store.

13. The method of claim 11, wherein testing for inconsistencies between each frame in the candidate segment and a corresponding frame in the one or more contiguous sequences of frames previously stored in the data store, based at least partially on the comparing comprises:

if any frame of the one or more contiguous sequences of frames previously stored in the data store does not have a corresponding match in the candidate segment, adding the frame of the one or more contiguous sequences of frames previously stored in the data store to the candidate segment.

14. The method of claim 11, wherein testing for inconsistencies between each frame in the candidate segment and a corresponding frame in the one or more contiguous sequences of frames previously stored in the data store, based at least partially on the comparing comprises:

testing for inconsistencies between each frame in the candidate segment and a corresponding frame in the one or more contiguous sequences of frames previously stored in the data store, based at least partially on the comparing; and

remediating any inconsistencies found via testing, the remediating operable to identify borders of the candidates.

15. The method of claim 1, wherein ingesting at least one audiovisual stream, including at least storing at least some data associated with one or more frames of the audiovisual stream into a data store of program content and testing the determined candidate segment for at least one match in a data store of known advertisements and, if testing the candidate segment returns at least one match in the data store of known advertisements, at least removing at least some data associated with the candidate segment from the data store of program content comprise:

ingesting at least two audiovisual streams, including at least storing at least some data associated with one or more frames of at least one of the at least two audiovisual streams into a data store of program content, the at least some data including at least one indication of the audiovisual stream from which the one or more frames were received, and testing the determined candidate segment for at least one match in a data store of known advertisements and, if testing the candidate segment returns at least one match in the data store of known advertisements, the at least one match related to any of the at least two audiovisual streams, at least removing at least some data associated with the candidate segment from the data store of program content.

16. The method of claim 1, wherein determining a candidate segment, the candidate segment including at least one or more sequential frames from the at least one ingested audiovisual stream comprises:

confirming a candidate segment having a low threshold match at least partially based on at least one duration of the candidate segment; and

confirming the candidate segment having a high threshold match at least partially based on a match of one or more fingerprints of the candidate segment with a known advertisement.

17. The method of claim 1, wherein determining a candidate segment, the candidate segment including at least one or more sequential frames from the at least one ingested audiovisual stream comprises:

determining a segment which may be at least one of an advertisement, a commercial, a public service announcement, a promotion, at least a portion of an ad break, at least a portion of an ad pod, or an upcoming television programming promotion.

18. The method of claim 1, wherein determining a candidate segment, the candidate segment including at least one or more sequential frames from the at least one ingested audiovisual stream comprises:

determining a candidate segment, the candidate segment including at least one or more sequential video frames from the at least one ingested audiovisual stream.

19. A computer program product, comprising:

at least one non-transitory computer-readable medium including at least;

one or more instructions for ingesting at least one audiovisual stream, including at least storing at least some data associated with one or more frames of the audiovisual stream into a data store of program content;

one or more instructions for determining a candidate segment, the candidate segment including at least one or more sequential frames from at least one ingested audiovisual stream; and

one or more instructions for testing a determined candidate segment for at least one match in a data store of known advertisements and, if testing the candidate segment returns at least one match in the data store of known advertisements, at least removing at least some data associated with the candidate segment from the data store of program content.

20. A system, comprising:

circuitry configured for ingesting at least one audiovisual stream, including at least storing at least some data associated with one or more frames of the audiovisual stream into a data store of program content;

circuitry configured for determining a candidate segment, the candidate segment including at least one or more sequential frames from at least one ingested audiovisual stream; and

circuitry configured for testing a determined candidate segment for at least one match in a data store of known advertisements and, if testing the candidate segment returns at least one match in the data store of known advertisements, at least removing at least some data associated with the candidate segment from the data store of program content.