Automatic content recognition and verification in a broadcast chain

• US 10,701,438 B2
• Filed: 12/31/2016
• Issued: 06/30/2020
• Est. Priority Date: 12/31/2016
• Status: Active Grant

• Current Assignee: Turner Broadcasting System Incorporated (Warner Bros. Discovery, Inc.)
• Original Assignee: Turner Broadcasting System Incorporated (Warner Bros. Discovery, Inc.)
• Inventors: Stigall, Gregory McClain, Sinha, Nishith Kumar
• Primary Examiner(s): Chokshi, Pinkal R
• Application Number:
  US15/396,596
• Publication Number:
  US 20180192119A1
• Time in Patent Office: 1,277 Days
• CPC Class Codes: G06T 7/00 Image analysis
• CPC Class Codes: H04N 17/004 for digital television systems
• CPC Class Codes: H04N 21/222 Secondary servers, e.g. proxy server, cable television Head-end provisioning of proxy services in data packet switching networks H04L67/56
• CPC Class Codes: H04N 21/23418 involving operations for analysing video streams, e.g. detecting features or characteristics television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/4316 for displaying supplemental content in a region of the screen, e.g. an advertisement in a separate window
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/4886 for displaying a ticker, e.g. scrolling banner for news, stock exchange, weather data
• CPC Class Codes: H04N 21/8146 involving graphical data, e.g. 3D object, 2D graphics

Abstract

A graphics verification system that handles content of a channel in a broadcast chain, partitions an on-screen view of the channel into a plurality of sections based on an overlay of a plurality of graphics on the on-screen view of the channel. The content may comprise media content and may include video, audio and/or ancillary data that is associated with any portion of a particular content. A placement of each of the overlaid plurality of graphics in the plurality of sections is determined with respect to associated content on the on-screen view of the channel. The placement is determined based on a concurrent look-up of the overlaid plurality of graphics in the plurality of sections and the associated content. The concurrent look-up is based on fingerprints or watermarks of the plurality of graphics and the associated content. One or more error parameters are detected based on the determined placement. The on-screen view of the channel is calibrated prior to multicasting of the channel, based on the one or more detected error parameters.

Claims

1. A system, comprising:

one or more processors in a graphics verification system that manages content of a channel in a broadcast chain, the one or more processors configured to;

partition an on-screen view of the channel into a plurality of sections based on an overlay of a plurality of graphics on the on-screen view of the channel;

determine a placement of each of the overlaid plurality of graphics in the plurality of sections with respect to associated content on the on-screen view of the channel,

wherein the placement of each of the overlaid plurality of graphics in the plurality of sections is determined based on a time-based parameter of a programming schedule and a region-based parameter of the programming schedule,

wherein the programming schedule corresponds to the plurality of graphics and the associated content on the on-screen view;

detect one or more error parameters based on the determined placement of each of the overlaid plurality of graphics in the plurality of sections with respect to the associated content,

wherein the one or more error parameters correspond to errors related to positions of the overlaid plurality of graphics on the on-screen view;

remove a first overlaid graphic of the overlaid plurality of graphics on the on-screen view of the channel based on a first graphics identifier (ID) of the first overlaid graphic that is in the programming schedule at a time code for a subsequent content item; and

reinstate the first overlaid graphic on the on-screen view of the channel after a number of video frames based on the detected one or more error parameters,

wherein the number of video frames is associated with the time code for the subsequent content item.

2. The system according to claim 1, wherein the plurality of graphics comprises at least one of a channel logo, a sign, a symbol, a text placeholder, full screen graphics, partial screen graphics, text crawl, a rating key, or a ticker.

3. The system according to claim 1, wherein the placement of each of the plurality of graphics in the plurality of sections with respect to the associated content is determined based on a concurrent look-up of each of the plurality of graphics in the plurality of sections and the associated content to identify correct placement of the plurality of graphics intended for specific content on the on-screen view of the channel.

4. The system according to claim 3, wherein the one or more processors are further configured to execute the concurrent look-up based on a graphics ID of each of the plurality of graphics in the plurality of sections and a content ID of the associated content of each of the plurality of graphics, and

wherein the graphics ID corresponds to fingerprints of each of the plurality of graphics, and wherein the content ID corresponds to fingerprints of the associated content.

5. The system according to claim 4, wherein the one or more processors are further configured to generate, by a graphics engine, graphics fingerprints, which correspond to the fingerprints of the plurality of graphics,

wherein the graphics fingerprints are generated based on at least one or more visual features in the plurality of graphics,

wherein the graphics fingerprints are generated for a plurality of coordinates that indicate the positions of the overlaid plurality of graphics on the on-screen view, and

wherein the generated graphics fingerprints are stored in a graphics fingerprint database.

6. The system according to claim 5, wherein the one or more processors are further configured to compare the fingerprints of the overlaid plurality of graphics detected on the on-screen view of the channel with the graphics fingerprints stored in the graphics fingerprint database, and

wherein the comparison is executed to detect the one or more error parameters of the overlaid plurality of graphics.

7. The system according to claim 6, wherein the overlaid plurality of graphics on the on-screen view of the channel is automatically corrected in the broadcast chain based on the detected one or more error parameters.

8. The system according to claim 1, wherein the one or more processors are further configured to generate, by an ingest engine, raster fingerprints for the associated content,

wherein the raster fingerprints for the associated content are generated based on one or more visual features of the associated content, and

wherein the generated raster fingerprints are stored in a content fingerprint database.

9. The system according to claim 8, wherein the one or more processors are further configured to compare fingerprints of the associated content on which the plurality of graphics are overlaid on the on-screen view of the channel with the raster fingerprints stored in the content fingerprint database, and

wherein the comparison is executed to detect the one or more error parameters related to the associated content.

10. The system according to claim 9, wherein the associated content that corresponds to a specific content item associated with the overlaid plurality of graphics for a specific time instant on the on-screen view of the channel is automatically corrected based on the detected one or more error parameters.

11. The system according to claim 10, wherein the one or more processors are further configured to verify each of the plurality of graphics in the plurality of sections and the associated content based on the time-based parameter of the programming schedule and the region-based parameter of the programming schedule, and

wherein the programming schedule is determined by a broadcast automation system.

12. The system according to claim 11, wherein the time-based parameter of the programming schedule is indicative of a timing identifier at which the plurality of graphics and the associated content are required to be overlaid on the on-screen view of the channel during multicasting of the channel.

13. The system according to claim 11, wherein the region-based parameter of the programming schedule is indicative of specific regions on the on-screen view of the channel where the plurality of graphics and the associated content are required to be overlaid on the on-screen view of the channel during multicasting of the channel, and

wherein the specific regions on the on-screen view of the channel correspond to the plurality of sections.

14. The system according to claim 10, wherein the one or more processors are further configured to utilize a plurality of automatic content recognition (ACR)-based inline-monitoring devices (IMDs) placed at a plurality of locations in the broadcast chain, to concurrently detect fingerprints of each of the plurality of graphics in the plurality of sections and the fingerprints of the associated content on which the plurality of graphics are overlaid.

31. The system according to claim 1, wherein the first content item currently associated with the overlaid plurality of graphics on the on-screen view of the channel is replaced with a second content item retrieved from a content database based on a comparison of fingerprints of the overlaid plurality of graphics detected on the on-screen view of the channel with graphics fingerprints stored in a graphics fingerprint database.

15. A method, comprising:

in a graphics verification system that manages content of a channel in a broadcast chain;

partitioning an on-screen view of the channel into a plurality of sections based on an overlay of a plurality of graphics on the on-screen view of the channel;

determining a placement of each of the overlaid plurality of graphics in the plurality of sections with respect to associated content on the on-screen view of the channel,

wherein the placement of each of the overlaid plurality of graphics in the plurality of sections is determined based on a time-based parameter of a programming schedule and a region-based parameter of the programming schedule,

wherein the programming schedule corresponds to the plurality of graphics and the associated content on the on-screen view;

detecting one or more error parameters based on the determined placement of each of the overlaid plurality of graphics in the plurality of sections with respect to the associated content, wherein the one or more error parameters correspond to errors related to positions of the overlaid plurality of graphics on the on-screen view;

removing a first overlaid graphic of the overlaid plurality of graphics on the on-screen view of the channel based on a first graphics identifier (ID) of the first overlaid graphic that is in the programming schedule at a time code for a subsequent content item; and

reinstating the first overlaid graphic on the on-screen view of the channel after a number of video frames based on the detected one or more error parameters,

wherein the number of video frames is associated with the time code for the subsequent content item.

16. The method according to claim 15, wherein the plurality of graphics comprises at least one of a channel logo, a sign, a symbol, a text placeholder, full screen graphics, partial screen graphics, text crawl, a rating key, or a ticker.

17. The method according to claim 15, wherein the placement of each of the plurality of graphics in the plurality of sections with respect to the associated content is determined based on a concurrent look-up of each of the plurality of graphics in the plurality of sections and the associated content to identify correct placement of the plurality of graphics intended for specific content on the on-screen view of the channel.

18. The method according to claim 17, further comprising executing the concurrent look-up based on a graphics ID of each of the plurality of graphics in the plurality of sections and a content ID of the associated content of each of the plurality of graphics, and

wherein the graphics ID corresponds to fingerprints of each of the plurality of graphics, and wherein the content ID corresponds to fingerprints of the associated content.

19. The method according to claim 18, further comprising generating, by a graphics engine, graphics fingerprints, which correspond to the fingerprints of the plurality of graphics,

wherein the graphics fingerprints are generated based on one or more visual features in the plurality of graphics,

wherein the graphics fingerprints are generated for a plurality of coordinates that indicate the positions of the overlaid plurality of graphics on the on-screen view, and

wherein the generated graphics fingerprints are stored in a graphics fingerprint database.

20. The method according to claim 19, further comprising comparing the fingerprints of the overlaid plurality of graphics detected on the on-screen view of the channel with the graphics fingerprints stored in the graphics fingerprint database, and

wherein the comparison is executed to detect the one or more error parameters of the overlaid plurality of graphics.

21. The method according to claim 20, wherein the overlaid plurality of graphics on the on-screen view of the channel is automatically corrected in the broadcast chain based on the detected one or more error parameters.

22. The method according to claim 15, further comprising generating, by an ingest engine, raster fingerprints for the associated content,

wherein the raster fingerprints for the associated content are generated based on one or more visual features of the associated content, and wherein the generated raster fingerprints are stored in a content fingerprint database.

23. The method according to claim 22, further comprising comparing fingerprints of the associated content on which the plurality of graphics are overlaid on the on-screen view of the channel with the raster fingerprints stored in the content fingerprint database, and

wherein the comparison is executed to detect the one or more error parameters related to the associated content.

24. The method according to claim 23, wherein the associated content that corresponds to a specific content item associated with the overlaid plurality of graphics for a specific time instant on the on-screen view of the channel is automatically corrected based on the detected one or more error parameters.

25. The method according to claim 24, further comprising verifying each of the plurality of graphics in the plurality of sections and the associated content on the on-screen view based on the time-based parameter and the region-based parameter of the programming schedule, and

wherein the programming schedule is determined by a broadcast automation system.

26. The method according to claim 25, wherein the time-based parameter of the programming schedule is indicative of a timing identifier at which the plurality of graphics and the associated content are required to be overlaid on the on-screen view of the channel during multicasting of the channel.

27. The method according to claim 25, wherein the region-based parameter of the programming schedule is indicative of specific regions on the on-screen view of the channel where the plurality of graphics and the associated content are required to be overlaid on the on-screen view of the channel during multicasting of the channel, and wherein the specific regions on the on-screen view of the channel correspond to at least the plurality of sections.

28. The method according to claim 24, further comprising utilizing a plurality of automatic content recognition (ACR)-based inline-monitoring devices (IMDs) placed at a plurality of locations in the broadcast chain, to concurrently detect fingerprints of each of the plurality of graphics in the plurality of sections and the fingerprints of the associated content on which the plurality of graphics are overlaid.

29. A non-transitory computer-readable medium having stored thereon computer-executable instructions which, when executed by one or more processors, cause the one or more processors to execute operations, the operations comprising:

in a graphics verification system that manages verification of content of a channel in a broadcast chain;

partitioning an on-screen view of the channel into a plurality of sections based on an overlay of a plurality of graphics on the on-screen view of the channel;

determining a placement of each of the overlaid plurality of graphics in the plurality of sections with respect to associated content on the on-screen view of the channel,

wherein the placement of each of the overlaid plurality of graphics in the plurality of sections is determined based on a time-based parameter of a programming schedule and a region-based parameter of the programming schedule,

wherein the programming schedule corresponds to the plurality of graphics and the associated content on the on-screen view;

detecting one or more error parameters based on the determined placement of each of the overlaid plurality of graphics in the plurality of sections with respect to the associated content, wherein the one or more error parameters correspond to errors related to positions of the overlaid plurality of graphics on the on-screen view;

removing a first overlaid graphic of the overlaid plurality of graphics on the on-screen view of the channel based on a first graphics identifier (ID) of the first overlaid graphic that is in the programming schedule at a time code for a subsequent content item; and

reinstating the first overlaid graphic on the on-screen view of the channel after a number of video frames based on the detected one or more error parameters,

wherein the number of video frames is associated with the time code for the subsequent content item.

30. The non-transitory computer-readable medium according to claim 29, wherein the plurality of graphics comprises at least one of a channel logo, a sign, a symbol, a text placeholder, full screen graphics, partial screen graphics, text crawl, a rating key, or a ticker.