DYNAMIC VIDEO OVERLAYS

• US 20170180795A1
• Filed: 12/16/2016
• Published: 06/22/2017
• Est. Priority Date: 12/16/2015
• Status: Active Grant

• Current Assignee: Roku, Inc.
• Original Assignee: Gracenote, Inc. (RR Donnelley & Sons Company)
• Inventors: Dunker, Peter, Harron, Wilson, Pearce, David Henry, Li, Sherman Ling Fung, Dimitriou, Konstantinos Antonios, Lee, Dewey Ho, Cremer, Markus K., Scherf, Steven D.
• Granted Patent: US 10,123,073 B2
• Time in Patent Office: Days
• CPC Class Codes: H04N 21/23418 involving operations for analysing video streams, e.g. detecting features or characteristics television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/431 Generation of visual interfaces for content selection or interaction; Content or additional data rendering
• CPC Class Codes: H04N 21/4312 involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations
• CPC Class Codes: H04N 21/4314 for fitting data in a restricted space on the screen, e.g. EPG data in a rectangular grid
• CPC Class Codes: H04N 21/4318 by altering the content in the rendering process, e.g. blanking, blurring or masking an image region image enhancement or restoration in general G06T5/00
• CPC Class Codes: H04N 21/4325 by playing back content from the storage medium reproduction of recorded television signals H04N5/76; reproduction of recorded television signals H04N9/79
• CPC Class Codes: H04N 21/4333 Processing operations in response to a pause request
• CPC Class Codes: H04N 21/4383 Accessing a communication channel
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/44016 involving splicing one content stream with another content stream, e.g. for substituting a video clip
• CPC Class Codes: H04N 21/458 Scheduling content for creating a personalised stream, e.g. by combining a locally stored advertisement with an incoming stream; Updating operations, e.g. for OS modules ; time-related management operations arrangements for replacing or switching information during the broadcast or during the distribution H04H20/10
• CPC Class Codes: H04N 21/47217 for controlling playback functions for recorded or on-demand content, e.g. using progress bars, mode or play-point indicators or bookmarks specific graphical features in visual interfaces H04N21/4312
• CPC Class Codes: H04N 21/812 involving advertisement data advertising per se G06Q30/02

Abstract

A client device accesses a video input stream from an intermediate device for display. The client device analyzes the video input stream to determine that the video input stream matches a template indicating a portion of the video input stream that is created by the intermediate device. Based on the video input stream matching the template, a video output stream generated by the client device is modified to include the portion of the video input stream generated by the intermediate device. For example, commercials transmitted from a national content provider to a smart TV via a set-top box may be replaced with targeted commercials. During the replacement, informational messages or menus generated by the set-top box may be detected and the replacement video altered by the smart TV to include the messages or menus generated by the set-top box.

Claims

1. A method comprising:

causing a video output stream to be presented on a display;

accessing a video input stream containing content from a content provider and an intermediate device;

accessing, from a database that stores a plurality of templates, a template corresponding to a screen portion generated by the intermediate device,

analyzing, by one or more processors, the video input stream to determine that at least a portion of content of a frame of the video input stream matches the template;

based on the at least a portion of content of the frame of the video input stream matching the template, modifying the video output stream to include the screen portion generated by the intermediate device; and

causing, by the one or more processors, the modified video output stream to be presented on the display.

2. The method of claim 1, further comprising:

accessing a second template corresponding to a second screen portion generated by the intermediate device;

analyzing the video input stream to determine that at least a portion of content of a second frame of the video input stream matches the second template; and

based on the at least a portion of content of the second frame of the video input stream matching the second template, ceasing modification of the video output stream.

3. The method of claim 1, wherein the modifying of the video output stream to includes the screen portion generated by the intermediate device comprises:

copying the screen portion generated by the intermediate device from the video input stream to the video output stream on a pixel-by-pixel basis.

4. The method of claim 1, wherein the modifying of the video output stream to include the screen portion generated by the intermediate device comprises:

identifying information contained in the screen portion generated by the intermediate device; and

modifying the video output stream to include the identified information.

5. The method of claim 1, further comprising:

identifying content of the video input stream from the content provider, and

based on the identified content from the content provider, identifying replacement content; and

wherein the causing of the video output stream to be presented on the display comprises causing the replacement content to be presented on the display.

6. The method of claim 1, wherein the analyzing of the video input stream to determine that at least a portion of content of the frame of the video input stream matches the template comprises determining that the at least a portion of content of the frame of the video input stream contains an icon added by the intermediate device.

7. The method of claim 1, wherein the analyzing of the video input stream to determine that at least a portion of content of the frame of the video input stream matches the template comprises determining that at least a portion of content of the frame of the video input stream contains predetermined text added by the intermediate device.

8. A system comprising:

a memory that stores instructions; and

one or more processors configured by the instructions to perform operations comprising;

causing a video output stream to be presented on a display;

accessing a video input stream containing content from a content provider and an intermediate device;

accessing, from a database, a template corresponding to a screen portion generated by the intermediate device;

analyzing the video input stream to determine that at least a portion of content of a frame of the video input stream matches the template;

based on the at least a portion of content of the frame of the video input stream matching the template, modifying the video output stream to include the screen portion generated by the intermediate device; and

causing the modified video output stream to be presented on the display.

9. The system of claim 8, wherein the operations further comprise:

accessing a second template corresponding to a second screen portion generated by the intermediate device;

analyzing the video input stream to determine that at least a portion of content of a second frame of the video input stream matches the second template; and

based on the at least a portion of content of the second frame of the video input stream matching the second template, ceasing modification of the video output stream.

10. The system of claim 8, wherein the modifying of the video output stream to include the screen portion generated by the intermediate device comprises:

copying the screen portion generated by the intermediate device from the video input stream to the video output stream on a pixel-by-pixel basis.

11. The system of claim 8, wherein the modifying of the video output stream to include the screen portion generated by the intermediate device comprises:

identifying information contained in the screen portion generated by the intermediate device; and

modifying the video output stream to include the identified information.

12. The system of claim 8, wherein:

the operations further comprise;

identifying content of the video input stream from the content provider; and

based on the identified content from the content provider, identifying replacement content; and

the causing of the video output stream to be presented on the display comprises causing the replacement content to be presented on the display.

13. The system of claim 8, wherein the analyzing of the video input stream to determine that at least a portion of content of the frame of the video input stream matches the template comprises determining that the at least a portion of content of the frame of the video input stream contains an icon added by the intermediate device.

14. The system of claim 8, wherein the analyzing of the video input stream to determine that at least a portion of content of the frame of the video input stream matches the template comprises determining that at least a portion of content of the frame of the video input stream contains predetermined text added by the intermediate device.

15. A non-transitory machine-readable medium that stores instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:

causing a video output stream to be presented on a display;

accessing a video input stream containing content from a content provider and an intermediate device;

accessing a template corresponding to a screen portion generated by the intermediate device;

analyzing the video input stream to determine that at least a portion of content of a frame of the video input stream matches the template;

based on the at least a portion of content of the frame of the video input stream matching the template, modifying the video output stream to include the screen portion generated by the intermediate device; and

causing the modified video output stream to be presented on the display.

16. The non-transitory machine-readable medium of claim 15, wherein the operations further comprise:

accessing a second template corresponding to a second screen portion generated by the intermediate device;

analyzing the video input stream to determine that at least a portion of content of a second frame of the video input stream matches the second template; and

based on the at least a portion of content of the second frame of the video input stream matching the second template, ceasing modification of the video output stream.

17. The non-transitory machine-readable medium of claim 15, wherein the modifying of the video output stream to include the screen portion generated by the intermediate device comprises:

copying the screen portion generated by the intermediate device from the video input stream to the video output stream on a pixel-by-pixel basis.

18. The non-transitory machine-readable medium of claim 15, wherein the modifying of the video output stream to include the screen portion generated by the intermediate device comprises:

identifying information contained in the screen portion generated by the intermediate device; and

modifying the video output stream to include the identified information.

19. The non-transitory machine-readable medium of claim 15, wherein:

the operations further comprise;

identifying content of the video input stream from the content provider; and

based on the identified content from the content provider, identifying replacement content; and

the causing of the video output stream to be presented on the display comprises causing the replacement content to be presented on the display.

20. The non-transitory machine-readable medium of claim 15, wherein the analyzing of the video input stream to determine that at least a portion of content of the frame of the video input stream matches the template comprises determining that the at least a portion of content of the frame of the video input stream contains an icon added by the intermediate device.