DESCRIPTIVE METADATA EXTRACTION AND LINKAGE WITH EDITORIAL CONTENT

• US 20160119691A1
• Filed: 10/28/2014
• Published: 04/28/2016
• Est. Priority Date: 10/28/2014
• Status: Active Application

• Current Assignee: Disney Enterprises Incorporated (The Walt Disney Company)
• Original Assignee: Disney Enterprises Incorporated (The Walt Disney Company)
• Inventors: ACCARDO, Anthony M., HEDINSSON, Skarphedinn S., SMOLIC, Aljosa Aleksej Andrej, BEELER, Thabo D., FARR GUIU, Miquel .
• Granted Patent: US 10,264,329 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: H04N 21/233 Processing of audio elementary streams monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/234 Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
• CPC Class Codes: H04N 21/23418 involving operations for analysing video streams, e.g. detecting features or characteristics television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/4316 for displaying supplemental content in a region of the screen, e.g. an advertisement in a separate window
• CPC Class Codes: H04N 21/4394 involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/4622 Retrieving content or additional data from different sources, e.g. from a broadcast channel and the Internet web site content organization and management for information retrieval from the Internet G06F16/958; transmission by internet of broadcast information H04H60/82; stock exchange data over packet-switching network H04L12/1804; push services including data channel over packet-switching network H04L12/1859
• CPC Class Codes: H04N 21/4722 for requesting additional data associated with the content
• CPC Class Codes: H04N 21/4725 using interactive regions of the image, e.g. hot spots details of information retrieval from the Internet by using URLs G06F16/955; processing chained hypermedia data for information retrieval G06F16/94
• CPC Class Codes: H04N 21/6581 Reference data, e.g. a movie identifier for ordering a movie or a product identifier in a home shopping application
• CPC Class Codes: H04N 21/8133 specifically related to the content, e.g. biography of the actors in a movie, detailed information about an article seen in a video program
• CPC Class Codes: H04N 21/8352 involving content or source identification data, e.g. Unique Material Identifier [UMID]
• CPC Class Codes: H04N 21/8358 involving watermark protecting executable software by watermarking G06F21/16; image watermarking in general G06T1/0021; watermarks inserted in still images for transmission purposes H04N1/32144; inserting watermarks during video coding H04N19/467
• CPC Class Codes: H04N 21/84 Generation or processing of descriptive data, e.g. content descriptors systems specially adapted for using meta-information in broadcast systems H04H60/73

Abstract

Embodiments provide techniques for distributing supplemental content based on content entities within video content. Embodiments include receiving video data containing an embedded watermark at a first position within the video data. The embedded watermark is detected at the first position within the video data. Embodiments also include transmitting, to a remote content server, a message specifying a time stamp corresponding to the first position within the video data. In response to transmitting the message, supplemental content corresponding to a content entity depicted within the video content at the first position within the video content is received from the remote content server. Embodiments also include outputting the video data for display together with at least an indication of the supplemental content.

Claims

1. A method, comprising:

receiving video data for a first instance of video content and containing an embedded watermark at a first position within the video data;

detecting the embedded watermark at the first position within the video data;

transmitting, to a remote content server, a message specifying a time stamp that identifies the first position within the video data at which the embedded watermark was detected;

in response to transmitting the message, receiving, from the remote content server, supplemental content corresponding to a content entity depicted within frames of the video content at the first position; and

outputting the video data for display together with at least an indication of the supplemental content.

2. The method of claim 1, wherein the embedded watermark comprises an audio watermark within an audio stream of the video data, and wherein the audio watermark is embedded at a position within the audio stream corresponding to the time stamp.

3. The method of claim 1, wherein playback of the video data is output in a playback region of a user interface, and wherein outputting the video data for display together with at least an indication of the supplemental content further comprises:

outputting the at least an indication of the supplemental content within the user interface in a region adjacent to the playback region.

4. The method of claim 1, wherein playback of the video data is output in a playback region of a user interface, and wherein outputting the video data for display together with at least an indication of the supplemental content further comprises:

outputting the at least an indication of the supplemental content within the user interface as an overlay on top of the playback region.

5. The method of claim 1, wherein outputting the video data for display together with at least an indication of the supplemental content further comprises:

outputting the at least an indication of the supplemental content as one or more links, such that an instance of supplemental content is loaded within the user interface responsive to the corresponding one of the one or more links being selected.

6. The method of claim 1, wherein the supplemental content comprises at least one of:

(i) image data, (ii) audio data, (iii) video data, (iv) bibliographical data, and (v) textual data.

7. The method of claim 1, wherein the video data is received from a content server that generates the video containing the embedded watermark at a first position within the video data by performing an operation, comprising:

analyze the video data to identify a known content entity within two or more frames of the video data;

perform a first temporal consistency enhancement operation across the frames of the video data to optimize the identification of the known content entity within the two or more frames;

determine, for each of the two or more frames, a region of pixels within the respective frame that correspond to the known content entity;

perform second a temporal consistency enhancement operation across the identified two or more frames of the video data to optimize the region of pixels corresponding to the known content entity with respect to both space and time;

determine supplemental content corresponding to the known content entity; and

embed a watermark at a first position within the video data, such that the watermark corresponds to an identifier associated with the determined supplemental content.

8. The method of claim 1, wherein the message specifying the time stamp corresponding to the first position within the video data further specifies a client device identifier, and wherein the remote content server is configured to select the supplemental content based at least in part on the client device identifier.

9. A non-transitory computer-readable medium containing program code that, when executed by operation of one or more computer processors, performs an operation comprising:

receiving video data for a first instance of video content and containing an embedded watermark at a first position within the video data;

detecting the embedded watermark at the first position within the video data;

transmitting, to a remote content server, a message specifying a time stamp that identifies the first position within the video data at which the embedded watermark was detected;

in response to transmitting the message, receiving, from the remote content server, supplemental content corresponding to a content entity depicted within frames of the video content at the first position; and

outputting the video data for display together with at least an indication of the supplemental content.

10. The non-transitory computer-readable medium of claim 9, wherein the embedded watermark comprises an audio watermark within an audio stream of the video data, and wherein the audio watermark is embedded at a position within the audio stream corresponding to the time stamp.

11. The non-transitory computer-readable medium of claim 9, wherein playback of the video data is output in a playback region of a user interface, and wherein outputting the video data for display together with at least an indication of the supplemental content further comprises:

outputting the at least an indication of the supplemental content within the user interface in a region adjacent to the playback region.

12. The non-transitory computer-readable medium of claim 9, wherein playback of the video data is output in a playback region of a user interface, and wherein outputting the video data for display together with at least an indication of the supplemental content further comprises:

outputting the at least an indication of the supplemental content within the user interface as an overlay on top of the playback region.

13. The non-transitory computer-readable medium of claim 9, wherein outputting the video data for display together with at least an indication of the supplemental content further comprises:

outputting the at least an indication of the supplemental content as one or more links, such that an instance of supplemental content is loaded within the user interface responsive to the corresponding one of the one or more links being selected.

14. The non-transitory computer-readable medium of claim 9, wherein the supplemental content comprises at least one of:

(i) image data, (ii) audio data, (iii) video data, (iv) bibliographical data, and (v) textual data.

15. The non-transitory computer-readable medium of claim 9, wherein the video data is received from a content server that generates the video containing the embedded watermark at a first position within the video data by performing an operation, comprising:

analyze the video data to identify a known content entity within two or more frames of the video data;

perform a first temporal consistency enhancement operation across the frames of the video data to optimize the identification of the known content entity within the two or more frames;

determine, for each of the two or more frames, a region of pixels within the respective frame that correspond to the known content entity;

perform second a temporal consistency enhancement operation across the identified two or more frames of the video data to optimize the region of pixels corresponding to the known content entity with respect to both space and time;

determine supplemental content corresponding to the known content entity; and

embed a watermark at a first position within the video data, such that the watermark corresponds to an identifier associated with the determined supplemental content.

16. A system, comprising:

a processor; and

a memory containing a program that, when executed by the processor, performs an operation comprising;

receiving video data for a first instance of video content and containing an embedded watermark at a first position within the video data;

detecting the embedded watermark at the first position within the video data;

transmitting, to a remote content server, a message specifying a time stamp that identifies the first position within the video data at which the embedded watermark was detected;

in response to transmitting the message, receiving, from the remote content server, supplemental content corresponding to a content entity depicted within frames of the video content at the first position within the video content; and

outputting the video data for display together with at least an indication of the supplemental content.

17. The system of claim 16, wherein the embedded watermark comprises an audio watermark within an audio stream of the video data, and wherein the audio watermark is embedded at a position within the audio stream corresponding to the time stamp.

18. The system of claim 16, wherein playback of the video data is output in a playback region of a user interface, and wherein outputting the video data for display together with at least an indication of the supplemental content further comprises:

outputting the at least an indication of the supplemental content within the user interface in a region adjacent to the playback region.

19. The system of claim 16, wherein playback of the video data is output in a playback region of a user interface, and wherein outputting the video data for display together with at least an indication of the supplemental content further comprises:

outputting the at least an indication of the supplemental content within the user interface as an overlay on top of the playback region.

20. The system of claim 16, wherein the video data is received from a content server that generates the video containing the embedded watermark at a first position within the video data by performing an operation, comprising:

analyze the video data to identify a known content entity within two or more frames of the video data;

perform a first temporal consistency enhancement operation across the frames of the video data to optimize the identification of the known content entity within the two or more frames;

determine, for each of the two or more frames, a region of pixels within the respective frame that correspond to the known content entity;

perform second a temporal consistency enhancement operation across the identified two or more frames of the video data to optimize the region of pixels corresponding to the known content entity with respect to both space and time;

determine supplemental content corresponding to the known content entity; and

embed a watermark at a first position within the video data, such that the watermark corresponds to an identifier associated with the determined supplemental content.