SYSTEM FOR OPTIMIZED EMBEDDING AND ANALYSIS OF BRANDED CONTENT

• US 20190096094A1
• Filed: 09/26/2017
• Published: 03/28/2019
• Est. Priority Date: 09/26/2017
• Status: Active Grant

• Current Assignee: ETH Zurich, Disney Enterprises Incorporated (The Walt Disney Company)
• Original Assignee: ETH Zurich, Disney Enterprises Incorporated (The Walt Disney Company)
• Inventors: Sorkine-Hornung, Alex, Meier, Simone, Bazin, Jean-Charles, Schriber, Sasha, Gross, Markus, Wang, Oliver
• Granted Patent: US 10,580,165 B2
• Time in Patent Office: Days
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 7/207 for motion estimation over a hierarchy of resolutions multi-resolution motion estimation or hierarchical motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/53
• CPC Class Codes: G06T 7/97 Determining parameters from multiple pictures depth or shape recovery from multiple images G06T7/55; stereo camera calibration G06T7/85
• CPC Class Codes: G11B 27/00 Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
• CPC Class Codes: G11B 27/036 Insert-editing
• CPC Class Codes: H04N 19/583 Motion compensation with overlapping blocks
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/8541 involving branching, e.g. to different story endings

Abstract

The present disclosure relates to an apparatus, system and method for processing transmedia content data. More specifically, the disclosure provides for identifying and inserting one item of media content within another item of media content, e.g. inserting a video within a video, such that the first item of media content appears as part of the second item. The invention involves analysing a first visual media item to identify one or more spatial locations to insert the second visual media item within the image data of the first visual media item, detecting characteristics of the one or more identified spatial locations, transforming the second visual media item according to the detected characteristics and combining the first visual media item and second visual media item by inserting the transformed second visual media item into the first visual media item at the one or more identified spatial locations.

Claims

1. A computer-implemented method for inserting a second visual media item into a first visual media item, visual media items being formed of image data, the method comprising:

analysing the first visual media item to identify one or more spatial locations to insert the second visual media item within the image data of the first visual media item;

detecting characteristics of the one or more identified spatial locations within the first visual media item;

transforming the second visual media item according to the detected characteristics; and

combining the first visual media item and second visual media item by inserting the transformed second visual media item into the first visual media item at the one or more identified spatial locations such that when the combined visual media item is output the second visual media item is displayed within the first media item at the one of more identified spatial locations.

2. The method of claim 1, wherein analysing the first visual media item to identify spatial locations comprises searching for one or more of flat, homogeneous, and repetitive textured areas with the image data of the first visual media item using one or more of homographies, texture segmentation, colour segmentation, saliency mapping and machine learning techniques.

3. The method of claim 2, wherein the method further comprises analysing the identified spatial locations to determine one or more preferred spatial locations.

4. The method of claim 1, wherein detecting characteristics of the one or more identified spatial locations comprises employing line detection, vanishing point computation, rectifying regular texture to determine spatial characteristics of the identified spatial location.

5. The method of claim 1, wherein the first visual media item is an image.

6. The method of claim 1, wherein the first visual media item is a video, and wherein the image data of the first visual media item comprises video frames.

7. The method of claim 6, wherein the method further comprises analysing the first visual media item to identify one or more temporal locations to insert the second visual media item within the image data of the first virtual media item.

8. The method of claim 7, wherein the first and second visual media items are combined such that when the combined visual media item is output, the second visual media item is displayed within the first media item at the one or more identified spatial locations and the one or more temporal locations.

9. The method of claim 7, wherein identifying one or more temporal locations comprises analysing the image data of the first visual media item to detect shot transitions.

10. The method of claim 9, wherein determining the characteristics of the identified spatial locations comprises determining the characteristics of the identified spatial locations at the identified temporal locations by measuring of the stability of tracking in the image data of the first visual media item.

11. The method of claim 10, wherein the stability of tracking is measured by KLT tracking, SfM or improved tracking for planes.

12. The method of claim 9, wherein determining the characteristics of the identified spatial locations comprises analysing motion of the spatial location within the image data of multiple video frames across the temporal location to determine the 3-dimensional structure of the spatial location.

13. The method of claim 1, wherein the method further comprises selecting the second visual media item from a collection of candidate visual media items.

14. The method of claim 13, wherein the selection of the second visual media item is based on an analysis of colour, lighting, or colour and lighting in the first visual media item at the identified spatial location and each candidate visual media item.

15. The method of claim 13, wherein the selection of the second visual media item is based on a comparison of at least two of:

metadata of the first visual media item, metadata of each candidate visual media item and user preference data.

16. The method of claim 1, wherein the second visual media item is an image or a video.

17. A non-transitory computer-readable medium containing computer readable instructions which, when executed by processing circuitry, cause the processing circuitry to:

analyze the first visual media item to identify one or more spatial locations to insert the second visual media item within the image data of the first visual media item;

detect characteristics of the one or more identified spatial locations within the first visual media item;

transform the second visual media item according to the detected characteristics; and

combine the first visual media item and second visual media item by inserting the transformed second visual media item into the first visual media item at the one or more identified spatial locations such that when the combined visual media item is output the second visual media item is displayed within the first media item at the one of more identified spatial locations.

18. An electronic device comprising:

processing circuitry; and

a memory coupled with and readable by the processing circuitry and storing therein a set of instructions which, when executed by the one or more processors, cause the processor to;

analyze the first visual media item to identify one or more spatial locations to insert the second visual media item within the image data of the first visual media item;

detect characteristics of the one or more identified spatial locations within the first visual media item;

transform the second visual media item according to the detected characteristics; and

combine the first visual media item and second visual media item by inserting the transformed second visual media item into the first visual media item at the one or more identified spatial locations such that when the combined visual media item is output the second visual media item is displayed within the first media item at the one of more identified spatial locations.