Inserting interactive objects into video content

• US 8,745,657 B2
• Filed: 02/12/2009
• Issued: 06/03/2014
• Est. Priority Date: 02/13/2008
• Status: Active Grant

• Current Assignee: Innovid
• Original Assignee: Innovid, Inc. (Innovid Corp.)
• Inventors: Chalozin, Tal, Netter, Izhak Zvi
• Primary Examiner(s): SHEPARD, JUSTIN E
• Application Number:
  US12/867,075
• Publication Number:
  US 20110016487A1
• Time in Patent Office: 1,937 Days
• Field of Search: 725/36
• US Class Current: 725/32
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: H04N 21/23418 involving operations for analysing video streams, e.g. detecting features or characteristics television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/235 Processing of additional data, e.g. scrambling of additional data or processing content descriptors
• CPC Class Codes: H04N 21/2353 specifically adapted to content descriptors, e.g. coding, compressing or processing of metadata
• CPC Class Codes: H04N 21/239 Interfacing the upstream path of the transmission network, e.g. prioritizing client content requests hybrid fiber coaxial [HFC] networks for upstream channel allocation for video distribution H04L12/2801; flow control in data networks H04L47/10; real-time communication protocols in data switching networks H04L65/00; scheduling or organising the servicing of application requests H04L67/60
• CPC Class Codes: H04N 21/24 Monitoring of processes or resources, e.g. monitoring of server load, available bandwidth, upstream requests monitoring of server performance or load G06F11/34; arrangements for observation, testing or troubleshooting for broadcast or for distribution combined with broadcast H04H20/12
• CPC Class Codes: H04N 21/25 Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies maintenance or administration in data networks H04L41/00
• CPC Class Codes: H04N 21/251 Learning process for intelligent management, e.g. learning user preferences for recommending movies details of learning user preferences for the retrieval of video data in a video database G06F16/739; computer systems using learning methods G06N3/08
• CPC Class Codes: H04N 21/252 Processing of multiple end-users' preferences to derive collaborative data
• CPC Class Codes: H04N 21/258 Client or end-user data management, e.g. managing client capabilities, user preferences or demographics, processing of multiple end-users preferences to derive collaborative data arrangements for services using the result on the distributing side of broadcast systems H04H60/66; profiles in network data switching protocols H04L67/30
• CPC Class Codes: H04N 21/25825 involving client display capabilities, e.g. screen resolution of a mobile phone optimising the visualisation of content during browsing in the Internet G06F16/9577; processing of terminal status or physical abilities in wireless networks H04W8/22; authentication in wireless network security H04W12/06
• CPC Class Codes: H04N 21/25883 being end-user demographical data, e.g. age, family status or address arrangements for identifying locations of users in broadcast systems H04H60/52
• CPC Class Codes: H04N 21/2668 Creating a channel for a dedicated end-user group, e.g. insertion of targeted commercials based on end-user profiles information retrieval from the Internet by querying with filtering and personalisation G06F16/9535; arrangements for replacing or switching information during the broadcast H04H20/10; push services over packet-switching network H04L12/1859; adaptation of message content in packet-switching networks H04L51/063
• CPC Class Codes: H04N 21/43 Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware real-time communication protocols in data switching networks H04L65/00
• CPC Class Codes: H04N 21/431 Generation of visual interfaces for content selection or interaction; Content or additional data rendering
• CPC Class Codes: H04N 21/4312 involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations
• CPC Class Codes: H04N 21/4314 for fitting data in a restricted space on the screen, e.g. EPG data in a rectangular grid
• CPC Class Codes: H04N 21/4316 for displaying supplemental content in a region of the screen, e.g. an advertisement in a separate window
• CPC Class Codes: H04N 21/4318 by altering the content in the rendering process, e.g. blanking, blurring or masking an image region image enhancement or restoration in general G06T5/00
• CPC Class Codes: H04N 21/435 Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
• CPC Class Codes: H04N 21/4355 : involving reformatting operations of additional data, e.g. HTML pages on a television screen optimising the visualization of content for information retrieval from the Internet G06F16/9577; adaptation of message content in packet-switching networks H04L51/066; media handling at the source in data packet switching networks H04L65/764
• CPC Class Codes: H04N 21/4356 : by altering the spatial resolution, e.g. to reformat additional data on a handheld device, attached to the STB
• CPC Class Codes: H04N 21/4358 : for generating different versions, e.g. for different peripheral devices
• CPC Class Codes: H04N 21/44 : Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
• CPC Class Codes: H04N 21/44012 : involving rendering scenes according to scene graphs, e.g. MPEG-4 scene graphs
• CPC Class Codes: H04N 21/442 : Monitoring of processes or resources, e.g. detecting the failure of a recording device, monitoring the downstream bandwidth, the number of times a movie has been viewed, the storage space available from the internal hard disk arrangements for monitoring broadcast services or broadcast-related services H04H60/29; arrangements for identifying or recognising characteristics with a direct linkage to broadcast information H04H60/35; monitoring of user activities for profile generation for accessing a video database G06F16/739; monitoring in wireless networks H04W24/00
• CPC Class Codes: H04N 21/44213 : Monitoring of end-user related data arrangements for monitoring the users' behaviour or opinions in broadcast systems H04H60/33
• CPC Class Codes: H04N 21/4524 : involving the geographical location of the client retrieval from the Internet by querying based on geographical locations G06F16/9537; systems specially adapted for using geographical information in broadcast systems H04H60/70; protocols in which the network application is adapted for the location of the user terminal in communication control or processing H04L67/52; services making use of the location of users or terminals in wireless networks H04W4/02; locating users or terminals in wireless networks H04W64/00
• CPC Class Codes: H04N 21/4725 : using interactive regions of the image, e.g. hot spots details of information retrieval from the Internet by using URLs G06F16/955; processing chained hypermedia data for information retrieval G06F16/94
• CPC Class Codes: H04N 21/812 : involving advertisement data advertising per se G06Q30/02
• CPC Class Codes: H04N 21/84 : Generation or processing of descriptive data, e.g. content descriptors systems specially adapted for using meta-information in broadcast systems H04H60/73
• CPC Class Codes: H04N 21/8586 : by using a URL processing chained hypermedia data for information retrieval G06F16/94; information retrieval from the Internet by using URLs G06F16/955; URL in broadcast information H04H20/93; Web-based protocols H04L67/02
• CPC Class Codes: H04N 5/2723 : Insertion of virtual advertisement; Replacing advertisements physical present in the scene by virtual advertisement data processing systems or methods specially adapted for marketing G06Q30/02

Abstract

Enriching video-based content with additional content is enabled by defining metadata for a video comprising a place holder and matching attribute. When the video is viewed, a matching server is utilized to select an object to be inserted in the place holder according to the matching attribute. The matching server selects the object according to video content, represented by matching attribute, according to user related information, and according to type of place holder. The place holder may be utilized to dynamically insert an interactive object to the video. The inserted object may be seamlessly inserted to the video.

Claims

1. A computerized apparatus comprising:

an interface for receiving video content of a captured video;

the video content comprises a set of one or more frames;

an interface for receiving indication of a region within the video content; and

a place holder creator module for creating a place holder within the video content;

the place holder is created based on the indication of a region;

the place holder is configured to enable an insertion of an object from a set of objects to the video content;

the place holder is further configured to enable the insertion such that the object is consistent with an environment of the video content;

a video metadata server for storing video metadata of the video content, wherein said video metadata server comprises characteristics of the video content;

wherein the object is inserted into the video content according to the characteristics of the video content;

a background subtraction for determining obstacles blocking view to the place holder according to the metadata.

2. The computerized apparatus of claim 1, wherein the video content comprises more than one frame.

3. The computerized apparatus of claim 2 further comprising a matching server for matching the object from a plurality of objects to the place holder.

4. The computerized apparatus of claim 3 operationally coupled to a storage device for storing data relating to the place holder.

5. The computerized apparatus of claim 2, wherein:

the indication of the region is in a key frame; and

the place holder creator module determines the place holder in a set of frames comprising at least the key frame and another frame.

6. The computerized apparatus of claim 5 further comprising a tracking module for computing tracking data;

the tracking data comprises tracking an area within a frame of the video content;

the place holder creator module determines the place holder according to the tracking data.

7. The computerized apparatus of claim 2, wherein the video metadata comprises a position of a camera that captured frames of the video content, and wherein the place holder creator modifies the object inserted into the video content according to changes in perspective of frames of the video content, caused by change in position of the camera and according to the positional reference of the place holder in the frame in which the object is inserted into the video content.

8. A computerized apparatus comprising:

an interface for receiving video metadata associated with a video content of a captured video;

the video content comprises a set of one or more frames;

the video metadata comprises a place holder having positional reference characteristics of specific frames of the set of one or more frames and a matching attribute;

the matching attribute is associated with the place holder;

a video metadata server for storing video metadata of the video content, wherein said video metadata server communicates with a video content database for obtaining the video metadata of the video content;

an interface to an object database comprising at least one object;

a matching module for selecting an object from the object database according to the video metadata and according to the content obtained from the video content database;

the object is designated to be inserted to a portion of the place holder;

a background subtraction for determining obstacles blocking view to the place holder according to the metadata.

9. The computerized apparatus of claim 8, wherein the place holder is configured to enable an insertion of the object to a portion of the place holder;

the place holder is further configured to enable the insertion such that the object is consistent with an environment of the video content.

10. The computerized apparatus of claim 8, wherein the object is configured to be inserted into more than one video.

11. The computerized apparatus of claim 8, wherein the video metadata comprises an additional place holder having characteristics associated with the specific frames of the set of one or more frames.

12. The computerized apparatus of claim 11, wherein the matching attribute is not associated with the additional place holder.

13. The computerized apparatus of claim 11, wherein the place holder defines a first region in a frame of the video content;

and the additional place holder defines a second region in the frame.

14. The computerized apparatus of claim 8, wherein the object is an interactive object.

15. The computerized apparatus of claim 8 further comprising an interface for sending the object to a computerized client.

16. The computerized apparatus of claim 15, further comprising the computerized client;

the computerized client is configured to display the video content and the object.

17. The computerized apparatus of claim 16, wherein the matching module comprises:

a client information module for receiving client information regarding the computerized client; and

a client-aware object matching module for selecting the object from the object database according to the video metadata and the client information.

18. The computerized apparatus of claim 17, wherein the client information comprises demographic information regarding a user of the computerized client.

19. A computerized system comprising:

an interface for receiving video content of a captured video;

the video content comprises a set of one or more frames;

an input device for receiving input from a first user;

a metadata creator module for determining video metadata associated with the video content;

the video metadata comprises;

characteristics of specific frames of the set of one or more frames;

a place holder defining a region within a portion of the set of one or more frames;

the region is based on input received by the input device;

a match attribute associated with the place holder;

a metadata server for storing the video metadata;

a computerized server for sending the video content to a computerized client;

an object database; and

a matching server for selecting an object from the object database according to the place holder, the match attribute and the computerized client;

wherein each frame of the video content is associated with a positional reference that defines the place holder;

a background subtraction for determining obstacles blocking view to the place holder according to the metadata.

20. The computerized system of claim 19 wherein the computerized client displays the captured video and the selected object to a second user;

the selected object is displayed according to the place holder.