Finding and populating spatial ad surfaces in video

• US 9,078,048 B1
• Filed: 07/15/2008
• Issued: 07/07/2015
• Est. Priority Date: 07/15/2008
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Gargi, Ullas, Mohanty, Pranab
• Primary Examiner(s): SALCE, JASON P
• Application Number:
  US12/173,714
• Time in Patent Office: 2,548 Days
• Field of Search: 725/32, 725 34- 36, 382/103, 382/108, 382/154, 382/165, 705/14.4, 705/14.41, 705/14.46, 705/14.49, 705/14.53, 705/14.68, 705/14.69, 705/14.71, 705/14.73, 715/723, 715/725
• US Class Current: 1/1
• CPC Class Codes: H04N 21/23412 for generating or manipulating the scene composition of objects, e.g. MPEG-4 objects
• CPC Class Codes: H04N 21/23424 involving splicing one content stream with another content stream, e.g. for inserting or substituting an advertisement
• CPC Class Codes: H04N 21/235 Processing of additional data, e.g. scrambling of additional data or processing content descriptors
• CPC Class Codes: H04N 21/2547 Third Party Billing, e.g. billing of advertiser
• CPC Class Codes: H04N 21/25833 involving client hardware characteristics, e.g. manufacturer, processing or storage capabilities allocation of resources considering hardware capabilities in multiprogramming arrangements G06F9/5044; allocation of resources considering software capabilities in multiprogramming arrangements G06F9/5055
• CPC Class Codes: H04N 21/25891 being end-user preferences retrieval of video data in a video database based on user preferences G06F16/739; arrangements for recognizing users' preferences H04H60/46; user profiles in network data switching protocols H04L67/306; processing of user preferences or user profiles in wireless networks H04W8/18
• CPC Class Codes: H04N 21/2668 Creating a channel for a dedicated end-user group, e.g. insertion of targeted commercials based on end-user profiles information retrieval from the Internet by querying with filtering and personalisation G06F16/9535; arrangements for replacing or switching information during the broadcast H04H20/10; push services over packet-switching network H04L12/1859; adaptation of message content in packet-switching networks H04L51/063
• CPC Class Codes: H04N 21/2743 Video hosting of uploaded data from client
• CPC Class Codes: H04N 21/431 Generation of visual interfaces for content selection or interaction; Content or additional data rendering
• CPC Class Codes: H04N 21/435 Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
• CPC Class Codes: H04N 21/44012 involving rendering scenes according to scene graphs, e.g. MPEG-4 scene graphs
• CPC Class Codes: H04N 21/47202 for requesting content on demand, e.g. video on demand
• CPC Class Codes: H04N 21/4782 Web browsing , e.g. WebTV
• CPC Class Codes: H04N 21/812 involving advertisement data advertising per se G06Q30/02
• CPC Class Codes: H04N 21/8153 comprising still images, e.g. texture, background image
• CPC Class Codes: H04N 21/8456 by decomposing the content in the time domain, e.g. in time segments
• CPC Class Codes: H04N 7/165 Centralised control of user terminal subsequent to an upstream request signal H04N7/17345; Registering at central by two-way working H04N7/17309

Abstract

A video hosting web site receives uploaded video content and processes the video to determine ad surfaces. The ad surfaces comprise spatio-temporal regions of the video suitable for placement of advertisement such as background surfaces or other regions of low interest. The uploaded video and ad surfaces are stored to a video database that is accessible to viewers visiting the video hosting web site. When a shared video is requested, the video hosting site provides the requested video content together with the ad surfaces and advertisements. The advertisements are blended with the ad surfaces in the video at playtime so that the advertisements appear as part of the video scene. This allows the video hosting web site to present advertisements to the viewer without significantly distracting the viewer from the requested content.

Claims

1. A computer-implemented method for placing advertisements in video, comprising:

receiving via a web-based interface, a video from a content provider client device;

detecting a scene of the video comprising a plurality of frames between consecutive scene boundaries;

segmenting each frame of the scene into a foreground image and a background image;

averaging pixels of the background images across the plurality of frames to determine an average background image for the scene;

processing the video to automatically determine ad surface parameters based on appearance characteristics of the average background image, the ad surface parameters defining an ad surface comprising a spatio-temporal region of the video, the ad surface suitable for placement of an advertisement such that when the advertisement is blended with the ad surface, the advertisement appears as part of an object depicted in the video;

storing the video content and the ad surface parameters to a computer readable storage;

receiving a request from a viewer client device for the stored video;

transmitting the video to the viewer client device together with the ad surface parameters;

selecting an advertisement to deliver to the viewer client device for placement on the ad surface; and

transmitting the advertisement to the viewer client device, wherein the viewer client device processes the advertisement and the video in accordance with the ad surface parameters to blend the advertisement with the ad surface.

2. The method of claim 1, wherein the advertisement is selected based on stored preferences of a user of the viewer client device.

3. The method of claim 1, wherein the advertisement is selected based on stored preferences of a content provider that provided the video.

4. The method of claim 1, wherein the advertisement is selected based on stored preferences of an advertiser that provided the advertisement.

5. The method of claim 1, further comprising:

transmitting a representation of the ad surface to a reviewer client device;

modifying the ad surface parameters responsive to receiving modifications from the reviewer client device; and

storing the modified ad surface parameters to the computer readable storage medium.

6. The method of claim 1, wherein the spatio-temporal region suitable for placement of the advertisement comprises a surface of a background object in the video.

7. The method of claim 1, wherein the spatio-temporal region suitable for placement of the advertisement comprises a time portion of the video having a start time and an end time.

8. The method of claim 1, wherein processing the video to automatically determine the ad surface parameters further comprises:

computing a texture map based on the average background image;

computing an edge map based on the average background image; and

determining the ad surface parameters based on the texture map and the edge map.

9. The method of claim 8, wherein determining the ad surface parameters based on the texture map and the edge map comprises:

detecting polygons in the average background image based on the edge map; and

refining the detected polygons in the average background image based on the texture map.

10. The method of claim 8, wherein determining the ad surface parameters based on the texture map and the edge map comprises:

detecting polygons in the average background image based on the texture map; and

refining the detected polygons in the average background image based on the edge map.

11. The method of claim 1, further comprising:

receiving a purchase request for the ad surface from an advertiser, wherein the ad surface has a cost based on a predicted effectiveness of displaying an advertisement on the ad surface; and

assigning the ad surface to the advertiser responsive to the purchase request.

12. The method of claim 1, further comprising:

determining a computational capacity of the viewer client device; and

selecting, responsive to the determined computational capacity, an blending technique for processing the advertisement and the video to blend the advertisement with the ad surface; and

providing an indication of the selected blending technique to the viewer client device, wherein the viewer client device is adapted to use the indicated blending technique to blend the advertisement with the ad surface.

13. The method of claim 1, wherein the viewer client device processes the advertisement and the video to blend the advertisement with the ad surface according to blending parameters, further comprising:

detecting a change in the ad surface or the advertisement; and

adjusting the blending parameters responsive to the detected change.

14. A non-transitory computer readable storage medium storing computer executable instructions for placing advertisements in video, the instructions when executed cause a processor to execute steps including:

receiving via a web-based interface, a video from a content provider client device;

detecting a scene of the video comprising a plurality of frames between consecutive scene boundaries;

segmenting each frame of the scene into a foreground image and a background image;

averaging pixels of the background images across the plurality of frames to determine an average background image for the scene;

processing the video to automatically determine ad surface parameters based on appearance characteristics of the average background image, the ad surface parameters defining an ad surface comprising a spatio-temporal region of the video, the ad surface suitable for placement of an advertisement such that when the advertisement is blended with the ad surface, the advertisement appears as part of an object depicted in the video;

storing the video content and the ad surface parameters to a computer readable storage;

receiving a request from a viewer client device for the stored video;

transmitting the video to the viewer client device together with the ad surface parameters;

selecting an advertisement to deliver to the viewer client device for placement on the ad surface; and

transmitting the advertisement to the viewer client device, wherein the viewer client device processes the advertisement and the video in accordance with the ad surface parameters to blend the advertisement with the ad surface.

15. The non-transitory computer readable storage medium of claim 14, wherein the advertisement is selected based on stored preferences of a user of the viewer client device.

16. The non-transitory computer readable storage medium of claim 14, wherein the advertisement is selected based on stored preferences of a content provider that provided the video.

17. The non-transitory computer readable storage medium of claim 14, wherein the advertisement is selected based on stored preferences of an advertiser that provided the advertisement.

18. The non-transitory computer readable storage medium of claim 14, the instructions when executed further cause the processing system to:

transmit a representation of the ad surface to a reviewer client device;

modify the ad surface parameters responsive to receiving modifications from the reviewer client device; and

store the modified ad surface parameters to the computer readable storage medium.

19. The non-transitory computer readable storage medium of claim 14, wherein the spatio-temporal region suitable for placement of the advertisement comprises a surface of a background object in the video.

20. The non-transitory computer readable storage medium of claim 14, wherein the spatio-temporal region suitable for placement of the advertisement comprises a time portion of the video having a start time and an end time.

21. The non-transitory computer readable storage medium of claim 14, wherein processing the video to automatically determine the ad surface parameters further comprises:

computing a texture map based on the average background image;

computing an edge map based on the average background image; and

determining the ad surface parameters based on the texture map and the edge map.

22. The non-transitory computer readable storage medium of claim 21, wherein determining the ad surface parameters based on the texture map and the edge map comprises:

detecting polygons in the average background image based on the edge map; and

refining the detected polygons in the average background image based on the texture map.

23. The non-transitory computer readable storage medium of claim 21, wherein determining the ad surface parameters based on the texture map and the edge map comprises:

detecting polygons in the average background image based on the texture map; and

refining the detected polygons in the average background image based on the edge map.

24. The non-transitory computer readable storage medium of claim 14, the instructions when executed further cause the processing system to:

receiving a purchase request for the ad surface from an advertiser, wherein the ad surface has a cost based on a predicted effectiveness of displaying an advertisement on the ad surface; and

assigning the ad surface to the advertiser responsive to the purchase request.

25. A system for placing advertisements in video, comprising:

a network interface for receiving via a web-based interface, a video from a content provider client device;

an input server for;

detecting a scene of the video comprising a plurality of frames between consecutive scene boundaries;

segmenting each frame of the scene into a foreground image and a background image;

averaging pixels of the background images across the plurality of frames to determine an average background image for the scene; and

processing the video to automatically determine ad surface parameters based on appearance characteristics of the average background image, the ad surface parameters defining an ad surface comprising a spatio-temporal region of the video, the ad surface suitable for placement of an advertisement such that when the advertisement is blended with the ad surface, the advertisement appears as part of the object depicted in the video;

a storage module for storing the video content and the ad surface parameters;

an output server for;

receiving a request from a viewer client device for the stored video;

transmitting the video to the viewer client device together with the ad surface parameters, and

delivering a selected advertisement to the viewer client device for placement on the ad surface, wherein the viewer client device processes the advertisement and the video to blend the advertisement with the ad surface.

26. The system of claim 25, further comprising:

an advertising management system for selecting the advertisement to deliver to the viewer client device for placement on the ad surface, wherein the viewer client device processes the advertisement and the video to blend the advertisement with the ad surface.

27. The system of claim 25, wherein the spatio-temporal region suitable for placement of the advertisement comprises a surface of a background object in the video.

28. The system of claim 25, further comprising:

reviewer tools for delivering a representation of the ad surface to a reviewer for reviewing, and for receiving manual modifications to the ad surface from the reviewer.