DEEP PRODUCT PLACEMENT

• US 20180013977A1
• Filed: 06/07/2017
• Published: 01/11/2018
• Est. Priority Date: 07/11/2016
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Martineau, Justin C.
• Granted Patent: US 10,726,443 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 2111/10 Numerical modelling
• CPC Class Codes: G06F 30/20 Design optimisation, verification or simulation optimisation, verification or simulation of circuit designs G06F30/30
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G06Q 30/0241 Advertisements
• CPC Class Codes: G06Q 30/0251 Targeted advertisements
• CPC Class Codes: G06Q 30/0641 Shopping interfaces
• CPC Class Codes: G06V 10/255 Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
• CPC Class Codes: G11B 27/036 Insert-editing
• CPC Class Codes: H04N 9/8205 involving the multiplexing of an additional signal and the colour video signal

Abstract

One embodiment provides a method comprising identifying a product placement opportunity for a product in a frame of a piece of content during playback of the piece of content on a display device. The method further comprises determining a location in the frame to insert product placement content for the product based on a learned statistical model representing learned placement patterns related to the product. The method further comprises modifying the product placement content based on one or more objects present in the frame, and inserting a product placement for the product in the piece of content by inserting the modified product placement content in the frame based on the location. The modified product placement content appears to occur naturally in the piece of content.

Claims

1. A method comprising:

identifying a product placement opportunity for a product in a frame of a piece of content during playback of the piece of content on a display device;

determining a location in the frame to place product placement content associated with the product based on a learned statistical model representing learned placement patterns related to the product; and

placing the product placement content in the frame based on the location, wherein the product placement content appears to occur naturally in the piece of content.

2. The method of claim 1, further comprising:

modifying the product placement content based on information related to the frame.

3. The method of claim 2, further comprising:

receiving a product placement profile, wherein the product placement profile includes the product placement content and one or more placement instructions for placing the product placement content in media content.

4. The method of claim 3, wherein identifying a product placement opportunity for a product in a frame comprises:

applying a learned object detector model to the frame to detect presence of the product and one or more objects that fit the product placement profile.

5. The method of claim 3, wherein determining a location in the frame to place product placement content associated with the product comprises:

applying a learned object localization model to the frame to localize the one or more objects present in the frame;

applying the learned statistical model to the frame to compute one or more average offset value relative to the one or more localized objects in the frame; and

determining coordinates for the location based on the one or more average offset values.

6. The method of claim 5, wherein modifying the product placement content based on information related to the frame comprises:

scaling and rotating the product placement content relative to the localized one or more objects in the frame; and

illuminating the product placement content based on lighting information indicative of one or more illumination sources in the frame and light direction from each illumination source.

7. The method of claim 1, further comprising:

reporting the product placement to a server managing sharing of advertisement revenue associated with the product placement.

8. The method of claim 2, wherein modifying the product placement content based on information related to the frame comprises:

obtaining contextual information associated with at least one of;

a viewer of the display device, an environment of the viewer, the display device, time, date, location, information captured by one or more other devices connected to the display device, or a combination thereof; and

modifying the product placement content based on the contextual information.

9. The method of claim 1, further comprising:

detecting viewer interaction with the product placement, wherein the viewer interaction comprises at least one of viewer feedback or a viewer impression; and

invoking an action based on the viewer interaction, wherein the action includes at least one of;

purchasing the product from an online site, disabling the product placement, rating the product placement, redirecting to additional content, or a combination thereof.

10. The method of claim 3, further comprising:

identifying an interaction opportunity between the product and an object present in the frame based on the product placement profile;

determining pose models for the product and the object present in the frame; and

generating an interaction action between the product and the object present in the frame by manipulating the pose models.

11. The method of claim 1, wherein the product placement content comprises at least one of visual content or audio content related to the product.

12. A system, comprising:

at least one processor; and

a non-transitory processor-readable memory device storing instructions that when executed by the at least one processor causes the at least one processor to perform operations including;

identifying a product placement opportunity for a product in a frame of a piece of content during playback of the piece of content on a display device;

determining a location in the frame to insert product placement content for the product based on a learned statistical model representing learned placement patterns related to the product;

modifying the product placement content based on one or more objects present in the frame; and

inserting a product placement for the product in the piece of content by inserting the modified product placement content in the frame based on the location, wherein the modified product placement content appears to occur naturally in the piece of content.

13. The system of claim 12, the operations further comprising:

receiving a product placement profile for the product from a server, wherein the product placement profile includes the product placement content and one or more placement instructions for inserting the product placement content in visual content.

14. The system of claim 13, wherein identifying a product placement opportunity for a product in a frame comprises:

applying a learned object detector model to the frame to detect presence of the product and one or more objects that fit the product placement profile.

15. The system of claim 13, wherein determining a location in the frame to insert product placement content for the product comprises:

applying a learned object localization model to the frame to localize the one or more objects present in the frame;

applying the learned statistical model to the frame to compute one or more average offset value relative to the one or more localized objects in the frame; and

determining coordinates for the location based on the one or more average offset values.

16. The system of claim 15, wherein modifying the product placement content based on one or more objects present in the frame comprises:

scaling and rotating the product placement content relative to the localized objects in the frame; and

illuminating the product placement content based on lighting information indicative of one or more illumination sources in the frame and light direction from each illumination source.

17. The system of claim 12, the operations further comprising:

obtaining contextual information associated with at least one of;

a viewer of the display device, an environment of the viewer, the display device, time, date, location, information captured by one or more other devices connected to the display device, or a combination thereof; and

personalizing one or more product placements inserted in the piece of content based the contextual information.

18. A method comprising:

in a training stage;

generating a product placement profile for a product based on one or more pieces of visual content with natural occurrences of the product, wherein the product placement profile comprises product placement content for the product and one or more placement instructions for inserting the product placement content in visual content; and

sending the product placement profile to a display device to allow the display device to dynamically insert a product placement for the product in a piece of content during playback of the piece of content based on the product placement profile.

19. The method of claim 18, wherein generating a product placement profile for a product comprises:

training an object detector model to detect the product in visual content based on a first set of training data comprising one or more pieces of visual content with natural occurrences of the product;

training an object localization model to determine a localization bounding box for the product in visual content based on a second set of training data comprising one or more pieces of visual content including visual representations of the product inserted at random; and

training a statistical model to determine a normal placement of the product in visual content with respect to one or more other objects in the same visual content based on one or more spatial relationships of the product to one or more objects co-occurring with the product in the first set of training data.

20. The method of claim 19, wherein generating a product placement profile for a product comprises:

receiving one or more pre-specified rules on product placement from an advertiser or a content provider via a user interface.