Context based augmented advertisement

• US 10,575,067 B2
• Filed: 03/29/2017
• Issued: 02/25/2020
• Est. Priority Date: 01/04/2017
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Ravindran, Sourabh, Yoo, Youngjun
• Primary Examiner(s): Pierorazio, Michael B.
• Application Number:
  US15/473,272
• Publication Number:
  US 20180192160A1
• Time in Patent Office: 1,063 Days
• Field of Search: 725 14
• CPC Class Codes: G06Q 30/0252 based on events or environment, e.g. weather or festivals
• CPC Class Codes: G06Q 30/0261 based on user location
• CPC Class Codes: G06Q 30/0277 Online advertisement
• CPC Class Codes: H04N 21/23418 involving operations for analysing video streams, e.g. detecting features or characteristics television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/23424 involving splicing one content stream with another content stream, e.g. for inserting or substituting an advertisement
• CPC Class Codes: H04N 21/812 involving advertisement data advertising per se G06Q30/02

Abstract

One embodiment provides a method comprising analyzing one or more frames of a piece of content to determine a context of the one or more frames, determining a product to advertise in the piece of content based on the context, and augmenting the piece of content with a product placement for the product. The product placement appears to occur naturally in the piece of content.

Claims

1. A method comprising:

analyzing one or more frames of a piece of content to determine context information associated with the one or more frames, wherein the context information comprises one or more descriptors related to a scene occurring in the one or more frames, and the one or more descriptors identify the scene and its context;

determining a product to advertise in the piece of content by determining a product relevant to the scene based on the one or more descriptors; and

augmenting the piece of content with a product placement for the product, wherein the product placement appears to occur naturally in the piece of content.

2. The method of claim 1, further comprising:

retrieving product placement content for the product from an advertisement server.

3. The method of claim 1, wherein analyzing one or more frames of a piece of content comprises:

applying one or more vision based scene modeling techniques to the one or more frames to extract one or more high-level semantics from the one or more frames;

extracting sensor based contextual information from sensor data associated with the one or more frames; and

generating a scene profile for the one or more frames based on the one or more high-level semantics and the sensor based contextual information, wherein the scene profile comprises the one or more descriptors.

4. The method of claim 3, wherein the one or more vision based scene modeling techniques comprise at least one of scene annotation or scene classification.

5. The method of claim 3, wherein determining a product to advertise in the piece of content comprises:

mapping the scene profile to one or more brand profiles for one or more products.

6. The method of claim 2, wherein augmenting the piece of content with a product placement for the product comprises:

extracting structural, depth and pose information from the one or more frames;

tracking one or more changes in orientation and perspective during one or more scene transitions; and

rendering the product placement content based on the structural, depth and pose information and the one or more changes tracked to align the product placement content with one or more objects occurring in the one or more frames.

7. The method of claim 6, wherein the rendering comprises modifying one of scale, rotation or illumination of the product placement content relative to the one or more objects.

8. The method of claim 1, further comprising:

uploading the augmented piece of content to a content server.

9. The method of claim 1, wherein the piece of content is user generated.

10. The method of claim 9, further comprising:

sharing advertisement revenue associated with the product placement with a creator of the piece of content.

11. The method of claim 1, wherein the piece of content is one or stored media content or a live stream of media content.

12. A system, comprising:

at least one processor; and

a non-transitory processor-readable memory device storing instructions that when executed by the at least one processor causes the at least one processor to perform operations including;

analyzing one or more frames of a piece of content to determine context information associated with the one or more frames, wherein the context information comprises one or more descriptors related to a scene occurring in the one or more frames, and the one or more descriptors identify the scene and its context;

determining a product to advertise in the piece of content by determining a product relevant to the scene based on the one or more descriptors; and

augmenting the piece of content with a product placement for the product, wherein the product placement appears to occur naturally in the piece of content.

13. The system of claim 12, wherein the operations further comprise:

retrieving product placement content for the product from an advertisement server.

14. The system of claim 12, wherein analyzing one or more frames of a piece of content comprises:

applying one or more vision based scene modeling techniques to the one or more frames to extract one or more high-level semantics from the one or more frames;

extracting sensor based contextual information from sensor data associated with the one or more frames; and

generating a scene profile for the one or more frames based on the one or more high-level semantics and the sensor based contextual information, wherein the scene profile comprises the one or more descriptors.

15. The system of claim 14, wherein the one or more vision based scene modeling techniques comprise at least one of scene annotation or scene classification.

16. The system of claim 14, wherein determining a product to advertise in the piece of content comprises:

mapping the scene profile to one or more brand profiles for one or more products.

17. The system of claim 13, wherein augmenting the piece of content with a product placement for the product comprises:

extracting structural, depth and pose information from the one or more frames;

tracking one or more changes in orientation and perspective during one or more scene transitions; and

rendering the product placement content based on the structural, depth and pose information and the one or more changes tracked to align the product placement content with one or more objects occurring in the one or more frames.

18. The system of claim 17, wherein the rendering comprises modifying one of scale, rotation or illumination of the product placement content relative to the one or more objects.

19. A non-transitory computer readable storage medium including instructions to perform a method comprising:

analyzing one or more frames of a piece of content to determine context information associated with the one or more frames, wherein the context information comprises one or more descriptors related to a scene occurring in the one or more frames, and the one or more descriptors identify the scene and its context;

determining a product to advertise in the piece of content by determining a product relevant to the scene based on the one or more descriptors; and

augmenting the piece of content with a product placement for the product, wherein the product placement appears to occur naturally in the piece of content.

20. The computer readable storage medium of claim 19, wherein analyzing one or more frames of a piece of content comprises:

applying one or more vision based scene modeling techniques to the one or more frames to extract one or more high-level semantics from the one or more frames;

extracting sensor based contextual information from sensor data associated with the one or more frames; and

generating a scene profile for the one or more frames based on the one or more high-level semantics and the sensor based contextual information, wherein the scene profile comprises one or more descriptors of the one or more frames.

21. The computer readable storage medium of claim 19, wherein augmenting the piece of content with a product placement for the product comprises:

extracting structural, depth and pose information from the one or more frames;

tracking one or more changes in orientation and perspective during one or more scene transitions; and

rendering the product placement content based on the structural, depth and pose information and the one or more changes tracked to align the product placement content with one or more objects occurring in the one or more frames.

22. The computer readable storage medium of claim 21, wherein the rendering comprises modifying one of scale, rotation or illumination of the product placement content relative to the one or more objects.

23. The computer readable storage medium of claim 19, wherein the method further comprises:

sharing advertisement revenue associated with the product placement with a creator of the piece of content.

24. A method comprising:

gathering contextual information associated with an electronic signage, wherein the contextual information comprises one or more descriptors related to an environment of the electronic signage, and the one or more descriptors identify the environment and its context;

determining a product to advertise on the electronic signage by determining a product relevant to the environment based on the one or more descriptors; and

updating a display of the electronic signage with product placement content for the product.

25. The method of claim 24, wherein the contextual information is indicative of at least one of:

temperature of the environment, weather at the environment, a current event relevant to a location of the electronic signage, or an audience member within proximity of the electronic signage.