PRESENTATION OF CONTENT ITEMS SYNCHRONIZED WITH MEDIA DISPLAY

• US 20180025405A1
• Filed: 07/25/2016
• Published: 01/25/2018
• Est. Priority Date: 07/25/2016
• Status: Active Grant

• Current Assignee: Meta Platforms, Inc. (f/k/a Facebook, Inc.)
• Original Assignee: Meta Platforms, Inc. (f/k/a Facebook, Inc.)
• Inventors: Jones, Christopher William, D'Arcy, Mark Gerard, Lee, Ji Byol
• Granted Patent: US 10,643,264 B2
• Time in Patent Office: Days
• CPC Class Codes: G06Q 30/0631 Item recommendations
• CPC Class Codes: G06Q 30/0633 Lists, e.g. purchase orders, compilation or processing
• CPC Class Codes: G06Q 30/0643 Graphical representation of items or shoppers
• CPC Class Codes: G06V 20/46 Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
• CPC Class Codes: G06V 2201/09 Recognition of logos
• CPC Class Codes: H04N 21/4532 involving end-user characteristics, e.g. viewer profile, preferences monitoring of user activities for profile generation for accessing a video database G06F16/739; user profiles in network data switching protocols H04L67/306; processing of user preferences or user profiles in wireless networks H04W8/18

Abstract

An online system displays relevant products along with a video watched by a user in a synchronized manner, which allows the user to explore relevant content (e.g., products recommended to the user) while the user is watching the video. The online system receives a video including a sequence of video frames to be displayed to the user, and the video may be associated with a specific theme, e.g., a soccer game, and one or more video frames may include points of interest (referred as “Q points”). The online system detects or receives Q points associated with corresponding video frames, where a Q point in a video frame identifies an object of interest in the video frame. The online system also correlates user profile information of the user with the detected points to select relevant products tailored to the user.

Claims

1. A computer-implemented method, the method comprising:

receiving a video for display to a user, the video comprising a plurality of video frames;

for each video frame;

analyzing visual features of the video frame; and

detecting one or more points of interest in the video frame based on the analysis of the visual features of the video frame, a point of interest in the video frame being associated with an object in the video frame, and the object in the video frame being related to one or more products;

selecting one or more products based on the detected points of interest in the plurality of video frames; and

providing for display the selected products along with the corresponding video frames being displayed to the user, wherein a selected product and a corresponding video frame showing the selected product are displayed concurrently in an user interface to the user.

2. The method of claim 1, providing for display the selected products along with the corresponding video frame comprising:

for each video frame currently being displayed to the user;

determining whether the video frame has a detected point interest; and

selecting one or more products associated with the detected point of interest in the video frame.

3. The method of claim 2, further comprising:

providing for display the video frame in a first portion of the display interface shown to the user; and

providing for display one or more selected products in a scrolling carousel of products located in a second portion of the display interface, wherein at least one of the selected products is same or similar to a product shown in the video frame.

4. The method of claim 3, wherein providing for display one or more selected products in a scrolling carousel of products comprises:

displaying one or more tools in the second portion of the display interface, wherein the user interacts with the displayed products through one or more tools.

5. The method of claim 4, wherein the one or more tools displayed in the second portion of the display interface comprise one of the following:

a purchase tool for the user to purchase the displayed product; and

a bookmark tool for the user to bookmark the displayed product.

6. The method of claim 1, wherein a point of interest in a video frame is identified by one of a timestamp associated with the video frame and frame number of the video frame.

7. The method of claim 1, wherein a selected product associated with a detected point of interest in a video frame shares one or more characteristics of the product related by identified object in the video frame.

8. The method of claim 1, further comprising:

analyzing user profile information associated with the user;

deriving user preference for products based on the analysis of the user profile information; and

updating the selection of the products based on the derived user preference for products.

9. The method of claim 1, wherein analyzing visual features of a video frame comprising:

applying a trained image feature extraction model to the video frame to extract a plurality of visual features;

identifying an object in the video frame based on the extracted visual features; and

identifying one or more items associated with the identified object, each identified item corresponding to a product that is likely to be of interest to the user.

10. The method of claim 1, wherein the video displayed to the user is one of a pre-recorded video and a broadcast video streamed to the user in real time.

11. A computer-implemented method, the method comprising:

detecting an opportunity to provide content to a user of an online system;

providing a content item including a video for display to the user and a plurality of selected products based on detected points of interest in video frames of the video;

receiving an indication from the user to play the video;

playing the video and displaying each of the plurality of selected products in a synchronized way with the playing of the video, wherein a selected product and a corresponding video frame showing the selected product are displayed concurrently in an user interface to the user;

receiving an interaction by the user with one of the selected products; and

providing access to content describing the product to the user.

12. The method of claim 11, wherein the content describing the product includes detailed product information of the product, wherein the product information includes at least one of:

price, way to purchase, style, size, color, gender, and brand name of the product.

13. A non-transitory computer readable storage medium storing computer program instructions, the computer program instructions when executed by a computer processor causes the processor to perform the steps of:

receiving a video for display to a user, the video comprising a plurality of video frames;

for each video frame;

analyzing visual features of the video frame; and

detecting one or more points of interest in the video frame based on the analysis of the visual features of the video frame, a point of interest in the video frame being associated with an object in the video frame, and the object in the video frame being related to one or more products;

selecting one or more products based on the detected points of interest in the plurality of video frames; and

providing for display the selected products along with the corresponding video frames being displayed to the user, wherein a selected product and a corresponding video frame showing the selected product are displayed concurrently in an user interface to the user.

14. The non-transitory computer readable storage medium of claim 13, providing for display the selected products along with the corresponding video frame comprising:

for each video frame currently being displayed to the user;

determining whether the video frame has a detected point interest; and

selecting one or more products associated with the detected point of interest in the video frame.

15. The non-transitory computer readable storage medium of claim 14, further comprising:

providing for display the video frame in a first portion of the display interface shown to the user; and

providing for display one or more selected products in a scrolling carousel of products located in a second portion of the display interface, wherein at least one of the selected products is same or similar to a product shown in the video frame.

16. The non-transitory computer readable storage medium of claim 15, wherein presenting one or more selected products in a scrolling carousel of products comprises:

displaying one or more tools in the second portion of the display interface, wherein the user interacts with the displayed products through one or more tools.

17. The non-transitory computer readable storage medium of claim 16, wherein the one or more tools displayed in the second portion of the display interface comprise one of the following:

a purchase tool for the user to purchase the displayed product; and

a bookmark tool for the user to bookmark the displayed product.

18. The non-transitory computer readable storage medium of claim 13, wherein a point of interest in a video frame is identified by one of a timestamp associated with the video frame and frame number of the video frame.

19. The non-transitory computer readable storage medium of claim 13, wherein a selected product associated with a detected point of interest in a video frame shares one or more characteristics of the product related by identified object in the video frame.

20. The non-transitory computer readable storage medium of claim 13, further comprising:

analyzing user profile information associated with the user;

deriving user preference for products based on the analysis of the user profile information; and

updating the selection of the products based on the derived user preference for products.

21. The non-transitory computer readable storage medium of claim 13, wherein analyzing visual features of a video frame comprising:

applying a trained image feature extraction model to the video frame to extract a plurality of visual features;

identifying an object in the video frame based on the extracted visual features; and

identifying one or more items associated with the identified object, each identified item corresponding to a product that is likely to be of interest to the user.

22. The non-transitory computer readable storage medium of claim 13, wherein the video displayed to the user is one of a pre-recorded video and a broadcast video streamed to the user in real time.