GENERATING A SHOPPABLE VIDEO

• US 20160371546A1
• Filed: 06/16/2015
• Published: 12/22/2016
• Est. Priority Date: 06/16/2015
• Status: Active Grant

• Current Assignee: Adobe Inc.
• Original Assignee: Adobe Systems Incorporated (Adobe Inc.)
• Inventors: YADAV, VIKAS, KRISHNAMURTHY, BALAJI, SARKAR, MAUSOOM, MANGLA, RAJIV, MALIK, GITESH
• Granted Patent: US 10,354,290 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06Q 30/0276 Advertisement creation
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20021 Dividing image into blocks, subimages or windows
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06V 10/757 Matching configurations of points or features
• CPC Class Codes: G06V 20/46 Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
• CPC Class Codes: G06V 20/47 Detecting features for summarising video content
• CPC Class Codes: G11B 27/034 on discs G11B27/036, G11B27/038 take precedence
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence
• CPC Class Codes: G11B 27/34 Indicating arrangements indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D
• CPC Class Codes: H04N 21/254 Management at additional data server, e.g. shopping server, rights management server arrangements for maintenance or administration in data networks H04L41/00; network services using third party service providers H04L67/53
• CPC Class Codes: H04N 21/44222 Analytics of user selections, e.g. selection of programs or purchase activity monitoring of user selections in data processing systems G06F11/34; arrangements for monitoring the user's behaviour or opinions in broadcast systems H04H60/33
• CPC Class Codes: H04N 5/7605 on discs or drums H04N5/781, H04N5/805, H04N5/83, H04N5/85 take precedence
• CPC Class Codes: H04N 5/93 Regeneration of the television signal or of selected parts thereof

Abstract

Embodiments of the present invention provide systems and methods for automatically generating a shoppable video. A video is parsed into one or more scenes. Products and their corresponding product information are automatically associated with the one or more scenes. The shoppable video is then generated using the associated products and corresponding product information such that the products are visible in the shoppable video based on a scene in which the products are found.

Claims

1. One or more computer storage media storing computer-useable instructions that, when used by one or more computing devices, cause the one or more computing devices to generate a shoppable video, comprising:

parsing a video into one or more scenes;

identifying one or more products in a first scene of the one or more scenes;

retrieving product information corresponding to the one or more products in the first scene;

automatically associating the one or more products and the corresponding product information with the first scene; and

generating the shoppable video using the associated one or more products and the corresponding product information.

2. The one or more computer storage media of claim 1, wherein the identifying of the one or more products in the first scene is based on a plurality of features identified in the video frames of the first scene compared to a plurality of features identified in a plurality of product images stored in a database.

3. The one or more computer storage media of claim 1, wherein automatically associating the one or more products and the corresponding product information with the first scene further comprises:

parsing the video into a plurality of frames;

for a first frame of the plurality of frames, producing one or more copy frames having different resolutions; and

partitioning the first frame and the one or more copy frames into a plurality of tiles.

4. The one or more computer storage media of claim 3, further comprising:

for each of the plurality of tiles in the first frame and the one or more copy frames,

(1) computing a feature vector,

(2) comparing values of the feature vector corresponding to each of the plurality of tiles to values of feature vectors of product images stored in a database, and

(3) based on the comparison, computing a product score for a plurality of products having corresponding product images stored in the database.

5. The one or more computer storage media of claim 4, further comprising based at least on the computed product score, determining the one or more products to associate with the at least the portion of the one or more scenes.

6. The one or more computer storage media of claim 4, further comprising:

for a second frame of the plurality of frames, producing one or more copy frames having different resolutions than the second frame;

partitioning the second frame and the one or more copy frames into a plurality of tiles; and

for each of the plurality of tiles in the second frame and the one or more copy frames,

(1) computing a feature vector,

(2) comparing values of the feature vector corresponding to each of the plurality of tiles to values of the feature vectors of the product images stored in the database, and

(3) based on the comparison, computing the product score for a plurality of products having corresponding product images stored in the database.

7. The one or more computer storage media of claim 4, wherein the feature vector is an n-dimensional vector of numerical features that represent an object.

8. The one or more computer storage media of claim 4, wherein the feature vector numerically represents one or more of color components, length, area, shape description, gradient magnitude, or gradient direction of the object in the plurality of tiles and the product images.

9. The one or more computer storage media of claim 4, wherein the feature vectors are computed using a deep convolutional neural network.

10. The one or more computer storage media of claim 1, further comprising:

determining the one or more products to automatically associate with the at least the portion of the one or more scenes, wherein the determining is based, at least, on a computed product scores for a plurality of products having corresponding product images stored in a database.

11. The one or more computer storage media of claim 10, wherein the computed product scores are based on, at least, monitoring votes received for the plurality of products, such that a first product having a higher vote count than a second product indicates an increased likelihood that the first product is visible in the video.

12. The one or more computer storage media of claim 3, wherein at least a portion of the plurality of tiles overlap with another tile.

13. The one or more computer storage media of claim 1, wherein the shoppable video presents the one or more products based on a scene in which each product is found.

14. A computerized method for generating a shoppable video, the computerized method comprising:

computing a feature vector for each of a plurality of product images stored in a database;

computing a feature vector for a first tile in a first frame of a video;

comparing values of the feature vector of the first tile to values of the feature vectors for the plurality of product images;

based on the comparing, determining that the first tile includes an image of a first product that corresponds to a first set of images of the plurality of product images stored in the database; and

generating the shoppable video that comprises product information corresponding to the first product.

15. The computerized method of claim 14, further comprising:

computing a feature vector for a second tile in a first frame of a video;

comparing values of the feature vector of the second tile to values of the feature vectors for the plurality of product images;

based on the comparing, determining that the second tile includes an image of a second product that corresponds to a second set of images of the plurality of product images stored in the database; and

generating the shoppable video that comprises the product information corresponding to the first product and the second product.

16. The computerized method of claim 14, further comprising grouping the plurality of frames into one or more scenes, wherein the shoppable video includes a presentation of the first product and one or more other products found in the video based on a scene in which each product is found.

17. The computerized method of claim 14, wherein determining that the first tile includes an image of a first product that corresponds to a first set of images of the plurality of product images stored in the database further comprises monitoring votes received for the first product, and wherein the first product having a higher vote count than a second product indicates an increased likelihood that the first product is visible in the video.

18. A computerized system comprising:

a datastore storing product images and associated feature vectors computed for the product images;

one or more processors; and

one or more computer storage media storing computer-useable instructions that, when used by the one or more processors, cause the one or more processors to;

divide a video into a plurality of frames,

for each of the plurality of frames, generate one or more copy frames having different resolutions,

partition each of the plurality of frames and each of the one or more copy frames into a plurality of tiles,

for each of the plurality of tiles, compute a feature vector,

compare values of the feature vector for each of the plurality of tiles to values of the feature vectors computed for the product images stored in the database, and

use the comparison of the values of the feature vectors to determine which products are visible in each of the plurality of frames.

19. The system of claim 18, wherein the one or more processors further generate a shoppable video using the products that are determined to be visible in each of the plurality of frames.

20. The system of claim 19, further comprising grouping the plurality of frames into one or more scenes, wherein the shoppable video includes a presentation of the products that are determined to be visible in each of the plurality of frames based on a scene in which each product is found.