In-video product annotation with web information mining
First Claim
1. A computer method for providing product annotation in a video to one or more users, the method comprising:
- generating a product visual signature for a product by at least;
collecting an unannotated expert product image of the product from an expert product repository,searching for a plurality of unannotated product images from a plurality of web resources different from the expert product repository, the plurality of unannotated product images related to the unannotated expert product image,selecting a subset of the plurality of unannotated product images by filtering the plurality of unannotated product images based on a similarity measure to the unannotated expert product image, andgenerating the product visual signature from the unannotated expert product image and the subset of the plurality of unannotated product images;
receiving a video for product annotation, the video comprising a plurality of video frames;
extracting a plurality of key frames from the video frames; and
for each key frame;
generating a visual representation of the key framed;
comparing the visual representation with a plurality of product visual signatures including the product visual signature; and
determining, based on the comparison, that the key frame contains the product identified by the product visual signature.
2 Assignments
0 Petitions
Accused Products
Abstract
A system provides product annotation in a video to one or more users. The system receives a video from a user, where the video includes multiple video frames. The system extracts multiple key frames from the video and generates a visual representation of the key frame. The system compares the visual representation of the key frame with a plurality of product visual signatures, where each visual signature identifies a product. Based on the comparison of the visual representation of the key frame and a product visual signature, the system determines whether the key frame contains the product identified by the visual signature of the product. To generate the plurality of product visual signatures, the system collects multiple training images comprising multiple of expert product images obtained from an expert product repository, each of which is associated with multiple product images obtained from multiple web resources.
-
Citations
17 Claims
-
1. A computer method for providing product annotation in a video to one or more users, the method comprising:
-
generating a product visual signature for a product by at least; collecting an unannotated expert product image of the product from an expert product repository, searching for a plurality of unannotated product images from a plurality of web resources different from the expert product repository, the plurality of unannotated product images related to the unannotated expert product image, selecting a subset of the plurality of unannotated product images by filtering the plurality of unannotated product images based on a similarity measure to the unannotated expert product image, and generating the product visual signature from the unannotated expert product image and the subset of the plurality of unannotated product images; receiving a video for product annotation, the video comprising a plurality of video frames; extracting a plurality of key frames from the video frames; and for each key frame; generating a visual representation of the key framed; comparing the visual representation with a plurality of product visual signatures including the product visual signature; and determining, based on the comparison, that the key frame contains the product identified by the product visual signature. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9)
-
-
10. A non-transitory computer-readable storage medium storing executable computer program instructions for providing on-demand digital assets hosting services to one or more users, the computer program instructions when executed by a processor cause a system to perform operations comprising:
-
generating a product visual signature for a product by at least; collecting an unannotated expert product image of the product from an expert product repository, searching for a plurality of unannotated product images from a plurality of web resources different from the expert product repository, the plurality of unannotated product images related to the unannotated expert product image, selecting a subset of the plurality of unannotated product images by filtering the plurality of unannotated product images based on a similarity measure to the unannotated expert product image, and generating the product visual signature from the unannotated expert product image and the subset of the plurality of unannotated product images; receiving a video from a user for product annotation, the video comprising a plurality of video frames; extracting a plurality of key frames from the video; and for each key frame; extracting a plurality of visual features from the key frame; grouping the plurality of visual features into a plurality of clusters; and generating a multi-dimensional bag visual words histogram as a visual representation of the key frame; comparing the visual representation with a plurality of product visual signatures comprising the product visual signature; determining, based on the comparison, whether the key frame contains the product identified by the product visual signature. - View Dependent Claims (11, 12, 13, 14, 15, 16, 17)
-
Specification