Text to image translation
First Claim
1. A method comprising:
- under control of one or more processors configured with executable instructions;
receiving a search query;
determining one or more semantic classes applicable to the search query;
identifying multiple terms that are (i) analogous to one another and (ii) within one or more of the semantic classes;
obtaining web images corresponding to at least a subset of the multiple terms;
obtaining near-duplicate images of a subset of the web images associated with the one or more semantic classes;
removing noisy images from the near-duplicate images to create a denoised image set;
determining representative text associated with the denoised image set;
based at least in part on the representative text associated with the denoised image set, selecting at least one representative image for the one or more semantic classes;
training at least one query-dependent classifier based at least in part on the representative text;
employing the at least one query-dependent classifier to create the denoised image set for individual ones of the semantic classes; and
selecting the at least one representative image from the denoised image set as an image answer for the search query.
2 Assignments
0 Petitions
Accused Products
Abstract
Techniques are described for online real time text to image translation suitable for virtually any submitted query. Semantic classes and associated analogous items for each of the semantic classes are determined for the submitted query. One or more requests are formulated that are associated with analogous items. The requests are used to obtain web based images and associated surrounding text. The web based images are used to obtain associated near-duplicate images. The surrounding text of images is analyzed to create high-quality text associated with each semantic class of the submitted query. One or more query dependent classifiers are trained online in real time to remove noisy images. A scoring function is used to score the images. The images with the highest score are returned as a query response.
124 Citations
19 Claims
-
1. A method comprising:
-
under control of one or more processors configured with executable instructions; receiving a search query; determining one or more semantic classes applicable to the search query; identifying multiple terms that are (i) analogous to one another and (ii) within one or more of the semantic classes; obtaining web images corresponding to at least a subset of the multiple terms; obtaining near-duplicate images of a subset of the web images associated with the one or more semantic classes; removing noisy images from the near-duplicate images to create a denoised image set; determining representative text associated with the denoised image set; based at least in part on the representative text associated with the denoised image set, selecting at least one representative image for the one or more semantic classes; training at least one query-dependent classifier based at least in part on the representative text; employing the at least one query-dependent classifier to create the denoised image set for individual ones of the semantic classes; and selecting the at least one representative image from the denoised image set as an image answer for the search query. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8)
-
-
9. A system comprising:
-
memory; one or more processors communicatively coupled to the memory; and instructions stored on the memory and when executed by the one or more processors, the instructions configure the system to; obtain semantic classes for a search query; identifying multiple terms that are (i) analogous to one another and (ii) within one or more of the semantic classes; collect web based images associated with a subset of the multiple terms; obtain near-duplicate images for a subset of the web based images; remove noisy images from the near-duplicate images to create a denoised image set; based at least in part on representative text associated with the denoised image set, select at least one representative image for individual ones of the semantic classes; train at least one query-dependent classifier or multiple one-on-one query-dependent classifiers based at least in part on representative text associated with the near-duplicate images; employ the at least one query-dependent classifier or multiple one-on-one query-dependent classifiers to create the denoised image set for individual ones of the one or more semantic classes; use a scoring function to rank images in the denoised image set based at least in part on a relevance score that measures a similarity of individual images of an associated denoised image set with its corresponding semantic class, and a confidence score proportional to a number of nearest neighbors associated with individual ones of the images of an associated denoised image set; and determine at least one representative image from the denoised image set as an image answer for the search query based at least in part on the rank of the individual images in the denoised image set. - View Dependent Claims (10, 11, 12, 13, 14, 15, 16, 17)
-
-
18. A method comprising:
-
determining semantic classes of one or more terms of a query; identifying multiple terms that are (i) analogous to one another and (ii) within one or more of the semantic classes; for each of the semantic classes of the query; obtaining web images corresponding to a subset of the multiple terms associated with a corresponding semantic class; obtaining near-duplicate images for a subset of the web images; removing noisy images from the near-duplicate images to create a denoised image set; determining representative text associated with images of the denoised image set; based at least in part on representative text associated with the denoised image set, selecting at least one representative image for individual ones of the of the semantic classes as an image answer; training at least one query-dependent classifier based at least in part on the representative text; employing the at least one query-dependent classifier to create the denoised image set for individual ones of the semantic classes; and selecting the at least one representative image from the denoised image set as an image answer for the search query. - View Dependent Claims (19)
-
Specification