Refining image annotations
First Claim
1. A method for automatically training an image relevance model and using the image relevance model to provide image search results in response to queries, the method being implemented by an image search apparatus comprising a data processing apparatus, and the method comprising:
- receiving, by the image search apparatus and for each image in a set of images, a corresponding set of text labels, each text label being determined to be indicative of subject matter of the image;
for each text label, determining, by the image search apparatus, one or more confidence values, each confidence value being a measure of confidence that the text label accurately describes the subject matter of a threshold number of respective images to which the text label corresponds;
identifying, as high confidence labels and by the image search apparatus, text labels for which each of the one or more confidence values meets at least one of a precision measurement threshold and a frequency measurement threshold;
training, by the image search apparatus and using a set of training text labels and images corresponding to the training text labels, the image relevance model, wherein the trained image relevance model determines a relevance of an image to a text query, the set of training labels including only labels that have been identified as high confidence labels;
identifying, using the trained image relevance model, one or more images to provide in response to a received text query received from a user device; and
providing, in response to the received text query and to the user device, one or more search results that depict the one or more images.
2 Assignments
0 Petitions
Accused Products
Abstract
Methods, systems and apparatus for refining image annotations. In one aspect, a method includes receiving, for each image in a set of images, a corresponding set of labels determined to be indicative of subject matter of the image. For each label, one or more confidence values are determined. Each confidence value is a measure of confidence that the label accurately describes the subject matter of a threshold number of respective images to which it corresponds. Labels for which each of the one or more confidence values meets a respective confidence threshold are identified as high confidence labels. For each image in the set of images, labels in its corresponding set of labels that are high confidence labels are identified. Images having a corresponding set of labels that include at least a respective threshold number of high confidence labels are identified as high confidence images.
-
Citations
24 Claims
-
1. A method for automatically training an image relevance model and using the image relevance model to provide image search results in response to queries, the method being implemented by an image search apparatus comprising a data processing apparatus, and the method comprising:
-
receiving, by the image search apparatus and for each image in a set of images, a corresponding set of text labels, each text label being determined to be indicative of subject matter of the image; for each text label, determining, by the image search apparatus, one or more confidence values, each confidence value being a measure of confidence that the text label accurately describes the subject matter of a threshold number of respective images to which the text label corresponds; identifying, as high confidence labels and by the image search apparatus, text labels for which each of the one or more confidence values meets at least one of a precision measurement threshold and a frequency measurement threshold; training, by the image search apparatus and using a set of training text labels and images corresponding to the training text labels, the image relevance model, wherein the trained image relevance model determines a relevance of an image to a text query, the set of training labels including only labels that have been identified as high confidence labels; identifying, using the trained image relevance model, one or more images to provide in response to a received text query received from a user device; and providing, in response to the received text query and to the user device, one or more search results that depict the one or more images. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8)
-
-
9. A system for automatically training an image relevance model and using the image relevance model to provide image search results in response to queries, the system comprising:
-
an image search apparatus comprising a data processing apparatus; and a memory storage apparatus in data communication with the data processing apparatus, the memory storage apparatus storing instructions executable by the data processing apparatus and that upon such execution cause the data processing apparatus to perform operations comprising; receiving, by the image search apparatus and for each image in a set of images, a corresponding set of text labels, each text label being determined to be indicative of subject matter of the image; for each text label, determining, by the image search apparatus, one or more confidence values, each confidence value being a measure of confidence that the text label accurately describes the subject matter of a threshold number of respective images to which the text label corresponds; identifying, as high confidence labels and by the image search apparatus, text labels for which each of the one or more confidence values meets at least one of a precision measurement threshold and a frequency measurement threshold; training, by the image search apparatus and using a set of training text labels and images corresponding to the training text labels, the image relevance model, wherein the trained image relevance model determines a relevance of an image to a text query, the set of training labels including only labels that have been identified as high confidence labels; identifying, using the trained image relevance model, one or more images to provide in response to a received text query received from a user device; and providing, in response to the received text query and to the user device, one or more search results that depict the one or more images. - View Dependent Claims (10, 11, 12, 13, 14, 15, 16)
-
-
17. A non-transitory computer storage medium encoded with a computer program for automatically training an image relevance model and using the image relevance model to provide image search results in response to queries, the method being, the program comprising instructions that when executed by an image search apparatus comprising a data processing apparatus cause the data processing apparatus to perform operations comprising:
-
receiving, for each image in a set of images, a corresponding set of text labels, each text label being determined to be indicative of subject matter of the image; for each text label, determining one or more confidence values, each confidence value being a measure of confidence that the text label accurately describes the subject matter of a threshold number of respective images to which the text label corresponds; identifying, as high confidence labels, text labels for which each of the one or more confidence values meets at least one of a precision measurement threshold and a frequency measurement threshold; training, using a set of training text labels and images corresponding to the training text labels, the image relevance model, wherein the trained image relevance model determines a relevance of an image to a text query, the set of training labels including only labels that have been identified as high confidence labels; identifying, using the trained image relevance model, one or more images to provide in response to a received text query received from a user device; and providing, in response to the received text query and to the user device, one or more search results that depict the one or more images. - View Dependent Claims (18, 19, 20, 21, 22, 23, 24)
-
Specification