Co-selected image classification

• US 9,594,826 B2
• Filed: 06/30/2014
• Issued: 03/14/2017
• Est. Priority Date: 06/03/2009
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Jing, Yushi, Yeo, Boon-Lock, Zhang, Yong, Li, Guang Hua, Li, Gangjiang, Li, Hui
• Primary Examiner(s): Reyes, Mariela
• Assistant Examiner(s): VU, THONG H
• Application Number:
  US14/319,888
• Publication Number:
  US 20150012547A1
• Time in Patent Office: 988 Days
• Field of Search: 707/749, 707/737, 707/769, 707/727, 707/728, 382/203, 382/224, 382/190, 382/155, 382/305
• US Class Current: 1/1
• CPC Class Codes: G06F 16/14 Details of searching files based on file metadata
• CPC Class Codes: G06F 16/313 Selection or weighting of terms for indexing
• CPC Class Codes: G06F 16/58 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/9538 Presentation of query results
• CPC Class Codes: G06F 18/41 Interactive pattern learning with a human teacher

Abstract

Co-selected images are labeled based on a topic score that is a measure of relevance of the co-selected image to a first topic to which a reference image belongs. The first topic to which the reference image belongs is identified based on a reference label associated with the reference image. The co-selected images are images that are selected for presentation subsequent to selection of the reference image during a user session. The co-selected images are identified based on selection data for user sessions in which the reference image was selected for presentation. The topic score is generated based on a frequency of selection of the co-selected image. Image search results for a second topic can be filtered to remove images that are labeled as belonging to the first topic or the image search results can be reordered to adjust the presentation positions at which images are referenced based on the topic to which the images belong.

Claims

1. A computer-implemented method, comprising:

accessing a reference text label that specifies a first topic of a reference image;

determining, in a data processing system, that a user interacted with an image presented in image search results as responsive to a particular query and during a same user session also interacted with the reference image also presented in the image search results as also responsive to the particular query;

generating, in the data processing system, a measure of relevance of the image to the first topic based on a number of user sessions in which image search results included both the image and the reference image as responsive to a query and in which a user interacted with both of the reference image and the image; and

labeling the image as an image that is relevant to the topic based on the measure of relevance meeting a specified threshold.

2. The method of claim 1, further comprising:

identifying, for a given user session, a first time of a user interaction with the reference image;

identifying subsequent times, during the given user session, of user interactions with other images; and

identifying, as co-selected images, only the other images having subsequent times of user interaction that are within a specified time of the first time, wherein the given user session has a session duration and the specified time is less than the session duration.

3. The method of claim 2, further comprising:

representing, in the data processing system, the reference image and co-selected images as nodes in a weighted graph, each node being connected to other nodes by weighted edges representing sequential selections of the images represented by the node and the other nodes; and

wherein the measure of relevance for each co-selected image is generated based on the weighted edges.

4. The method of claim 3, wherein a weighted edge connecting a pair of nodes has a weight that is indicative of a number of sequential selections of images represented by the pair of nodes.

5. The method of claim 4, wherein each weighted edge has directional components indicating a source image and a destination image, the source image being an image that was selected for presentation prior to the destination image being selected for presentation.

6. The method of claim 1, further comprising:

receiving image search results responsive to a query that is classified as belonging to a second topic that is different than the first topic; and

filtering the image search results to remove references to images having labels that indicate that the images belong to the first topic.

7. The method of claim 1, further comprising:

receiving image search results responsive to a query that is classified as belonging to a second topic that is different than the first topic; and

reordering the image search results when the image search results include references to images having labels that indicate that the images belong to the first topic.

8. A system, comprising:

a data store storing a reference text label that specifies a first topic of a reference image and user session data specifying, for user sessions, user interactions with images presented in image search results; and

an image classification apparatus that interacts with the data store, the image classification apparatus comprising at least one processor that executes instructions that cause the image classification apparatus to perform operations comprising;

accessing the reference text label that specifies the first topic of the reference image;

determining that a user interacted with an image presented in image search results as responsive to a particular query and during a same user session also interacted with the reference image also presented in the image search results as also responsive to the particular query;

generating a measure of relevance of the image to the first topic based on a number of user sessions in which image search results included both the image and the reference image as responsive to a query and in which a user interacted with both of the reference image and the image; and

labeling the image as an image that is relevant to the topic based on the measure of relevance meeting a specified threshold.

9. The system of claim 8, wherein the instructions cause the image classification apparatus to perform operations further comprising:

identifying, for a given user session, a first time of a user interaction with the reference image;

identifying subsequent times, during the given user session, of user interactions with other images; and

identifying, as co-selected images, only the other images having subsequent times of user interaction that are within a specified time of the first time, wherein the given user session has a session duration and the specified time is less than the session duration.

10. The system of claim 9, wherein the instructions cause the image classification apparatus to perform operations further comprising:

representing the reference image and co-selected images as nodes in a weighted graph, each node being connected to other nodes by weighted edges representing sequential selections of the images represented by the node and the other nodes; and

wherein the measure of relevance for each co-selected image is generated based on the weighted edges.

11. The system of claim 9, wherein a weighted edge connecting a pair of nodes has a weight that is indicative of a number of sequential selections of images represented by the pair of nodes.

12. The system of claim 11, wherein each weighted edge has directional components indicating a source image and a destination image, the source image being an image that was selected for presentation prior to the destination image being selected for presentation.

13. The system of claim 8, wherein the instructions cause the image classification apparatus to perform operations further comprising:

receiving image search results responsive to a query that is classified as belonging to a second topic that is different than the first topic; and

filtering the image search results to remove references to images having labels that indicate that the images belong to the first topic.

14. The system of claim 8, wherein the instructions cause the image classification apparatus to perform operations further comprising:

receiving image search results responsive to a query that is classified as belonging to a second topic that is different than the first topic; and

reordering the image search results when the image search results include references to images having labels that indicate that the images belong to the first topic.

15. A non-transitory computer readable medium encoded with a computer program comprising instructions that when executed cause one or more data processing apparatus to perform operations comprising:

accessing a reference text label that specifies a first topic of a reference image;

determining that a user interacted with an image presented in image search results as responsive to a particular query and during a same user session also interacted with the reference image also presented in the image search results as also responsive to the particular query;

generating a measure of relevance of the image to the first topic based on a number of user sessions in which image search results included both the image and the reference image as responsive to a query and in which a user interacted with both of the reference image and the image; and

labeling the image as an image that is relevant to the topic based on the measure of relevance meeting a specified threshold.

16. The computer readable medium of claim 15, wherein the instructions cause the one or more data processing apparatus to perform operations further comprising:

identifying, for a given user session, a first time of a user interaction with the reference image;

identifying subsequent times, during the given user session, of user interactions with other images; and

identifying, as co-selected images, only the other images having subsequent times of user interaction that are within a specified time of the first time, wherein the given user session has a session duration and the specified time is less than the session duration.

17. The computer readable medium of claim 16, wherein the instructions cause the one or more data processing apparatus to perform operations further comprising:

representing the reference image and co-selected images as nodes in a weighted graph, each node being connected to other nodes by weighted edges representing sequential selections of the images represented by the node and the other nodes; and

wherein the measure of relevance for each co-selected image is generated based on the weighted edges.

18. The computer readable medium of claim 17, wherein a weighted edge connecting a pair of nodes has a weight that is indicative of a number of sequential selections of images represented by the pair of nodes.

19. The computer readable medium of claim 18, wherein each weighted edge has directional components indicating a source image and a destination image, the source image being an image that was selected for presentation prior to the destination image being selected for presentation.

20. The computer readable medium of claim 15, wherein the instructions cause the one or more data processing apparatus to perform operations further comprising:

receiving image search results responsive to a query that is classified as belonging to a second topic that is different than the first topic; and

filtering the image search results to remove references to images having labels that indicate that the images belong to the first topic.