Using senses of a query to rank images associated with the query

• US 8,923,655 B1
• Filed: 10/12/2012
• Issued: 12/30/2014
• Est. Priority Date: 10/14/2011
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Weston, Jason, Bengio, Samy
• Primary Examiner(s): Bayat, Ali
• Application Number:
  US13/650,489
• Time in Patent Office: 809 Days
• Field of Search: 382/115, 382/128, 382/159, 382/170, 382/203, 382/181, 382/305, 707/821, 707/831, 358/403, 358/404
• US Class Current: 382/305
• CPC Class Codes: G06F 16/54 Browsing; Visualisation therefor

Abstract

A server device determines a plurality of images for a query. One or more images, of the plurality of images, are associated with one or more senses of the query. The server device maps the plurality of images into a space by representing the plurality of images with corresponding points in the space; determines one or more hyperplanes in the space based on the corresponding points in the space; calculates one or more scores for the plurality of images based on the corresponding points and the one or more hyperplanes; and ranks the one or more images based on the one or more scores.

Claims

1. A method comprising:

determining, by one or more server devices, a plurality of images for a query,

one or more images, of the plurality of images, being associated with one or more senses of the query;

mapping, by the one or more server devices, the plurality of images into a space by representing the plurality of images with corresponding points in the space;

determining, by the one or more server devices, one or more hyperplanes in the space based on the corresponding points in the space,

a first hyperplane, of the one or more hyperplanes, being associated with a first sense of the one or more senses, and

a second hyperplane, of the one or more hyperplanes, being associated with a second sense of the one or more senses;

calculating, by the one or more server devices, one or more scores for the plurality of images based on the corresponding points and the one or more hyperplanes; and

ranking the one or more images based on the one or more scores.

2. The method of claim 1, further comprising:

receiving the query; and

providing a search result document for the query based on the ranking of the one or more images.

3. The method of claim 2, where the search result document comprises:

a first group of images, of the one or more images, that are associated with the first sense, and

a second group of images, of the one or more images, that are associated with the second sense.

4. The method of claim 2, where the search result document does not include one of the plurality of images that is not associated with at least one of the one or more senses.

5. The method of claim 1, where determining the plurality of images for the query includes:

identifying the plurality of images for the query based on at least one of click-through information associated with the plurality of images or terms used to label the plurality of images.

6. The method of claim 1, where mapping the plurality of images into the space includes:

placing one or more first points, of the corresponding points, that are associated with the first sense into a first area of the space,

placing one or more second points, of the corresponding points, that are associated with the second sense into a second area of the space,

the second area being different from the first area, and

placing one or more third points, of the corresponding points, that are not associated with the first sense or the second sense into a third area of the space,

the third area being different from the first area and the second area.

7. The method of claim 6,

where the one or more first points are located on a first side of the first hyperplane,

where the one or more second points are located on a first side of the second hyperplane, and

where the one or more third points are located on a second side of the first hyperplane and located on a second side of the second hyperplane.

8. The method of claim 1, where calculating the one or more scores includes:

determining a first score based on a first distance, in the space, between a particular point, of the corresponding points, and the first hyperplane;

determining a second score based a second distance, in the space, between the particular point and the second hyperplane; and

determining a particular score, of the one or more scores, based on the first score and the second score,

the particular score being for a particular image, of the plurality of images, that corresponds to the particular point.

9. The method of claim 8,

where the particular score is equal to the first score when the first score is greater than the second score, and

where the particular score is equal to the second score when the second score is greater than the first score.

10. The method of claim 8,

where determining the particular score includes:

determining a first weighted score based on a first weight and the first score,

determining a second weighted score based on a second weight and the second score, and

determining the particular score based on the first weighted score and the second weighted score,

where the first weight is based on a first click-through rate for a first portion of the one or more images associated with the first sense, and

where the second weight is based on a second click-through rate for a second portion of the one or more images associated with the second sense.

11. A system comprising:

one or more processors to;

determine a plurality of images for a query,

one or more images, of the plurality of images, being associated with two or more senses of the query;

place a plurality points in a space to represent the plurality of images;

determine a first hyperplane in the space based on one or more first points, of the plurality of points, that are associated with a first sense of the two or more senses;

determine a second hyperplane in the space based on one or more second points, of the plurality of points, that are associated with a second sense of the two or more senses;

calculate a plurality of scores for the plurality of images based on the plurality of points, the first hyperplane, and the second hyperplane; and

rank the plurality of images based on the plurality of scores.

12. The system of claim 11, where the one or more processors are further to:

identify the query from a client device,

retrieve, based on the query and the ranking of the plurality of images, a first set of images, of the one or more images, that are associated with the first sense and a second set of images, of the one or more images, that are associated with the second sense, and

provide the first set of images and the second set of images to the client device.

13. The system of claim 11, where, when calculating the plurality of scores, the one or more processors are to:

determine a first score based on a first distance, in the space, between a particular point, of the plurality of points, and the first hyperplane,

determine a second score based a second distance, in the space, between the particular point and the second hyperplane, and

determine a particular score, of the plurality of scores, based on the first score and the second score,

the particular score being for a particular image, of the plurality of images, that corresponds to the particular point.

14. The system of claim 13, where, when determining the particular score, the one or more processors are to:

determine a first weighted score based on the first score and a first weight that is based on a first click-through rate associated with the first sense,

determine a second weighted score based on the second score and a second weight that is based on a second click-through rate associated with the second sense, and

determine the particular score based on the first weighted score and the second weighted score.

15. The system of claim 13, where, when determining the particular score, the one or more processors are to:

determine that the first score is greater than the second score,

determine that the particular image is associated with the first sense based on the first score being greater than the second score, and

determine that the particular score is equal to the first score.

16. A non-transitory computer-readable medium storing instructions, the instructions comprising:

one or more instructions that, when executed by at least one device, cause the at least one device to;

identify a plurality of images associated with a query,

one or more images, of the plurality of images, being associated with two or more senses of the query;

place, into a space, a plurality of points that correspond to the plurality of images;

determine a first hyperplane in the space based on one or more first points, of the plurality of points, that are associated with a first sense of the two or more senses;

determine a second hyperplane in the space based on one or more second points, of the plurality of points, that are associated with a second sense of the two or more senses;

determine a particular score for a particular image, of the plurality of images, based on a first distance from a particular point, of the plurality of points, to the first hyperplane and a second distance from the particular point to the second hyperplane,

the particular point corresponding to the particular image; and

rank the particular image, relative to the plurality of images, based on the particular score.

17. The non-transitory computer-readable medium of claim 16, where the one or more instructions to determine the particular score for the particular image include:

one or more instructions that, when executed by the at least one device, cause the at least one device to;

determine a first score based on the first distance,

determine a second score based on the second distance,

determine that the first score is greater than the second score,

determine that the particular image is associated with the first sense based on the first score being greater than the second score, and

determine that the particular score is equal to the first score after determining that the first score is greater than the second score.

18. The non-transitory computer-readable medium of claim 16, where the one or more instructions to place the plurality of points include:

one or more instructions that, when executed by the at least one device, cause the at least one device to;

place one or more first points, of the plurality of points, that are associated with the first sense into a first area of the space,

place one or more second points, of the plurality of points, that are associated with the second sense into a second area of the space,

the second area being different from the first area, and

place one or more third points, of the plurality of points, that are not associated with the first sense or the second sense into a third area of the space,

the third area being different from the first area and the second area.

19. The non-transitory computer-readable medium of claim 16, where the instructions further comprise:

one or more instructions that, when executed by the at least one device, cause the at least one device to;

receive the query from a client device,

identify a first quantity of images, of the one or more images, that are associated with the first sense,

identify a second quantity of images, of the one or more images, that are associated with the second sense, and

provide, to the client device, results based on the first quantity of images and the second quantity of images.

20. The non-transitory computer-readable medium of claim 16,

where the one or more instructions to rank the images include:

one or more instructions that, when executed by the at least one device, cause the at least one device to;

determine that other images, of the plurality of images, are not associated with the two or more senses of the query, and

determine a ranking of only the one or more images that are associated with the two or more senses of the query, and

where the one or more instructions further include;

one or more instructions that, when executed by the at least one device, cause the at least one device to;

receive the query from a client device, and

provide, based on the query and the ranking, results that include only images of the one or more images.