PRESENTING IMAGE SEARCH RESULTS

• US 20150169645A1
• Filed: 03/08/2013
• Published: 06/18/2015
• Est. Priority Date: 12/06/2012
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Li, Yuan, Lee, Taehee, Adam, Hartwig
• Granted Patent: US 9,424,279 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06F 16/248 Presentation of query results
• CPC Class Codes: G06F 16/532 Query formulation, e.g. graphical querying
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06F 16/5846 using extracted text
• CPC Class Codes: G06F 16/5854 using shape and object relationship
• CPC Class Codes: G06F 16/5866 using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
• CPC Class Codes: G06F 16/9537 Spatial or temporal dependent retrieval, e.g. spatiotemporal queries

Abstract

A system and computer-implemented method is provided for organizing multiple user submitted results responsive to an image query. A plurality of content submissions may be received from a variety of submitting users, each content submission including an image and an associated label. An image query may provide an image of an object as a request to identify the object. In response to receiving the image query, one or more results of the plurality of content submissions may be identified. A similarity between the labels for each of the one or more results may be determined and used to group the one or more results. Grouped results may be ranked and sorted for accurate and concise presentation to a querying user.

Claims

1-20. -20. (canceled)

21. A computer-implemented method comprising:

obtaining, from each of one or more users, one or more submitted images and, for each submitted image, an annotation for the image;

receiving, from a particular user, a query image;

obtaining a set of the submitted images that are identified as similar to the query image;

selecting a representative annotation from among the annotations associated with the set of submitted images that are identified as similar to the query image; and

providing the representative annotation for output in response to receiving the query image.

22. The method of claim 21, wherein selecting a representative annotation from among the annotations associated with the set of submitted images that are identified as similar to the query image comprises:

grouping the submitted images of the obtained set into groups with similar annotations; and

selecting the representative annotation based on the groups.

23. The method of claim 21, comprising, for each of the submitted images of the set, generating a score for the particular submitted image that represents a similarity between the particular submitted image and the query image,

wherein selecting a representative annotation from among the annotations associated with the set of submitted images that are identified as similar to the query image is based on the scores of the submitted images of the set.

24. The method of claim 23, wherein selecting a representative annotation from among the annotations associated with the set of submitted images that are identified as similar to the query image comprises:

grouping the submitted images of the obtained set into groups with similar annotations;

for each group, aggregating the scores of the submitted images in the group; and

selecting a group to select the representative annotation from based on the aggregated scores of the groups.

25. The method of claim 24, wherein grouping the submitted images of the obtained set into groups with similar annotations is based on determining edit distances or similar meanings between the annotations.

26. The method of claim 24, wherein providing the representative annotation for output in

response to receiving the query image comprises:

providing for display a list of representative annotations from two or more of the groups ordered by the aggregated scores.

27. The method of claim 21, wherein obtaining a set of the submitted images that are identified as similar to the query image comprises:

obtaining at most one submitted image from each of the one or more users.

28. The method of claim 21, wherein obtaining a set of the submitted images that are identified as similar to the query image is based on geographical locations where the one or more users submitted the submitted image.

29. A system comprising:

one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising;

receiving, from each of one or more image users, one or more submitted images and, for each submitted image, an annotation for the image;

receiving, from a particular user, a query image;

obtaining a set of the submitted images that are identified as similar to the query image;

selecting a representative annotation from among the annotations associated with the set of submitted images that are identified as similar to the query image; and

providing the representative annotation for output in response to receiving the query image.

30. The system of claim 29, wherein selecting a representative annotation from among the annotations associated with the set of submitted images that are identified as similar to the query image comprises:

grouping the submitted images of the obtained set into groups with similar annotations; and

selecting the representative annotation based on the groups.

31. The system of claim 29, the operations comprising, for each of the submitted images of the set, generating a score for the particular submitted image that represents a similarity between the particular submitted image and the query image,

wherein selecting a representative annotation from among the annotations associated with the set of submitted images that are identified as similar to the query image is based on the scores of the submitted images of the set.

32. The system of claim 31, wherein selecting a representative annotation from among the annotations associated with the set of submitted images that are identified as similar to the query image comprises:

grouping the submitted images of the obtained set into groups with similar annotations;

for each group, aggregating the scores of the submitted images in the group; and

selecting a group to select the representative annotation from based on the aggregated scores of the groups.

33. The system of claim 32, wherein grouping the submitted images of the obtained set into groups with similar annotations is based on determining edit distances or similar meanings between the annotations.

34. The system of claim 32, wherein providing the representative annotation for output in

response to receiving the query image comprises:

providing for display a list of representative annotations from two or more of the groups ordered by the aggregated scores.

35. A non-transitory computer-readable medium storing software comprising instructions executable by one or more computers which, upon such execution, cause the one or more computers to perform operations comprising:

receiving, from each of one or more users, one or more submitted images and, for each submitted image, an annotation for the image;

receiving, from a particular user, a query image;

obtaining a set of the submitted images that are identified as similar to the query image;

selecting a representative annotation from among the annotations associated with the set of submitted images that are identified as similar to the query image; and

providing the representative annotation for output in response to receiving the query image.

36. The medium of claim 35, wherein selecting a representative annotation from among the annotations associated with the set of submitted images that are identified as similar to the query image comprises:

grouping the submitted images of the obtained set into groups with similar annotations; and

selecting the representative annotation based on the groups.

37. The medium of claim 35, the operations comprising, for each of the submitted images of the set, generating a score for the particular submitted image that represents a similarity between the particular submitted image and the query image,

wherein selecting a representative annotation from among the annotations associated with the set of submitted images that are identified as similar to the query image is based on the scores of the submitted images of the set.

38. The medium of claim 37, wherein selecting a representative annotation from among the annotations associated with the set of submitted images that are identified as similar to the query image comprises:

grouping the submitted images of the obtained set into groups with similar annotations;

for each group, aggregating the scores of the submitted images in the group; and

selecting a group to select the representative annotation from based on the aggregated scores of the groups.

39. The medium of claim 38, wherein grouping the submitted images of the obtained set into groups with similar annotations is based on determining edit distances or similar meanings between the annotations.

40. The medium of claim 38, wherein providing the representative annotation for output in response to receiving the query image comprises:

providing for display a list of representative annotations from two or more of the groups ordered by the aggregated scores.