Hybrid use of location sensor data and visual query to return local listings for visual query

• US 10,346,463 B2
• Filed: 06/15/2017
• Issued: 07/09/2019
• Est. Priority Date: 12/03/2009
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Petrou, David, Flynn, John, Adam, Hartwig, Neven, Hartmut
• Primary Examiner(s): Peng, Huawen A
• Application Number:
  US15/624,056
• Publication Number:
  US 20170286493A1
• Time in Patent Office: 754 Days
• CPC Class Codes: G06F 16/242 Query formulation
• CPC Class Codes: G06F 16/24575 using context
• CPC Class Codes: G06F 16/29 Geographical information databases
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: H04W 4/02 Services making use of location information
• CPC Class Codes: H04W 4/029 Location-based management or tracking services
• CPC Class Codes: H04W 64/00 Locating users or terminals or network equipment for network management purposes, e.g. mobility management

Abstract

A visual query is received from a client system, along with location information for the client system, and processed by a server system. The server system sends the visual query and the location information to a visual query search system, and receives from the visual query search system enhanced location information based on the visual query and the location information. The server system then sends a search query, including the enhanced location information, to a location-based search system. The search system receives and provides to the client one or more search results to the client system.

Claims

1. A computer-implemented method for providing search results in response to a visual query, comprising:

receiving an image query from a client system, the image query including an image;

identifying a location of the client system;

determining an accuracy value that specifies a level of accuracy at which the identified location matches an actual location of the client system;

identifying a plurality of search results based on the received image query and the identified location, each search result having an associated prominence value that specifies a relative importance of the search result and an associated closeness metric that specifies how close a location of an entity referenced by the search result is to the identified location of the client system;

determining whether the accuracy value satisfies a threshold level of accuracy;

in response to determining that the accuracy value satisfies the threshold level of accuracy;

for each of the plurality of search results, weighting the closeness metric for the search result using a first weighting factor and weighting the prominence value for the search result using a second weighting factor such that the closeness metric for the search result is given more weight than the prominence value of the search result;

selecting one or more of the plurality of search results based on the weighted prominence value and the weighted closeness metric for each search result of the plurality of search results; and

providing the one or more selected search results to the client system.

2. The method of claim 1, further comprising:

in response to determining that the accuracy value does not satisfy the threshold level of accuracy;

for each of the plurality of search results;

adjusting at least one of (i) the first weighting factor for the closeness metric or (ii) the second weighting factor for the prominence value, such that the closeness metric for the search result is given less weight than the prominence value for the search result in selecting the one or more search results.

3. The method of claim 1, wherein identifying the plurality of search results comprises:

for each particular search result of a set of search results;

identifying a location of a particular entity referenced by the particular search result; and

identifying a positional accuracy value that specifies a level of accuracy for the identified location of the particular entity; and

selecting the plurality of search results from the set of search results based on the positional accuracy value for each particular search result.

4. The method of claim 3, wherein selecting the plurality of search results from the set of search results comprises:

identifying a closeness metric for each particular search result, the closeness metric for a particular search result specifying how close the entity referenced by the particular search result is to the location of the client system; and

selecting, for inclusion in the plurality of search results, each particular search result that has a closeness metric that satisfies a closeness threshold and a positional accuracy that satisfies an accuracy threshold.

5. The method of claim 4, wherein selecting the plurality of search results from the set of search results comprises:

identifying, for inclusion in a proper subset of search results, each search result of the set of search results that has a positional accuracy that does not satisfy the accuracy threshold; and

selecting, for inclusion in the plurality of search results, each search result of the proper subset that references an entity that is located within a threshold distance of an entity referenced by a search result that has a closeness metric that satisfies the closeness threshold and a positional accuracy that satisfies the accuracy threshold.

6. The method of claim 1, wherein identifying the location of the client system comprises:

obtaining location information from the client system, the location information specifying a first location of the client system;

comparing the image of the image query with a plurality of images of entities located within a predefined range of the first location of the client;

determining an enhanced location of the client system using image location information specifying a location of an entity depicted in at least one of the plurality of image; and

designating the enhanced location of the client system as the location of the client system.

7. A system, comprising:

a data processing apparatus; and

a memory storage apparatus in data communication with the data processing apparatus, the memory storage apparatus storing instructions executable by the data processing apparatus and that upon such execution cause the data processing apparatus to perform operations comprising;

receiving an image query from a client system, the image query including an image;

identifying a location of the client system;

determining an accuracy value that specifies a level of accuracy at which the identified location matches an actual location of the client system;

in response to determining that the accuracy value satisfies the threshold level of accuracy;

for each of the plurality of search results, weighting the closeness metric for the search result using a first weighting factor and weighting the prominence value for the search result using a second weighting factor such that the closeness metric for the search result is given more weight than the prominence value of the search result;

selecting one or more of the plurality of search results based on the weighted prominence value and the weighted closeness metric for each search result of the plurality of search results; and

providing the one or more selected search results to the client system.

8. The system of claim 7, wherein the operations further comprise:

in response to determining that the accuracy value does not satisfy the threshold level of accuracy;

for each of the plurality of search results;

adjusting at least one of (i) the first weighting factor for the closeness metric or (ii) the second weighting factor for the prominence value, such that the closeness metric for the search result is given less weight than the prominence value for the search result in selecting the one or more search results.

9. The system of claim 7, wherein identifying the plurality of search results comprises:

for each particular search result of a set of search results;

identifying a location of a particular entity referenced by the particular search result; and

identifying a positional accuracy value that specifies a level of accuracy for the identified location of the particular entity; and

selecting the plurality of search results from the set of search results based on the positional accuracy value for each particular search result.

10. The system of claim 7, wherein selecting the plurality of search results from the set of search results comprises:

identifying a closeness metric for each particular search result, the closeness metric for a particular search result specifying how close the entity referenced by the particular search result is to the location of the client system; and

selecting, for inclusion in the plurality of search results, each particular search result that has a closeness metric that satisfies a closeness threshold and a positional accuracy that satisfies an accuracy threshold.

11. The system of claim 10, wherein selecting the plurality of search results from the set of search results comprises:

identifying, for inclusion in a proper subset of search results, each search result of the set of search results that has a positional accuracy that does not satisfy the accuracy threshold; and

selecting, for inclusion in the plurality of search results, each search result of the proper subset that references an entity that is located within a threshold distance of an entity referenced by a search result that has a closeness metric that satisfies the closeness threshold and a positional accuracy that satisfies the accuracy threshold.

12. The system of claim 7, wherein identifying the location of the client system comprises:

obtaining location information from the client system, the location information specifying a first location of the client system;

comparing the image of the image query with a plurality of images of entities located within a predefined range of the first location of the client;

determining an enhanced location of the client system using image location information specifying a location of an entity depicted in at least one of the plurality of image; and

designating the enhanced location of the client system as the location of the client system.

13. A non-transitory computer storage medium encoded with a computer program, the program comprising instructions that when executed by a data processing apparatus cause the data processing apparatus to perform operations comprising:

receiving an image query from a client system, the image query including an image;

identifying a location of the client system;

determining an accuracy value that specifies a level of accuracy at which the identified location matches an actual location of the client system;

identifying a plurality of search results based on the received image query and the identified location, each search result having an associated prominence value that specifies a relative importance of the search result and an associated closeness metric that specifies how close a location of an entity referenced by the search result is to the identified location of the client system;

determining whether the accuracy value satisfies a threshold level of accuracy;

in response to determining that the accuracy value satisfies the threshold level of accuracy;

for each of the plurality of search results, weighting the closeness metric for the search result using a first weighting factor and weighting the prominence value for the search result using a second weighting factor such that the closeness metric for the search result is given more weight than the prominence value of the search result;

selecting one or more of the plurality of search results based on the weighted prominence value and the weighted closeness metric for each search result of the plurality of search results; and

providing the one or more selected search results to the client system.

14. The computer storage medium of claim 13, wherein the operations further comprise:

in response to determining that the accuracy value does not satisfy the threshold level of accuracy;

for each of the plurality of search results;

adjusting at least one of (i) the first weighting factor for the closeness metric or (ii) the second weighting factor for the prominence value, such that the closeness metric for the search result is given less weight than the prominence value for the search result in selecting the one or more search results.

15. The computer storage medium of claim 13, wherein identifying the plurality of search results comprises:

for each particular search result of a set of search results;

identifying a location of a particular entity referenced by the particular search result; and

identifying a positional accuracy value that specifies a level of accuracy for the identified location of the particular entity; and

selecting the plurality of search results from the set of search results based on the positional accuracy value for each particular search result.

16. The computer storage medium of claim 15, wherein selecting the plurality of search results from the set of search results comprises:

identifying a closeness metric for each particular search result, the closeness metric for a particular search result specifying how close the entity referenced by the particular search result is to the location of the client system; and

selecting, for inclusion in the plurality of search results, each particular search result that has a closeness metric that satisfies a closeness threshold and a positional accuracy that satisfies an accuracy threshold.

17. The computer storage medium of claim 16, wherein selecting the plurality of search results from the set of search results comprises:

identifying, for inclusion in a proper subset of search results, each search result of the set of search results that has a positional accuracy that does not satisfy the accuracy threshold; and

selecting, for inclusion in the plurality of search results, each search result of the proper subset that references an entity that is located within a threshold distance of an entity referenced by a search result that has a closeness metric that satisfies the closeness threshold and a positional accuracy that satisfies the accuracy threshold.