Actionable Search Results for Street View Visual Queries

• US 20110131235A1
• Filed: 08/10/2010
• Published: 06/02/2011
• Est. Priority Date: 12/02/2009
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Flynn, John, Petrou, David, Power, Theodore
• Granted Patent: US 9,405,772 B2
• Time in Patent Office: Days
• US Class Current: 707/769
• CPC Class Codes: G06F 16/532 Query formulation, e.g. graphical querying
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content

Abstract

A server system receives a visual query and information about the current location of the querying device. The server system sends the query and location information to a visual query search system and receives one or more search results in accordance with the visual query and the location information. From the one or more search results, the server system identifies an entity in the visual query. The server system further identifies a client-side action corresponding to the identified entity and creates an actionable search result element configured to launch the client-side action. For example, if the identified entity were a restaurant the actionable search result elements could include buttons to initiate a telephone call, to read reviews, to map the address, or to make a reservation. If the actionable search result is selected, the client system launches the client-side action in an application distinct from the visual query client application.

Claims

1. A computer-implemented method of processing a visual query comprising:

at a server system having one or more processors and memory storing one or more programs for execution by the one or more processors;

receiving a visual query from a client system;

receiving location information from the client system;

sending the visual query and the location information to a visual query search system;

receiving one or more search results in accordance with both the visual query and the location information from the visual query search system;

identifying, from the one or more search results, an entity in the visual query;

identifying one or more client-side actions corresponding to the identified entity;

creating an actionable search result element configured to launch a respective client-side action of the identified one or more client-side actions; and

sending the actionable search result element to the client system.

2. The method of claim 1, wherein the location information is obtained from one or more of the following:

GPS information, cell tower information, and local wireless network information.

3. The method of claim 1, further comprising:

sending, along with the actionable search result element, at least one of the one or more search results to the client system.

4. The method of claim 3, wherein the search results include search results within a specified distance from the location information.

5. The method of claim 3, wherein the search results include search results similar to the identified entity.

6. The method of claim 3, wherein at least one of the one or more search results includes an actionable search results element configured to launch a client-side action corresponding to an entity in the search result.

7. The method of claim 1, wherein the identified entity is a restaurant and a respective client-side action of the one or more client-side actions is selected from the group consisting of:

initiating a phone call, providing a review;

initiating a reservation request, providing mapping information, launching the restaurant'"'"'s website, providing additional information, and sharing any of the above.

8. The method of claim 1, wherein receiving one or more search results includes:

receiving from the visual query search system enhanced location information based on the visual query and the location information;

sending a search query to a location-based search system, the search query including the enhanced location information; and

receiving the one or more search results in accordance with the enhanced location information.

9. The method of claim 1, wherein a respective client-side action of the one or more client-side actions is selected from the group consisting of:

initiating a call to a telephone number, instant messaging, paging, faxing, emailing, communicating via a social network, and communicating by another communication mechanism.

10. The method of claim 1, wherein the identified entity, is selected from the group consisting of:

a person, a name or other identifier associated with the person, a bar code, a logo, a business, an organization, a building, a group of buildings or physical structures, a postal address, a landmark, a geographical entity, a product, and a service.

11. The method of claim 1, wherein the sending includes sending to the client system a representation of the visual query with the actionable search result element overlaying at least a portion of the representation of the visual query.

12. The method of claim 1, wherein the sending includes sending to the client system information for visually presenting the actionable search result element overlaying at least a portion of the visual query.

13. The method of claim 1, wherein the identified entity is a postal address and the actionable search result element is a button for mapping the address, wherein the mapping includes at least one of:

providing a map identifying the location of the postal address, providing an aerial photograph including the postal address, and providing a street view image corresponding to the postal address.

14. The method of claim 1, wherein the actionable search result element is configured to add information to a contacts list, wherein the information is selected from a group consisting of one or more of:

an email address, a phone number, a fax number, a postal address, a company name, an organization name, and a URL.

15. The method of claim 1, wherein the identified entity is an identifier associated with an entity, the one or more search results includes a communication address associated with the entity, and the actionable search result element is configured to launch a communication using the communication address.

16. A computer-implemented method of processing a visual query comprising:

at a client system having one or more processors, a display, and memory storing one or more programs for execution by the one or more processors;

receiving an image;

receiving location information;

creating a visual query from the image;

sending the visual query and the location information to a visual query search system;

receiving from the visual query search system an actionable search result element configured to launch a client-side action, wherein the actionable search result element corresponds to an entity in the visual query;

displaying the actionable search result element on the display using a visual query client application;

receiving a user selection of the actionable search result element; and

in a client-side application distinct from the visual query client application, launching the client-side action corresponding to the selected actionable search result element.

17. The method of claim 16, further comprising:

receiving from the visual query search system one or more search results in accordance with both the visual query and the location information; and

displaying on the display, along with the actionable search result element, the one or more search results.

18. The method of claim 16, wherein the client-side application distinct from the visual query client application is selected from the group consisting of:

an email application, a browser application;

a phone application;

an instant messaging application;

a social networking application, and a mapping application.

19. A server system, for processing a visual query, comprising:

one or more central processing units for executing programs;

memory storing one or more programs be executed by the one or more central processing units;

the one or more programs comprising instructions for;

receiving a visual query from a client system;

receiving location information from the client system;

sending the visual query and the location information to a visual query search system;

receiving one or more search results in accordance with both the visual query and the location information from the visual query search system;

identifying, from the one or more search results, an entity in the visual query;

identifying one or more client-side actions corresponding to the identified entity;

creating an actionable search result element configured to launch a respective client-side action of the identified one or more client-side actions; and

sending the actionable search result element to the client system.

20. A client system, for processing a visual query, comprising:

one or more central processing units for executing programs;

a display; and

memory storing one or more programs be executed by the one or more central processing units;

the one or more programs comprising instructions for;

receiving an image;

receiving location information;

creating a visual query from the image;

sending the visual query and the location information to a visual query search system;

receiving from the visual query search system an actionable search result element configured to launch a client-side action, wherein the actionable search result element corresponds to an entity in the visual query;

displaying the actionable search result element on the display using a visual query client application;

receiving a user selection of the actionable search result element; and

in a client-side application distinct from the visual query client application, launching the client-side action corresponding to the selected actionable search result element.

21. A non-transitory computer readable storage medium storing one or more programs configured for execution by a computer, the one or more programs comprising instructions for:

receiving a visual query from a client system;

receiving location information from the client system;

sending the visual query and the location information to a visual query search system;

receiving one or more search results in accordance with both the visual query and the location information from the visual query search system;

identifying, from the one or more search results, an entity in the visual query;

identifying one or more client-side actions corresponding to the identified entity;

creating an actionable search result element configured to launch a respective client-side action of the identified one or more client-side actions; and

sending the actionable search result element to the client system.

22. A non-transitory computer readable storage medium storing one or more programs configured for execution by a computer, the one or more programs comprising instructions for:

receiving an image;

receiving location information;

creating a visual query from the image;

sending the visual query and the location information to a visual query search system;

receiving from the visual query search system an actionable search result element configured to launch a client-side action, wherein the actionable search result element corresponds to an entity in the visual query;

displaying the actionable search result element on a display using a visual query client application;

receiving a user selection of the actionable search result element; and

in a client-side application distinct from the visual query client application, launching the client-side action corresponding to the selected actionable search result element.