Providing image search templates

• US 9,805,292 B2
• Filed: 03/30/2017
• Issued: 10/31/2017
• Est. Priority Date: 08/24/2012
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Chinen, Troy, Makadia, Ameesh, Cortes, Corinna, Adam, Hartwig, Petrovic, Nemanja, Ko, Teresa, Pueblas, Sebastian
• Primary Examiner(s): Liew, Alex
• Application Number:
  US15/474,356
• Publication Number:
  US 20170206439A1
• Time in Patent Office: 215 Days
• Field of Search: 382103, 382181-231
• CPC Class Codes: G06F 16/532 Query formulation, e.g. graphical querying
• CPC Class Codes: G06F 16/5854 using shape and object relationship
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06F 2218/16 by matching signal segments
• CPC Class Codes: G06V 10/751 Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
• CPC Class Codes: G06V 10/761 Proximity, similarity or dissimilarity measures
• CPC Class Codes: H04N 23/633 for displaying additional information relating to control or operation of the camera
• CPC Class Codes: H04N 23/64 Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image
• CPC Class Codes: H04N 23/69 Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming

Abstract

Techniques for providing image search templates are provided. An image search template may be associated with an image search query to aid the user in capturing an image that will be appropriate for processing the search query. The template may be displayed as an overlay during an image capturing process to indicate an appropriate image capturing pose, range, angle, or other view characteristics that may provide more accurate search results. The template may also be used in the image search query to segment the image and identify features relevant to the search query. Images in an image database may be clustered using characteristics of the images or metadata associated with the images in order to establish groups of images from which templates may be derived. The generated templates may be provided to users to assist in capturing images to be used as search engine queries.

Claims

1. A computer-implemented method, comprising:

receiving, by a mobile device that includes a camera and a display, data identifying a selected image acquisition template associated with a particular type of object, from among multiple image acquisition templates that are stored on the multiple device and that are each associated with a different type of object;

providing a pattern associated with the selected image acquisition template for output on the display;

generating, by the camera included on the mobile device, a query image while the pattern associated with the selected image acquisition template is provided for output on the display;

providing, to a search engine, an image search query that includes (i) the query image, and (ii) an indication of the selected image acquisition template; and

receiving, from the search engine, one or more image search results in response to the image search query.

2. The computer-implemented method of claim 1, wherein receiving the data identifying the selected image acquisition template is based on at least one of a selection by a user and automatic selection of the selected image acquisition template by the mobile device based on identifying a general category of the particular type of the object.

3. The computer-implemented method of claim 2, further comprising:

receiving, at the mobile device, a selection of a specific category of the particular type of the object in response to the client device identifying the general category of the particular type of the object.

4. The computer-implemented method of claim 1, wherein providing the pattern associated with the selected image acquisition template for the output on the display is provided to depict a capturing feature such that capturing an object in the query image yields optimal results in an image search operation.

5. The computer-implemented method of claim 1, wherein generating the query image while the pattern associated with the selected image acquisition template is provided for the output on the display comprises:

providing, at the mobile device, metadata associated with the selected image acquisition template in the generated query image to aid in an identification of the particular type of the object.

6. The computer-implemented method of claim 1, further comprising:

detecting, at the mobile device, an inappropriate alignment between a general shape of the particular type of the object and the pattern associated with the selected image acquisition template.

7. The computer-implemented method of claim 6, further comprising:

providing an indication of the inappropriate alignment between the particular type of the object and the pattern associated with the selected image acquisition template to the display of the mobile device.

8. A system comprising:

one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising;

receiving, by a mobile device that includes a camera and a display, data identifying a selected image acquisition template associated with a particular type of object, from among multiple image acquisition templates that are stored on the multiple device and that are each associated with a different type of object;

providing a pattern associated with the selected image acquisition template for output on the display;

generating, by the camera included on the mobile device, a query image while the pattern associated with the selected image acquisition template is provided for output on the display;

providing, to a search engine, an image search query that includes (i) the query image, and (ii) an indication of the selected image acquisition template; and

receiving, from the search engine, one or more image search results in response to the image search query.

9. The system of claim 8, wherein receiving the data identifying the selected image acquisition template is based on at least one of a selection by a user and automatic selection of the selected image acquisition template by the mobile device based on identifying a general category of the particular type of the object.

10. The system of claim 9, wherein the operations further comprise:

receiving, at the mobile device, a selection of a specific category of the particular type of the object in response to the client device identifying the general category of the particular type of the object.

11. The system of claim 8, wherein providing the pattern associated with the selected image acquisition template for the output on the display is provided to depict a capturing feature such that capturing an object in the query image yields optimal results in an image search operation.

12. The system of claim 8, wherein generating the query image while the pattern associated with the selected image acquisition template is provided for the output on the display the operations further comprise:

providing, at the mobile device, metadata associated with the selected image acquisition template in the generated query image to aid in an identification of the particular type of the object.

13. The system of claim 8, wherein the operations further comprise:

detecting, at the mobile device, an inappropriate alignment between a general shape of the particular type of the object and the pattern associated with the selected image acquisition template.

14. The system of claim 13, wherein the operation further comprise:

providing an indication of the inappropriate alignment between the particular type of the object and the pattern associated with the selected image acquisition template to the display of the mobile device.

15. A non-transitory computer-readable medium storing software comprising instructions executable by one or more computers which, upon such execution, cause the one or more computers to perform operations comprising:

receiving, by a mobile device that includes a camera and a display, data identifying a selected image acquisition template associated with a particular type of object, from among multiple image acquisition templates that are stored on the multiple device and that are each associated with a different type of object;

providing a pattern associated with the selected image acquisition template for output on the display;

generating, by the camera included on the mobile device, a query image while the pattern associated with the selected image acquisition template is provided for output on the display;

providing, to a search engine, an image search query that includes (i) the query image, and (ii) an indication of the selected image acquisition template; and

receiving, from the search engine, one or more image search results in response to the image search query.

16. The computer-readable medium of claim 15, wherein receiving the data identifying the selected image acquisition template is based on at least one of a selection by a user and automatic selection of the selected image acquisition template by the mobile device based on identifying a general category of the particular type of the object.

17. The computer-readable medium of claim 16, wherein the operations further comprise:

receiving, at the mobile device, a selection of a specific category of the particular type of the object in response to the client device identifying the general category of the particular type of the object.

18. The computer-readable medium of claim 15, wherein providing the pattern associated with the selected image acquisition template for the output on the display is provided to depict a capturing feature such that capturing an object in the query image yields optimal results in an image search operation.

19. The computer-readable medium of claim 15, wherein generating the query image while the pattern associated with the selected image acquisition template is provided for the output on the display the operations further comprise:

providing, at the mobile device, metadata associated with the selected image acquisition template in the generated query image to aid in an identification of the particular type of the object.

20. The computer-readable medium of claim 15, wherein the operations further comprise:

detecting, at the mobile device, an inappropriate alignment between a general shape of the particular type of the object and the pattern associated with the selected image acquisition template.