Region of Interest Selector for Visual Queries

• US 20110128288A1
• Filed: 08/09/2010
• Published: 06/02/2011
• Est. Priority Date: 12/02/2009
• Status: Abandoned Application

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Ting, Pin, Cohen, Zak, Petrou, David, Lee, Dar-Shyang
• Application Number:
  US12/853,188
• Publication Number:
  US 20110128288A1
• Time in Patent Office: Days
• US Class Current: 345/428
• CPC Class Codes: G06F 16/434 using image data, e.g. images, photos, pictures taken by a user
• CPC Class Codes: G06F 16/438 Presentation of query results
• CPC Class Codes: G06F 16/532 Query formulation, e.g. graphical querying
• CPC Class Codes: G06V 10/235 based on user input or interaction
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59

Abstract

A client system receives an image such as a photograph, a screen shot, a scanned image, or a video frame. The image has a first resolution which is likely larger than a maximum resolution for visual queries. As such, if a visual query were created from the image some resolution would be lost. Instead, a user selects a region of interest within the image. The region of interest has a second resolution, which is smaller than the first resolution. The client system then creates a visual query from the region of interest. The visual query has a resolution no larger than a pre-defined maximum resolution for visual queries. Because the visual query is created from the region of interest rather, than the entire received image, most of the resolution is concentrated specifically on the region of interest. The visual query is then sent to a server system.

Claims

1. A computer-implemented method of processing an visual query comprising:

at a client system having one or more processors, a display, and memory storing one or more programs for execution by the one or more processors;

receiving an image from a client application, the image having a first two-dimensional image resolution, the first two-dimensional image resolution having first and second components corresponding to first and second axes of the image;

displaying the image on the display;

receiving a selection of a region of interest within the image from a user, the region of interest having a second two-dimensional image resolution, the second two-dimensional image resolution having first and second components corresponding to the first and second axes of the region of interest;

creating a visual query from the region of interest, the visual query having a third two-dimensional image resolution, the third two-dimensional image resolution having first and second components corresponding to first and second axes of the visual query, such that the first and second components of the third two-dimensional image resolution are each no larger than corresponding components of a predefined maximum two-dimensional image resolution for visual queries, the predefined maximum two-dimensional image resolution having first and second components corresponding to the first and second axes of the visual query; and

sending the visual query to a server system.

2. The computer-implemented method of claim 1, wherein creating the visual query includes:

when the second two-dimensional image resolution has at least one component that is larger than a corresponding component of the predefined maximum two-dimensional image resolution for visual queries, producing a reduced resolution image corresponding to the region of interest of the image, the reduced resolution image having said third two-dimensional image resolution.

3. The computer-implemented method of claim 1, wherein creating the visual query includes:

when both components of the second two-dimensional image resolution are smaller than the corresponding components of the predefined maximum two-dimensional image resolution for visual queries, producing a maximum resolution image corresponding to the region of interest of the image, the maximum resolution image having said second two-dimensional image resolution.

4. The computer-implemented method of claim 1, wherein the client system comprises a touch sensitive display, and the receiving a selection comprises receiving a touch by the user on the region of interest on the touch sensitive display.

5. The computer-implemented method of claim 4, wherein receiving the selection comprises receiving a selection gesture comprising a line drawn across the region of interest on the touch sensitive display.

6. The computer-implemented method of claim 4, wherein the sending is initiated when the user ceases touching the region of interest.

7. The computer-implemented method of claim 1, wherein the client system comprises a camera, the received image comprises a camera preview image, and the creating a visual query includes taking a picture with the camera.

8. The computer-implemented method of claim 7, wherein during the receiving a selection of a region of interest, the camera focuses on one or more subjects in the region of interest.

9. The computer-implemented method of claim 8, wherein when the region of interest includes two or more subjects, the camera focuses on a most important subject.

10. The computer-implemented method of claim 1, further comprising displaying the image such that the region of interest is visually distinguished from a portion of image not including the region of interest.

11. The computer-implemented method of claim 10, wherein the region of interest is visually distinguished by means of at least one of the set consisting of:

transparency, shading, color, background pattern, and border.

12. The computer-implemented method of claim 1, further comprising:

receiving visual query results from the visual query server system corresponding to the region of interest; and

displaying the visual query results concurrently with only the region of interest in a results display region of the display.

13. The computer-implemented method of claim 12, further comprising:

receiving a selection of a sub-region of interest having a fourth two-dimensional image resolution, the fourth two-dimensional image resolution having first and second components corresponding to first and second axes of the sub-region of interest;

creating a new visual query from the sub-region of interest, the new visual query having a fifth two-dimensional image resolution, the fifth two-dimensional image resolution having first and second components corresponding to first and second axes of the new visual query, such that the first and second components of the fifth two-dimensional image resolution are each no larger than corresponding components of the predefined maximum two-dimensional image resolution for visual queries; and

sending the new visual query to the server system.

14. The computer-implemented method of claim 1, further comprising:

receiving an interactive results document from the server system, the interactive results document comprising one or more visual identifiers for respective sub-portions of the region of interest, wherein each visual identifier includes at least one user selectable link to at least one search result corresponding to a recognized entity in the region of interest; and

displaying the interactive results document.

15. A client system, for processing a visual query, comprising:

one or more central processing units for executing programs;

a display; and

memory storing one or more programs be executed by the one or more central processing units;

the one or more programs comprising instructions for;

receiving an image from a client application, the image having a first two-dimensional image resolution, the first two-dimensional image resolution having first and second components corresponding to first and second axes of the image;

displaying the image on the display;

receiving a selection of a region of interest within the image from a user, the region of interest having a second two-dimensional image resolution, the second two-dimensional image resolution having first and second components corresponding to the first and second axes of the region of interest;

creating a visual query from the region of interest, the visual query having a third two-dimensional image resolution, the third two-dimensional image resolution having first and second components corresponding to first and second axes of the visual query, such that the first and second components of the third two-dimensional image resolution are each no larger than corresponding components of a predefined maximum two-dimensional image resolution for visual queries, the predefined maximum two-dimensional image resolution having first and second components corresponding to the first and second axes of the visual query; and

sending the visual query to a server system.

16. The client system of claim 15, wherein creating the visual query includes:

when the second two-dimensional image resolution has at least one component that is larger than a corresponding component of the predefined maximum two-dimensional image resolution for visual queries, producing a reduced resolution image corresponding to the region of interest of the image, the reduced resolution image having said third two-dimensional image resolution.

17. The client system of claim 15, further comprising a touch sensitive display, and wherein instructions for the receiving a selection comprises instructions for receiving a touch by the user on the region of interest on the touch sensitive display.

18. The client system of claim 15, further comprising a camera, wherein the received image comprises a camera preview image and the instructions for creating a visual query include instructions for taking a picture with the camera.

19. The client system of claim 18, further comprising instructions for focusing one or more subjects in the region of interest while receiving a selection of a region of interest.

20. A non-transitory computer readable storage medium storing one or more programs configured for execution by a computer, the one or more programs comprising instructions for:

receiving an image from a client application, the image having a first two-dimensional image resolution, the first two-dimensional image resolution having first and second components corresponding to first and second axes of the image;

displaying the image;

receiving a selection of a region of interest within the image from a user, the region of interest having a second two-dimensional image resolution, the second two-dimensional image resolution having first and second components corresponding to the first and second axes of the region of interest;

creating a visual query from the region of interest, the visual query having a third two-dimensional image resolution, the third two-dimensional image resolution having first and second components corresponding to first and second axes of the visual query, such that the first and second components of the third two-dimensional image resolution are each no larger than corresponding components of a predefined maximum two-dimensional image resolution for visual queries, the predefined maximum two-dimensional image resolution having first and second components corresponding to the first and second axes of the visual query; and

sending the visual query to a server system.

21. The computer readable storage medium of claim 20, wherein creating the visual query includes:

when the second two-dimensional image resolution has at least one component that is larger than a corresponding component of the predefined maximum two-dimensional image resolution for visual queries, producing a reduced resolution image corresponding to the region of interest of the image, the reduced resolution image having said third two-dimensional image resolution.