Using multiple image verification challenges to determine object location in an image

• US 10,026,180 B1
• Filed: 08/19/2016
• Issued: 07/17/2018
• Est. Priority Date: 08/19/2016
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Gu, Jiexing, Liu, Ying, Malenfant, Aaron
• Primary Examiner(s): Patel, Nirav G
• Application Number:
  US15/241,951
• Time in Patent Office: 697 Days
• Field of Search: None
• CPC Class Codes: G06F 21/36 by graphic or iconic representation
• CPC Class Codes: G06F 2221/2103 Challenge-response
• CPC Class Codes: G06F 2221/2133 Verifying human interaction, e.g., Captcha
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06T 2200/24 involving graphical user interfaces [GUIs]
• CPC Class Codes: G06T 2207/10004 Still image; Photographic image
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/13 Edge detection
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 10/235 based on user input or interaction
• CPC Class Codes: G06V 10/50 by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
• CPC Class Codes: G06V 20/00 Scenes; Scene-specific elements control of digital cameras H04N23/60
• CPC Class Codes: H04L 63/08 for authentication of entities cryptographic mechanisms or cryptographic arrangements for entity authentication H04L9/32
• CPC Class Codes: H04L 63/10 for controlling access to devices or network resources
• CPC Class Codes: H04L 9/3271 using challenge-response

Abstract

Multiple image verification challenges can be used to identify the location of an object within an initial image. For instance, a first set of tiles is generated using the initial image. This first set is provided to a client computing device for display in a first verification challenge requesting that the user select tiles including the object. In response, a user selection of tiles of the first set is received. These selected tiles are then used to generate a second set of tiles corresponding to a sub-portion of the initial image. The second set of tiles is provided to a client computing device for display in a second verification challenge. In response to the second verification challenge, a user selection of tiles of the second set is received. This user selection of tiles of the second set is then used to determine a location of the object in the image.

Claims

1. A method of determining a location of a known object within an image using multiple image verification challenges, the method comprising:

identifying, by one or more computing devices, an initial image including an object, wherein the object is known to be depicted within the image;

generating, by the one or more computing devices, a first set of tiles including at least two or more tiles using the initial image, each tile of the first set of tiles including a first sub-portion of the initial image;

providing, by one or more computing devices, the first set of tiles to a client computing device for display in a first verification challenge requesting selection of tiles that include the object;

receiving, by the one or more computing devices, a response to the first verification challenge, including user selected tiles of the first set of tiles;

using the user selected tiles to generate a second set of tiles including at least two or more tiles that together correspond to a second sub-portion of the initial image, each tile of the second set of tiles including a sub-portion of the second sub-portion of the initial image;

providing by the one or more computing devices, the second set of tiles to the client computing device for display in a second verification challenge requesting selection of tiles that include the object;

receiving, by the one or more computing devices, a response to the second verification challenge, including user selected tiles of the second set of tiles; and

using, by the one or more computing devices, the user selected tiles of the second set of tiles to determine a location of the object in the image.

2. The method of claim 1, further comprising receiving, by one or more computing devices, a request to engage in the first verification challenge from the client computing device, the request identifying a network resource to which the client computing device is attempting to gain access.

3. The method of claim 2, further comprising providing the client computing device with access to the network resource based on the user selected tiles of the first set of tiles without reference to the user selected tiles of the second set of tiles.

4. The method of claim 1, further comprising modifying the initial image by enlarging a size of the object to create the second set of tiles.

5. The method of claim 4, wherein modifying the initial image further comprises shifting the location of the object within the initial image to create the second set of tiles.

6. The method of claim 1, further comprising receiving user selection of tiles of the second set of tiles from a plurality of client computing devices, and wherein determining the location is further based on the user selection of tiles of the second set of tiles received from the plurality of client computing devices.

7. The method of claim 6, further comprising, prior to receiving user selection of tiles of the second set of tiles from the plurality of client computing devices, providing the second verification challenge for display to each of the plurality of client computing devices.

8. The method of claim 1, further comprising:

using the user selected tiles of the second set of tiles to generate a third set of tiles corresponding to a third sub-portion of the initial image;

providing the third set of tiles to a different client computing device for display in a third verification challenge requesting selection of tiles including the object; and

receiving, by the one or more computing devices, a response to the third verification challenge, including user selected tiles of the third set of tiles.

9. The method of claim 8, wherein determining the location is further based on the user selected tiles of the third set of tiles.

10. A system for determining a location of a known object within an image using multiple image verification challenges, the system comprising one or more computing devices having one or more processors configured to:

identify an initial image including an object, wherein the object is known to be depicted within the image;

generate a first set of tiles including at least two or more tiles using the initial image, each tile of the first set of tiles including a first sub-portion of the initial image;

provide the first set of tiles to a client computing device for display in a first verification challenge requesting selection of tiles that include the object;

receive a response to the first verification challenge, including user selected tiles of the first set of tiles;

use the user selected tiles to generate a second set of tiles including at least two or more tiles that together correspond to a second sub-portion of the initial image, each tile of the second set of tiles including a sub-portion of the second sub-portion of the initial image;

provide the second set of tiles to the client computing device for display in a second verification challenge requesting selection of tiles that include the object;

receive a response to the second verification challenge, including user selected tiles of the second set of tiles; and

use the user selected tiles of the second set of tiles to determine a location of the object in the image.

11. The system of claim 10, wherein the one or more processors are further configured to receive a request to engage in the first verification challenge from the client computing device, the request identifying a network resource to which the client computing device is attempting to gain access.

12. The system of claim 11, wherein the one or more processors are further configured to provide the client computing device with access to the network resource based on the user selected tiles of the first set of tiles without reference to the user selected tiles of the second set of tiles.

13. The system of claim 10, wherein the one or more processors are further configured to modifying the initial image by enlarging a size of the object to create the second set of tiles.

14. The system of claim 13, wherein modifying the initial image further comprises shifting the location of the object within the initial image to create the second set of tiles.

15. The system of claim 10, wherein the one or more processors are further configured to receive user selection of tiles of the second set of tiles from a plurality of client computing devices, and wherein determining the location is further based on the user selection of tiles of the second set of tiles received from the plurality of client computing devices.

16. The system of claim 15, wherein the one or more processors are further configured to receive, prior to receiving user selection of tiles of the second set of tiles from the plurality of client computing devices, provide the second verification challenge for display to each of the plurality of client computing devices.

17. The system of claim 10, wherein the one or more processors are further configured to:

use the user selected tiles of the second set of tiles to generate a third set of tiles corresponding to a third sub-portion of the initial image;

provide the third set of tiles to a different client computing device for display in a third verification challenge requesting selection of tiles including the object; and

receive a response to the third verification challenge, including user selected tiles of the third set of tiles.

18. The system of claim 17, wherein determining the location is further based on the user selected tiles of the third set of tiles.

19. A non-transitory computer-readable medium storing instructions, which when executed by one or more processors, cause the one or more processors to perform a method of determining a location of a known object within an image using multiple image verification challenges, the method comprising:

identifying an initial image including an object, wherein the object is known to be depicted within the image;

generating a first set of tiles including at least two tiles using the initial image, each tile of the first set of tiles including a first sub--portion of the initial image;

providing the first set of tiles to a client computing device for display in a first verification challenge requesting selection of tiles that include the object;

receiving a response to the first verification challenge, including user selected tiles of the first set of tiles;

using the user selected tiles to generate a second set of tiles including at least two tiles that together correspond to a second sub-portion of the initial image, each tile of the second set of tiles including a sub-portion of the second sub-portion of the initial image;

providing the second set of tiles to the client computing device for display in a second verification challenge requesting selection of tiles that include the object;

receiving a response to the second verification challenge, including user selected tiles of the second set of tiles; and

using the user selected tiles of the second set of tiles to determine a location of the object in the image.

20. The medium of claim 19, wherein the instructions further cause the one or more processors to:

use the user selected tiles of the second set of tiles to generate a third set of tiles corresponding to a third sub-portion of the initial image;

provide the third set of tiles to a different client computing device for display in a third verification challenge requesting selection of tiles including the object; and

receive a response to the third verification challenge, including user selected tiles of the third set of tiles, and wherein determining the location is further based on the user selected tiles of the third set of tiles.