OBJECT DETECTION VIA VALIDATION WITH VISUAL SEARCH

• US 20140247963A1
• Filed: 03/01/2013
• Published: 09/04/2014
• Est. Priority Date: 03/01/2013
• Status: Active Grant

• Current Assignee: Adobe Systems Incorporated (Adobe Inc.)
• Original Assignee: Adobe Systems Incorporated (Adobe Inc.)
• Inventors: Lin, Zhe, Brandt, Jonathan, Shen, Xiaohui
• Granted Patent: US 9,224,066 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06V 10/751 Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
• CPC Class Codes: G06V 30/248 involving plural approaches, e.g. verification by template match; Resolving confusion among similar patterns, e.g. "O" versus "Q" G06V30/242 takes precedence
• CPC Class Codes: G06V 40/16 Human faces, e.g. facial parts, sketches or expressions
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation

Abstract

One exemplary embodiment involves receiving, at a computing device comprising a processor, a test image having a candidate object and a set of object images detected to depict a similar object as the test image. The embodiment involves localizing the object depicted in each one of the object images based on the candidate object in the test image to determine a location of the object in each respective object image and then generating a validation score for the candidate object in the test image based at least in part on the determined location of the object in the respective object image and known location of the object in the same respective object image. The embodiment also involves computing a final detection score for the candidate object based on the validation score that indicates a confidence level that the object in the test image is located as indicated by the candidate object.

Claims

1. A method comprising:

identifying a candidate region of a test image as potentially depicting an instance of an object;

identifying a first region in an object image that is similar to the candidate region of the test image, the object image being different from the test image;

performing a comparison of the first region with a second region of the object image, the second region known to actually depict the instance of the object; and

determining a validation score for the candidate region based on the comparison,

wherein identifying the candidate region, identifying the first region, performing the comparison, and determining the validation score are performed by a processor of a computer device executing instructions.

4. The method of claim 1, wherein the object is a face.

5. A computer-implemented method comprising:

receiving, at a computing device comprising a processor, a test image having at least one candidate rectangle and a set of object images detected to depict a similar object as the test image;

localizing an object depicted in each one of the object images based on the test image to determine a location of the object in each respective object image; and

generating a validation score for the at least one candidate rectangle based at least in part on the determined location of the object in the respective object image and a known location of the object in the same respective object image.

6. The computer-implemented method of claim 5, wherein the identified location is manually identified.

7. The computer-implemented method of claim 5, wherein determining the location of the object depicted in the object image is based at least in part on the at least one candidate sub-rectangle in the test image.

8. The computer-implemented method of claim 5, wherein the determined location of the object depicted in the object image is represented by a first sub-rectangle and the known location of the object depicted in the object image is represented by a second sub-rectangle.

9. The computer-implemented method of claim 5, wherein the validation score for the at least one candidate sub-rectangle is based at least in part on an amount of overlap between the determined location of the object in the object image and the known location of the object in the object image.

10. The computer-implemented method of claim 8, wherein the validation score for the at least one candidate sub-rectangle is based at least in part on an area of overlap between the first sub-rectangle and the second sub-rectangle.

11. The computer-implemented method of claim 5, further comprising generating a final detection score for the at least one candidate sub-rectangle based at least in part on a similarity score between the test image and the object image and a validation score of the at least one candidate sub-rectangle, the final detection score representing a confidence level that an object is located as indicated by the at least one candidate sub-rectangle.

12. A system comprising:

a processor for executing instructions stored in computer-readable medium on one or more devices,

the instructions comprising one or more modules configured to perform the steps comprising;

receiving a test image having a candidate face and a set of top-ranked object images, each one of the top-ranked images having a face detected to be similar to the candidate face of the test image;

ranking the object images in the set of top-ranked object images based at least in part on a similarity score;

generating a validation score for the candidate face based at least in part on localization of the object in each of the top-ranked images using the test image; and

generating a detection score for the candidate face based at least in part on the validity score for the candidate face and a similarity score between the test image and the respective object image.

13. The system of claim 12, wherein generating the validity score for the candidate face based at least in part on localization of the object in each of the top-ranked images using the test image comprises:

determining a location of the object depicted in the object image based at least in part on generating a similarity measure between matching features of the candidate face of test image and the respective object image; and

comparing the determined location of the object in the object image with an identified location of the object in the object image.

14. The system of claim 13, wherein the localization yields a sub-rectangle based on spatial information of the object depicted in the test image.

15. The system of claim 13, wherein the determined location of the object depicted in the object image is represented by a first sub-rectangle and the identified location of the object depicted in the object image is represented by a second sub-rectangle.

16. The system of claim 12, wherein the validation score is based at least in part on an amount of overlap between the determined location of the object in the object image and the known location of the object in the object image.

17. The system of claim 15, wherein the validation score is based at least in part on an area of overlap between the first sub-rectangle and the second sub-rectangle.

18. The system of claim 13, wherein the identified location is manually identified.

19. The system of claim 15, where in the validation score is based at least in part on an overlap ratio between the first sub-rectangle and the second sub-rectangle.

20. The system of claim 19, wherein the detection score for the candidate face represents a confidence level that a face depicted in the test image is located as indicated by the candidate face.

21. A method comprising:

receiving a test image having a candidate object and a set of object images detected to depict an object similar to the test image;

ranking the object images based on a similarity between the respective object image and the test image;

localizing each object image to determine a location of the object in the object image based at least in part on the candidate object in the test image; and

generating a validity score for the candidate object based at least in part on the determined location of the object in the object image and an identified location of the object in the object image.

2. The method of claim 21, wherein the validation score is determined based on multiple comparisons involving multiple object images, each comparison comprising a respective localized region based on the candidate region and a respective known region at which a respective instance of the object is known.

3. The method of claim 2, wherein the multiple object images are a subset of possible object images selected based on the object images of the subset having higher similarity scores with respect to the test image than other object images of the possible object images.

22. The method of claim 21 further comprising generating a detection score for the candidate object based at least in part on the validity score for the respective object image and a similarity score between the respective object image and the test image.

23. The method of claim 21, wherein the identified location of the object in the object image is manually identified.

24. The method of claim 21, wherein the validity score for the candidate object image is further based at least in part on an amount of overlap between the determined location of the object in the object image and the identified location of the object in the object image.