ROBUST KEYPOINT FEATURE SELECTION FOR VISUAL SEARCH WITH SELF MATCHING SCORE

• US 20140185941A1
• Filed: 12/09/2013
• Published: 07/03/2014
• Est. Priority Date: 01/02/2013
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Li, Zhu, Nagar, Abhishek, Srivastava, Gaurav, Bhat, Kong Posh, Fernandes, Felix Carlos, Xin, Xin
• Granted Patent: US 9,235,780 B2
• Time in Patent Office: Days
• US Class Current: 382/201
• CPC Class Codes: G06V 10/462 Salient features, e.g. scale invariant feature transforms [SIFT]
• CPC Class Codes: G06V 10/757 Matching configurations of points or features

Abstract

To improve feature selection accuracy during a visual search, interest points within a query image are two-way matched to features in an affine transformed image or otherwise transformed version of the query image. A user device implements a method for selecting local descriptors in the visual search. The method includes: detecting a first set of interest points for the original image; computing an affine transform matrix; computing a new image as a transformation of the original image using the affine transform matrix; detecting a second set of interest points from the and new image; performing a two-way matching between the first set of interest points and the second set of interest points; sorting matching pairs according to a specified self-matching score (SMS); assigning an infinite value to SMS of unmatched interest points from the original image; selecting the interest points based on SMS. Significant performance gains reduce false positive matches.

Claims

1. A method, comprising:

detecting a first set of key points for an original image to be used as a query image for the visual search request;

generating a affine-transformed image as an affine-transformation of the original image;

detecting a second set of key points from the transformed image;

determining matches between key points for the original image and key points for the transformed image; and

selecting matching key points for the original and transformed images to transmit to the visual search server in the visual search request.

2. The method of claim 1, further comprising:

in the device, assigning scores to the matching key points based upon Scale Invariant Feature Transform (SIFT) algorithm distances between a respective key point in the original image from a respective key point in the transformed image.

3. The method of claim 1, wherein detecting key points comprises using at least one of:

a Scale Invariant Feature Transform (SIFT) algorithm, a Speeded Up Robust Features (SURF) algorithm, and a Binary Robust Invariant Scalable Keypoints (BRISK) algorithm.

4. The method of claim 1, wherein generating a transformed image comprises using an affine transform matrix including at least two randomly selected skew parameters between 0 through 1/√

{square root over (2)}.

5. The method of claim 1, wherein selecting matching key points comprises selecting up to a predetermined number of key points to transmit.

6. The method of claim 1, determining matches between key points for the original image and key points for the transformed image comprises two-way matching.

7. The method of claim 1, wherein determining matches between key points for the original image and key points for the transformed image further comprises:

selecting key points for the original image meeting at least a predetermined criteria for comparison to key points for the transformed image.

8. The method of claim 7, wherein the predetermined criteria comprises a strength of the respective key point.

9. The method of claim 7, wherein generating the transformed image comprises transforming image patches corresponding to the selected key points.

10. The method of claim 1, further comprising:

transmitting a predetermined number of key points for the original image in the visual search request.

11. An apparatus for submitting a visual search request to a visual search server, comprising:

a processor configured

to detect a first set of key points for an original image to be used as a query image for the visual search request,

to generate a transformed image as an affine-transformation of the original image,

to detect a second set of key points from the transformed image,

to determine matches between key points for the original image and key points for the transformed image, and

to select matching key points for the original and transformed images to transmit to the visual search server in the visual search request.

12. The apparatus of claim 11, wherein the processor is configured to assign scores to the matching key points based upon Scale Invariant Feature Transform (SIFT) algorithm distances between a respective key point in the original image from a respective key point in the transformed image.

13. The apparatus of claim 11, wherein detecting key points comprises using at least one of:

a Scale Invariant Feature Transform (SIFT) algorithm, a Speeded Up Robust Features (SURF) algorithm, and a Binary Robust Invariant Scalable Keypoints (BRISK) algorithm.

14. The apparatus of claim 11, wherein generating a transformed image comprises using an affine transform matrix including at least two randomly selected skew parameters between 0 and 1/√

{square root over (2)}.

15. The apparatus of claim 11, wherein selecting matching key points comprises selecting up to a predetermined number of key points to transmit.

16. The apparatus of claim 11, determining matches between key points for the original image and key points for the transformed image comprises two-way matching.

17. The apparatus of claim 11, wherein the processor is further configured to determine matches between key points for the original image and key points for the transformed image by selecting key points for the original image meeting at least a predetermined criteria for comparison to key points for the transformed image.

18. The apparatus of claim 17, wherein the predetermined criteria comprises a strength of the respective key point.

19. The apparatus of claim 17, wherein generating the transformed image comprises affine-transforming image patches corresponding to the selected key points.

20. The apparatus of claim 11, further comprising:

a transmitter configured to transmit a predetermined number of key points for the original image in the visual search request.