ROBUST FEATURE IDENTIFICATION FOR IMAGE-BASED OBJECT RECOGNITION

• US 20150310306A1
• Filed: 04/24/2015
• Published: 10/29/2015
• Est. Priority Date: 04/24/2014
• Status: Active Grant

• Current Assignee: Nant Holdings IP, LLC (NantWorks, LLC)
• Original Assignee: NantWorks, LLC
• Inventors: Song, Bing, Jaber, Mustafa, Lin, Liwen
• Granted Patent: US 9,558,426 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06T 3/00 Geometric image transformations in the plane of the image
• CPC Class Codes: G06T 3/40 Scaling of whole images or parts thereof, e.g. expanding or contracting
• CPC Class Codes: G06V 10/42 Global feature extraction by analysis of the whole pattern, e.g. using frequency domain transformations or autocorrelation
• CPC Class Codes: G06V 10/50 by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
• CPC Class Codes: G06V 10/75 Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries

Abstract

Techniques are provided that include identifying robust features within a training image. Training features are generated by applying a feature detection algorithm to the training image, each training feature having a training feature location within the training image. At least a portion of the training image is transformed into a transformed image in accordance with a predefined image transformation. Transform features are generated by applying the feature detection algorithm to the transformed image, each transform feature having a transform feature location within the transformed image. The training feature locations of the training features are mapped to corresponding training feature transformed locations within the transformed image in accordance with the predefined image transformation, and a robust feature set is compiled by selecting robust features, wherein each robust feature represents a training feature having a training feature transformed location proximal to a transform feature location of one of the transform features.

Claims

1. A method of identifying robust features within a training image at a feature detection device, the method comprising:

generating, by the feature detection device, training features by applying a feature detection algorithm to the training image, each training feature having a training feature location within the training image;

transforming, by the feature detection device, at least a portion of the training image into a transformed image in accordance with a predefined image transformation;

generating, by the feature detection device, transform features by applying the feature detection algorithm to the transformed image, each transform feature having a transform feature location within the transformed image;

mapping, by the feature detection device, the training feature locations of the training features to corresponding training feature transformed locations within the transformed image in accordance with the predefined image transformation; and

compiling, by the feature detection device, a robust feature set by selecting robust features, wherein each robust feature represents a training feature having a training feature transformed location proximal to a transform feature location of one of the transform features.

2. The method of claim 1, wherein each of the training feature locations comprises a pixel coordinate.

3. The method of claim 1, wherein each of the transform feature locations comprises a transformed pixel coordinate.

4. The method of claim 1, wherein obtaining the training image includes capturing a video frame of a video stream via an image capture device.

5. The method of claim 1, wherein the feature detection algorithm includes at least one of a scale-invariant feature transform (SIFT), Fast Retina Keypoint (FREAK), Histograms of Oriented Gradient (HOG), Speeded Up Robust Features (SURF), DAISY, Binary Robust Invariant Scalable Keypoints (BRISK), FAST, Binary Robust Independent Elementary Features (BRIEF), Harris Corners, Edges, Gradient Location and Orientation Histogram (GLOH), Energy of image Gradient (EOG) or Transform Invariant Low-rank Textures (TILT) feature detection algorithm.

6. The method of claim 1, wherein the predefined image transformation comprises a geometric transform.

7. The method of claim 6, wherein transforming at least a portion of the training image includes scaling the training image by a scaling factor to form the transformed image.

8. The method of claim 7, wherein the scaling factor is based on one or more subject-matter characteristics of an image recognition search.

9. The method of claim 8, wherein the one or more subject-matter characteristics of the image recognition search are determined to be indicative of query images being predominantly at least one of near-sequence images, far-sequence images, skewed images, rich texture images, medium texture images or poor texture images.

10. The method of claim 7, wherein the scaling factor comprises a linear scaling factor between 50% and 90% of an original size of the training image.

11. The method of claim 7, wherein the scaling factor comprises a linear scaling factor between 75% and 80% of an original size of the training image.

12. The method of claim 6, wherein transforming at least a portion of the training image includes at least one of skewing, shearing, rotating and digitally transforming the training image by an angle to form the transformed image.

13. The method of claim 12, wherein the angle is between 20 degrees and 45 degrees.

14. The method of claim 1, wherein the predefined image transformation comprises an image processing transform.

15. The method of claim 14, wherein the image processing transform includes at least one of a Gaussian filter, a color transform and lossy compression.

16. The method of claim 1, wherein transforming at least a portion of the training image includes applying a compound transform as the predefined image transformation to form the transformed image, the compound transform including a plurality of geometric transforms or image processing transforms.

17. The method of claim 16, wherein the compound transform is determined based on one or more subject-matter characteristics of an image-based object recognition search.

18. The method of claim 1, further comprising adjusting one or more tuning parameters of the predefined image transformation and generating a second robust feature set based on the adjusted tuning parameters.

19. The method of claim 18, wherein the one or more tuning parameters include at least one of a scaling factor and an angle.

20. The method of claim 18, wherein the one or more tuning parameters are automatically adjusted based on one or more subject-matter characteristics of an image-based object recognition search.

21. The method of claim 18, further comprising optimizing the adjusted tuning parameters based on a recognized efficiency of the adjusted tuning parameters when used in relation to a test library of images.

22. The method of claim 21, wherein the test library of images does not include the training image.

23. The method of claim 21, further comprising validating the adjusted tuning parameters based on the test library of images.

24. The method of claim 1, wherein compiling the robust feature set includes:

determining a transform feature location that is within a determined threshold distance of a training feature transformed location; and

comparing a training feature associated with the training feature transformed location with a transform feature associated with the transform feature location based on a similarity measure.

25. The method of claim 24, wherein the similarity measure is based on at least one of Euclidean distance, Hellinger distance and Hamming distance.

26. The method of claim 1, wherein each of the training features and transform features are described by a feature descriptor in accordance with the feature detection algorithm.

27. The method of claim 1, wherein the training image is one of an undistorted image, an infrared-filtered image, an x-ray image, a 360-degree view image, a machine-view image, a frame of video data, a graphical rendering and a perspective-view of a three-dimensional object.

28. The method of claim 1, further comprising presenting a plurality of image transformations to a user for selection as the predefined image transformation.

29. The method of claim 1, wherein the predefined image transformation is selected independently from a method used to capture the training image.

30. A system for identifying robust features within a training image, the system comprising:

a processor;

a memory device; and

a feature detector executable on the processor according to software instructions stored in the memory device, the feature detector being configured to;

generate training features by applying a feature detection algorithm to the training image, each training feature having a training feature location within the training image;

transform at least a portion of the training image into a transformed image in accordance with a predefined image transformation;

generate transform features by applying the feature detection algorithm to the transformed image, each transform feature having a transform feature location within the transformed image;

map the training feature locations of the training features to corresponding training feature transformed locations within the transformed image in accordance with the predefined image transformation; and

compile a robust feature set by selecting robust features, wherein each robust feature represents a training feature having a training feature transformed location proximal to a transform feature location of one of the transform features.

31. A computer program product embedded in a non-transitory computer readable medium comprising instructions executable by a computer processor to identify robust features within an image, the instructions being executable by a computer processor to execute processing comprising:

generating training features by applying a feature detection algorithm to the training image, each training feature having a training feature location within the training image;

transforming at least a portion of the training image into a transformed image in accordance with a predefined image transformation;

generating transform features by applying the feature detection algorithm to the transformed image, each transform feature having a transform feature location within the transformed image;

mapping the training feature locations of the training features to corresponding training feature transformed locations within the transformed image in accordance with the predefined image transformation; and

compiling a robust feature set by selecting robust features, wherein each robust feature represents a training feature having a training feature transformed location proximal to a transform feature location of one of the transform features.