Methods and systems for recognizing objects based on one or more stored training images

• US 10,169,684 B1
• Filed: 06/10/2016
• Issued: 01/01/2019
• Est. Priority Date: 10/01/2015
• Status: Active Grant

• Current Assignee: IntelliVision Technologies Corp. (Melrose Industries Plc)
• Original Assignee: IntelliVision Technologies Corp. (Melrose Industries Plc)
• Inventors: Nathan, Vaidhi, Gupta, Gagan, Jindal, Nitin, Gope, Chandan
• Primary Examiner(s): Shin, Soo
• Application Number:
  US15/179,966
• Time in Patent Office: 935 Days
• Field of Search: None
• CPC Class Codes: G06F 16/51 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/5854 using shape and object relationship
• CPC Class Codes: G06F 16/7837 using objects detected or recognised in the video content
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06F 18/23 Clustering techniques
• CPC Class Codes: G06F 18/24147 Distances to closest patterns, e.g. nearest neighbour classification
• CPC Class Codes: G06V 10/17 using hand-held instruments
• CPC Class Codes: G06V 10/50 by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
• CPC Class Codes: G06V 10/7557 based on appearance, e.g. active appearance models [AAM]
• CPC Class Codes: G06V 10/774 Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59
• CPC Class Codes: G06V 20/46 Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames

Abstract

The present invention discloses methods and systems for recognizing an object in an input image based on stored training images. An object recognition system the input image, computes a signature of the input image, compares the signature with one or more stored signatures and retrieves one or more matching images from the set of training images. The matching images are then displayed to the user for further action.

Claims

1. An object recognition system, comprising of:

a processor;

a non-transitory storage element coupled to the processor;

encoded instructions stored in the non-transitory storage element,

wherein the encoded instructions when implemented by the processor, configure the object recognition system to;

generate a signature for an input image of an object by an image signature generation unit, wherein the image signature generation unit is further comprising of;

a feature detection unit configured to detect one or more feature points in the input image; and

a feature description unit configured to compute a description for each feature point of the one or more the feature points, wherein the feature description unit is further configured to;

identify a dominant gradient direction in a region around the feature point, wherein an angle of the dominant gradient direction is α

;

center a patch around the feature point, wherein the patch is tilted at the angle α

;

divide the patch in R segments;

compute a vector of length N for each segment of the R segments, wherein the vector is computed based on a horizontal gradient (dx) and a vertical gradient (dy) corresponding to each pixel in the segment;

compute a consolidated vector of length R*N by consolidating vectors computed for all the R segments; and

compute a byte vector of length R*N, wherein the byte vector is computed by normalizing the consolidated vector, wherein the byte vector is the description of the feature point;

whereby, the signature of the input image comprises the description corresponding to each of the one or more feature points in the input image;

store the set of training images in a data storage, wherein each training image of the set of training images is associated with one or more training feature descriptors, the data storage further comprising;

an index mapping unit configured to create an index mapping based on training feature descriptors; and

identify a matching image of the set of training images by a search engine comparing the signature of the input image with the training feature descriptors using the index mapping.

2. The object recognition system of claim 1, wherein the object recognition system is a part of at least one of the group comprising a mobile phone, a computer and a server.

3. The object recognition system of claim 1 further comprising of a compactor unit configured to compact a size associated with the description of the feature point, wherein the size corresponds to memory required to store the description of the feature point.

4. The object recognition system of claim 1, wherein the index mapping is at least one data structure of the group comprising an array, a hash table, a lookup table and k-dimensional (k-d) tree.

5. The object recognition system of claim 4, wherein the index mapping unit is further configured to prune the k-dimensional (k-d) tree.

6. The object recognition system of claim 1, wherein the search engine is further configured to compare the signature of the input image with the one or more training feature descriptors of a training image of the set of the training images using spatial consistency.

7. The object recognition system of claim 6, wherein the search engine is further configured to compute a spatial transformation between the input image and the training image.

8. The object recognition system of claim 7, wherein the search engine is further configured to compute a matching score between the input image and the training image.

9. The object recognition system of claim 1, wherein the data storage is further configured to cluster the set of training images into Similar Image Clusters (SICs).

10. The object recognition system of claim 1, the input image is a frame of a video.

11. The object recognition system of claim 10, wherein the search engine is further configured to perform a temporal verification check on the matching image.

12. The object recognition system of claim 1, further comprising a compactor unit configured to compact a size associated with the description of the feature point, wherein the size corresponds to memory required to store the description of the feature point;

and

the processor computes the index mapping of the feature points based on the training feature descriptors.

13. A method for recognizing an object in one or more input images based on one or more training images stored in a data storage, the method comprising;

generating a signature for an input image of the object comprising;

detecting one or more feature points in the input image; and

computing the description for each feature point of the one or more feature points, comprising;

identifying a dominant gradient direction in a region around a feature point, wherein an angle of the dominant gradient direction is α

;

centering a patch around the feature point, wherein the patch is tilted at the angle α

;

dividing the patch in R segments;

computing a vector of length N for each segment of the R segments, wherein the vector is computed based on at least a horizontal gradient (dx) and at least a vertical gradient (dy) corresponding to each pixel in the segment

computing a consolidated vector of length R*N by consolidating vectors computed for all the R segments; and

computing a byte vector of length R*N, wherein the byte vector is computed by normalizing the consolidated vector, wherein the byte vector is the description of the feature point;

whereby, the signature of the input image comprises the description corresponding to each of the one or more feature points in the input image; and

identifying a matching image of the set of training images by comparing the signature of the input image with training feature descriptors of the set of training images using an index mapping.

14. The method for recognizing an object of claim 13, wherein the index mapping is at least one data structure of the group comprising an array, a hash table, a lookup table and k-dimensional (k-d) tree.

15. The method for recognizing an object of claim 14 further comprising pruning the k-dimensional (k-d) tree.

16. The method for recognizing an object of claim 13 further comprising comparing the signature of the input image with the training feature descriptors of the set of training images using spatial consistency.

17. The method for recognizing an object of claim 16 further comprising computing a spatial transformation between the input image and a training image of the set of training images.

18. The method for recognizing an object of claim 17 further comprising computing a matching score between the input image and the training image of the set of training images.

19. The method for recognizing an object of claim 13 further comprising clustering the set of training images into Similar Image Clusters (SICs).