Image-based feature detection using edge vectors

• US 9,858,497 B2
• Filed: 11/23/2016
• Issued: 01/02/2018
• Est. Priority Date: 05/06/2014
• Status: Active Grant

• Current Assignee: Nant Holdings IP, LLC (NantWorks, LLC)
• Original Assignee: Nant Holdings IP, LLC (NantWorks, LLC)
• Inventors: Song, Bing, Witchey, Nicholas J.
• Primary Examiner(s): Liu, Li
• Application Number:
  US15/360,865
• Publication Number:
  US 20170076174A1
• Time in Patent Office: 405 Days
• CPC Class Codes: G06F 16/51 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06T 2207/10004 Still image; Photographic image
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/10116 X-ray image
• CPC Class Codes: G06T 2207/20076 Probabilistic image processing
• CPC Class Codes: G06T 2207/20164 Salient point detection; Corner detection
• CPC Class Codes: G06T 2207/30004 Biomedical image processing
• CPC Class Codes: G06T 3/40 Scaling of whole images or parts thereof, e.g. expanding or contracting
• CPC Class Codes: G06T 3/403 Edge-driven scaling; Edge-based scaling
• CPC Class Codes: G06T 5/40 using histogram techniques
• CPC Class Codes: G06T 7/13 Edge detection
• CPC Class Codes: G06T 7/143 involving probabilistic approaches, e.g. Markov random field [MRF] modelling
• CPC Class Codes: G06T 7/181 involving edge growing; involving edge linking
• CPC Class Codes: G06T 7/248 involving reference images or patches
• CPC Class Codes: G06V 10/44 Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
• CPC Class Codes: G06V 10/46 Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features colour feature extraction G06V10/56
• CPC Class Codes: G06V 10/462 Salient features, e.g. scale invariant feature transforms [SIFT]
• CPC Class Codes: G06V 10/758 : Involving statistics of pixels or of feature values, e.g. histogram matching
• CPC Class Codes: G06V 20/64 : Three-dimensional objects

Abstract

Techniques are provided in which a plurality of edges are detected within a digital image. An anchor point located along an edge of the plurality of edges is selected. An analysis grid associated with the anchor point is generated, the analysis grid including a plurality of cells. An anchor point normal vector comprising a normal vector of the edge at the anchor point is calculated. Edge pixel normal vectors comprising normal vectors of the edge at locations along the edge within the cells of the analysis grid are calculated. A histogram of similarity is generated for each of one or more cells of the analysis grid, each histogram of similarity being based on a similarity measure between each of the edge pixel normal vectors within a cell and the anchor point normal vector, and a descriptor is generated for the analysis grid based on the histograms of similarity.

Claims

1. An object feature tracking apparatus, comprising:

a digital image capturing device;

a non-transitory computer readable memory storing feature detection software instructions; and

a processor coupled with the digital image capturing device and the non-transitory computer readable memory, and configured to operate as a feature detector upon execution of the software instructions, wherein the feature detector is configurable to;

obtain, via the digital image capturing device, digital image frames that comprise digital images of at least one object;

detect a plurality of edges within a plurality of frames of the digital image frames;

track, across the plurality of frames, image features based on one or more anchor points detected on at least one edge of the plurality of edges;

assign a vector for each of at least some points of a plurality of points along the at least one edge such that a plurality of vectors is associated with the at least one edge, wherein each anchor point of the one or more anchor points corresponds to a vector of the plurality of vectors, thereby defining an anchor point vector, and wherein each anchor point defines a geometric center of an analysis grid located on the at least one edge;

generate an edge similarity measure as a function of relative measures between one of the anchor point vectors and at least some of the plurality of vectors within an analysis grid that corresponds to the one of the anchor point vectors; and

retrieve data associated with the at least one object as a function of the edge similarity measure.

2. The apparatus of claim 1, wherein the digital image frames comprise video data.

3. The apparatus of claim 2, wherein the video data comprises a video data stream.

4. The apparatus of claim 2, wherein the video data comprises real-time video data.

5. The apparatus of claim 1, wherein the at least one object comprises at least one of the following:

a texture-rich object, a texture-poor object, a person, an animal, a plant, a toy, a building, a product, a bar code, printed matter, an action figure, a logo, and a vehicle.

6. The apparatus of claim 1, wherein the at least one object comprises medical information.

7. The apparatus of claim 1, wherein the at least one object comprises at least two dimensions.

8. The apparatus of claim 7, wherein the at least one object comprises at least three dimensions.

9. The apparatus of claim 1, wherein the digital image capturing device comprises a digital camera.

10. The apparatus of claim 9, wherein the digital camera includes at least one of the following types of sensors:

a charge-coupled device (CCD) sensor, a complementary metal-oxide semiconductor (CMOS) sensor, and an ultrasound sensor.

11. The apparatus of claim 1, wherein the digital image capturing device comprises an image server.

12. The apparatus of claim 1, wherein the digital image capturing device comprises an image database.

13. The apparatus of claim 1, wherein the edge similarity measure comprises an edge-based feature descriptor.

14. The apparatus of claim 13, wherein the edge-based feature descriptor comprises a histogram of similarity generated as the function of the relative measures.

15. The apparatus of claim 13, wherein the feature detector is further configurable to generate edge-based feature descriptors in real-time with respect to obtaining the digital image frames.

16. The apparatus of claim 1, wherein the plurality of vectors includes at least one of the following:

an edge normal vector, an eigenvector, a Harris eigenvector, and a geometric normal vector.

17. The apparatus of claim 1, wherein the relative measures comprise at least one of the following:

a dot product among vectors, a relative magnitude among vectors, a Euclidean distance, a normalization, and a Gaussian weight.

18. The apparatus of claim 1, wherein the feature detector is further configurable to track the digital images within the digital image frames in real-time.

19. The apparatus of claim 1, wherein the feature detector is further configurable to track the digital images frame-by-frame within the digital image frames.

20. The apparatus of claim 1, wherein the detected image features include at least one of the following types of features:

scale-invariant feature transform (SIFT), Fast Retina Keypoint (FREAK), Histograms of Oriented Gradient (HOG), Speeded Up Robust Features (SURF), DAISY, Binary Robust Invariant Scalable Keypoints (BRISK), FAST, Binary Robust Independent Elementary Features (BRIEF), Harris Corners, Edges, Gradient Location and Orientation Histogram (GLOH), Energy of image Gradient (EOG), and Transform Invariant Low-rank Textures (TILT) features.

21. The apparatus of claim 1, wherein the plurality of edges comprises at least one of the following types of edges:

a Canny edge, a tensor voted edge, and a composition of edgelets.

22. The apparatus of claim 1, further comprising at least one of the following computing devices that incorporates the non-transitory computer readable memory and the processor:

a server, a mobile phone, a robot, and a tablet computer.

23. The apparatus of claim 1, wherein the digital image frame comprises as least one of the following types of digital imagery:

a robot-view, infrared imagery, x-ray imagery, 360-degree imagery, graphic rendering imagery, and synthetic imagery.