Edge-based recognition, systems and methods

• US 9,665,606 B2
• Filed: 02/16/2015
• Issued: 05/30/2017
• Est. Priority Date: 02/14/2014
• Status: Active Grant

• Current Assignee: Nant Holdings IP, LLC (NantWorks, LLC)
• Original Assignee: Nant Holdings IP, LLC (NantWorks, LLC)
• Inventors: Song, Bing, Siddiqui, Matheen
• Primary Examiner(s): Thai, Hanh
• Application Number:
  US14/623,341
• Publication Number:
  US 20150261803A1
• Time in Patent Office: 834 Days
• Field of Search: 707705, 707715, 707754, 707804, 382103, 382173, 382181
• CPC Class Codes: G06F 16/2246 Trees, e.g. B+trees
• CPC Class Codes: G06F 16/51 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/5854 using shape and object relationship
• CPC Class Codes: G06T 7/13 Edge detection
• CPC Class Codes: G06V 10/255 Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
• CPC Class Codes: G06V 10/46 Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features colour feature extraction G06V10/56
• CPC Class Codes: G06V 2201/031 of internal organs
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46

Abstract

Edge-based recognition systems and methods are presented. Edges of the object are identified from the image data based on co-circularity of edgels, and edge-based descriptors are constructed based on the identified edges. The edge-based descriptors along with additional perception metrics are used to obtain a list of candidate objects matched with the edge-based descriptors. Through various filtering processes and verification processes, false positive candidate objects are further removed from the list to determine the final candidate object.

Claims

1. A method of enabling a device action based on image data, the method comprising:

obtaining, by an image processing device, image data having a representation of an object;

generating, by the image processing device, a collection of edges representing curves in the image data by applying a co-circularity algorithm to at least a portion of the image data associated with the object, each edge in the collection of edges comprising perception measure;

selecting, by the image processing device, a set of candidate edges from the collection of edges at least in part based on the perception measures of the edges within the collection of edges;

constructing, by the image processing device, sets of edgelets, at a pixel level within the image data, for the edges in the set of candidate edges;

deriving, by the image processing device, a plurality of edge-based descriptors from the sets of edgelets, each edge-based descriptor representing a constellation of edgelets among edges in the collection of edges; and

configuring a device to initiate an action based on the plurality of edge-based descriptors.

2. The method of claim 1, wherein the action includes indexing content information related to the object into a content database according an indexing scheme based on an edge-descriptor space.

3. The method of claim 2, wherein the indexing scheme comprises a tree structure.

4. The method of claim 3, wherein the tree structure comprises at least one of a spill tree and a k-d tree.

5. The method of claim 2, wherein the content information related to the object comprises non-edge attributes.

6. The method of claim 4, wherein the non-edge attributes comprise at least one of the following:

scale information, image descriptor, and metadata.

7. The method of claim 1, further comprising providing access to a descriptor database storing known descriptors linked to content information related to the object.

8. The method of claim 7, further comprising identifying pairs of nearest neighbor edge-based descriptors from the known descriptors and the plurality of edge-based descriptors.

9. The method of claim 8, further comprising grouping the pairs of nearest neighbor edge-based descriptors in groups of paired descriptors according to a similarity filter applied to the pairs of nearest neighbor edge-based descriptors.

10. The method of claim 9, wherein the similarity filter comprises a Hough transform.

11. The method of claim 9, further comprising generating a candidate set of matched descriptors from the groups of paired descriptors by randomly selecting descriptor pairs within each group of paired descriptors and identifying pairs that match an affine hypothesis.

12. The method of claim 11, further comprising generating a result set by reducing false positives from the candidate set of matched descriptors via identifying alignment edges in the image data associated with the plurality of edge-based descriptors to modeled image data associated with the known descriptors through a homography geometry.

13. The method of claim 1, wherein the edgelets comprise a chain of pixels.

14. The method of claim 1, wherein the perception measure comprises at least one of the following:

a length of the curve, a smoothness measure of the curve, a curvature of the curve, an intersection of curves, and a conjunction of curves.