Edge-based recognition, systems and methods

• US 10,083,366 B2
• Filed: 04/14/2017
• Issued: 09/25/2018
• Est. Priority Date: 02/14/2014
• Status: Active Grant

• Current Assignee: Nant Holdings IP, LLC (NantWorks, LLC)
• Original Assignee: Nant Holdings IP, LLC (NantWorks, LLC)
• Inventors: Song, Bing, Siddiqui, Matheen
• Primary Examiner(s): THAI, HANH B
• Application Number:
  US15/488,371
• Publication Number:
  US 20170220892A1
• Time in Patent Office: 529 Days
• Field of Search: 707705-804
• CPC Class Codes: G06F 16/2246 Trees, e.g. B+trees
• CPC Class Codes: G06F 16/51 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/5854 using shape and object relationship
• CPC Class Codes: G06T 7/13 Edge detection
• CPC Class Codes: G06V 10/255 Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
• CPC Class Codes: G06V 10/46 Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features colour feature extraction G06V10/56
• CPC Class Codes: G06V 2201/031 of internal organs
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46

Abstract

Edge-based recognition systems and methods are presented. Edges of the object are identified from the image data based on co-circularity of edgels, and edge-based descriptors are constructed based on the identified edges. The edge-based descriptors along with additional perception metrics are used to obtain a list of candidate objects matched with the edge-based descriptors. Through various filtering processes and verification processes, false positive candidate objects are further removed from the list to determine the final candidate object.

Claims

1. A method of enabling a device action based on image data, the method comprising:

obtaining, by an image processing device, image data of a scene depicting an object associated with medical or biometric information;

generating, by the image processing device, a collection of edges representing curves in the image data by executing an implementation of a co-circularity algorithm to at least a portion of the image data associated with the object, wherein each edge in the collection of edges is characterized by one or more perception measures;

selecting, by the image processing device, a set of candidate edges from the collection of edges based at least in part on the perception measures of the edges within the collection of edges;

constructing, by the image processing device, sets of edgelets, within the image data, for the edges in the set of candidate edges;

deriving, by the image processing device, a plurality of edge-based descriptors from the sets of edgelets;

generating, by the image processing device, at least one perception metric related to the portion of the image data associated with the object based on the plurality of edge-based descriptors; and

configuring a device to take an action based on the plurality of edge-based descriptors, wherein configuring comprises storing or indexing content information associated with the object and the at least one perception metric into a content database according to an edge-based descriptor space.

2. The method of claim 1, wherein the object comprises a tissue or an organ of an organism.

3. The method of claim 1, wherein the image processing device comprises one of a medical imaging device or a biometric device.

4. The method of claim 1, wherein the image data includes at least one of still image data, real-time image data, and video image data.

5. The method of claim 1, wherein the plurality of edge-based descriptors includes at least one of color information and scale information.

6. The method of claim 1, wherein the action includes indexing content information related to the object into a content database according an indexing scheme based on the edge-based descriptor space.

7. The method of claim 6, wherein the indexing scheme comprises a tree structure.

8. The method of claim 7, wherein the tree structure comprises at least one of a spill tree and a k-d tree.

9. The method of claim 6, wherein the content information related to the object comprises non-edge attributes.

10. The method of claim 9, wherein the non-edge attributes comprise at least one of the following:

scale information, color information, and metadata.

11. The method of claim 1, further comprising providing access to a descriptor database storing known descriptors linked to content information related to the object.

12. The method of claim 11, further comprising updating the known descriptors associated with the object in the descriptor database based on one or more of the plurality of edge-based descriptors.

13. The method of claim 11, further comprising identifying pairs of nearest neighbor edge-based descriptors from the known descriptors and the plurality of edge-based descriptors.

14. The method of claim 13, further comprising grouping the pairs of nearest neighbor edge-based descriptors in groups of paired descriptors according to a similarity filter applied to the pairs of nearest neighbor edge-based descriptors.

15. The method of claim 14, wherein the similarity filter comprises a Hough transform.

16. The method of claim 14, further comprising generating a candidate set of matched descriptors from the groups of paired descriptors by randomly selecting descriptor pairs within each group of paired descriptors and identifying pairs that match an affine hypothesis.

17. The method of claim 16, further comprising generating a result set by reducing false positives from the candidate set of matched descriptors via identifying alignment edges in the image data associated with the plurality of edge-based descriptors to modeled image data associated with the known descriptors through a homography geometry.

18. The method of claim 16, further comprising generating the result set by reducing false positives from the candidate set of matched descriptors via filtering the candidate set based at least in part on at least one of scale information, color information, and one or more of the perception measures.

19. The method of claim 1, wherein the edgelets comprise one or more pixels.

20. The method of claim 1, wherein the one or more perception measures comprise at least one of the following:

a length of a curve, a smoothness measure of a curve, a curvature of a curve, an intersection of curves, and a conjunction of curves.

21. An apparatus for enabling a device action based on image data, the apparatus comprising:

a processor;

a memory device storing software instructions for enabling a device action based on image data; and

an image processing device executable on the processor according to software instructions stored in the memory device and configured to;

obtain image data of a scene depicting an object associated with medical or biometric information;

generate a collection of edges representing curves in the image data by executing an implementation of a co-circularity algorithm to at least a portion of the image data associated with the object, wherein each edge in the collection of edges is characterized by one or more perception measures;

select a set of candidate edges from the collection of edges based at least in part on the perception measures of the edges within the collection of edges;

construct sets of edgelets, within the image data, for the edges in the set of candidate edges;

derive a plurality of edge-based descriptors from the sets of edgelets;

generate at least one perception metric related to the portion of the image data associated with the object based on the plurality of edge-based descriptors; and

configure a device to take an action based on the plurality of edge-based descriptors, wherein configuring comprises storing or indexing content information associated with the object and the at least one perception metric into a content database according to an edge-based descriptor space.

22. A non-transitory computer-readable medium having computer instructions stored thereon for enabling a device action based on image data, which, when executed by a processor, cause the processor to perform one or more steps comprising:

obtaining image data of a scene depicting an object associated with medical or biometric information;

generating a collection of edges representing curves in the image data by executing an implementation of a co-circularity algorithm to at least a portion of the image data associated with the object, wherein each edge in the collection of edges is characterized by one or more perception measures;

selecting a set of candidate edges from the collection of edges based at least in part on the perception measures of the edges within the collection of edges;

constructing sets of edgelets, within the image data, for the edges in the set of candidate edges;

deriving a plurality of edge-based descriptors from the sets of edgelets;

generating at least one perception metric related to the portion of the image data associated with the object based on the plurality of edge-based descriptors; and

configuring a device to take an action based on the plurality of edge-based descriptors, wherein configuring comprises storing or indexing content information associated with the object and the at least one perception metric into a content database according to an edge-based descriptor space.