Metric-based recognition, systems and methods

• US 10,346,712 B2
• Filed: 10/05/2018
• Issued: 07/09/2019
• Est. Priority Date: 08/19/2013
• Status: Active Grant

• Current Assignee: Nant Holdings IP, LLC (NantWorks, LLC)
• Original Assignee: Nant Holdings IP, LLC (NantWorks, LLC)
• Inventors: Siddiqui, Matheen
• Primary Examiner(s): Seth, Manav
• Application Number:
  US16/153,458
• Publication Number:
  US 20190042873A1
• Time in Patent Office: 277 Days
• CPC Class Codes: G06F 16/3331 Query processing
• CPC Class Codes: G06F 16/51 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06F 16/7335 Graphical querying, e.g. query-by-region, query-by-sketch, query-by-trajectory, GUIs for designating a person/face/object as a query predicate end-user interface involving hot spots associated with the video H04N21/4725; end-user interface for selecting a Region of Interest H04N21/4728
• CPC Class Codes: G06F 16/785 using colour or luminescence
• CPC Class Codes: G06F 16/907 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06V 10/56 relating to colour
• CPC Class Codes: G06V 30/422 Technical drawings; Geographical maps
• CPC Class Codes: H04N 1/40012 Conversion of colour to monochrome

Abstract

Apparatus, methods and systems of object recognition are disclosed. Embodiments of the inventive subject matter generates map-altered image data according to an object-specific metric map, derives a metric-based descriptor set by executing an image analysis algorithm on the map-altered image data, and retrieves digital content associated with a target object as a function of the metric-based descriptor set.

Claims

1. A method of generating a metric-based recognition map comprising:

receiving, by a computing device configured to operate as an image processing engine, image data of at least part of a person;

compiling, by the image processing engine, an initial object-specific metric map for the person from the image data of the at least part of the person;

generating, by the image processing engine, a metric-based descriptor set by executing an image analysis algorithm on the image data as a function of the initial object-specific metric map; and

storing the metric-based descriptor set in an object recognition memory.

2. The method of claim 1, further comprising adjusting the initial object-specific metric map to generate a new object-specific metric map by tuning metric values in a manner effective to enhance differentiation of descriptors generated by the image analysis algorithm as executed on the portion of the image data.

3. The method of claim 2, wherein adjusting the initial object-specific metric map includes accepting user input that alters at least some metric values within the initial object-specific metric map.

4. The method of claim 2, wherein adjusting the initial object-specific metric map includes the image processing engine recommending at least one a metric value that increases a confidence of a descriptor.

5. The method of claim 2, wherein adjusting the initial object-specific metric map includes the image processing engine automatically adjusting metric values of the initial object-specific metric map.

6. The method of claim 2, wherein the new metric-based map comprises a non-linear mapping from metric values within the initial metric-based map.

7. The method of claim 1, further comprising generating an object-specific color map based the object-specific metric map.

8. The method of claim 7, further comprising storing the object-specific color map as part of the metric-based descriptor set.

9. The method of claim 1, further comprising identifying at least one of a position and an orientation of an imaging device configured to capture the image data.

10. The method of claim 9, further comprising storing the at least one of the position and the orientation with the metric-based descriptor set.

11. The method of claim 1, further comprising removing specularity from the image data.

12. The method of claim 1, wherein the object-specific metric map comprises a pixel-level metric map.

13. The method of claim 1, wherein the metric-based descriptor set comprises lighting invariant descriptors.

14. The method of claim 1, wherein the metric-based descriptor set comprises metric-based invariant descriptors.

15. The method of claim 14, wherein the metric-based descriptors comprise metric-based scale invariant descriptors.

16. The method of claim 1, wherein the object comprises a physical object.

17. The method of claim 1, further comprising storing a key frame bundle that includes the metric-based descriptor set.

20. The method of claim 1, wherein the map for the person comprises at least one tissue-specific map.

21. The method of claim 20 further comprising:

the at least one tissue-specific map aiding a recognition engine in differentiating structures or features of internal organs during surgery, and

the recognition engine identifying one or more features such as tumors because such features fail to conform to the person'"'"'s tissue-specific map.

26. The method of claim 1, wherein the map for the person is based on skin tone.

27. The method of claim 1, wherein the map for the person is based on skin features.

28. The method of claim 1, wherein the map for the person is based on tissue type.

29. The method of claim 1, wherein the map for the person is based on tissue features.

30. The method of claim 1, wherein the map for the person identifies a location of a tumor.

31. The method of claim 30, wherein the tumor is identified as a feature that fails to conform to a tissue-specific map for the person.

18. A non-transitory computer readable medium storing instructions executable on a processor for processing image data of at least part of a person to generate a metric-based recognition map, the instructions comprising instructions to:

compile an initial object-specific metric map for the person from the image data of the at least part of the person;

generate a metric-based descriptor set by executing an image analysis algorithm on the image data as a function of the initial object-specific metric map; and

store the metric-based descriptor set in an object recognition memory.

22. The computer readable medium of claim 18, wherein the map for the person comprises at least one tissue-specific map.

23. The computer readable medium of claim 22, further comprising:

instructions for aiding a recognition engine in differentiating structures or features of internal organs during surgery, and

the recognition engine identifying one or more features such as tumors because such features fail to conform to the person'"'"'s tissue-specific map.

19. A device for generating a metric-based recognition map comprising:

a computing device configured to operate as an image processing engine, the image processing engine configured to;

receive image data of at least part of a person;

compile an initial object-specific metric map for the person from the image data of the at least part of the person;

generate a metric-based descriptor set by executing an image analysis algorithm on the image data as a function of the initial object-specific metric map; and

store the metric-based descriptor set in an object recognition memory.

24. The device of claim 19, wherein the map for the person comprises at least one tissue-specific map.

25. The device of claim 24, further configured to:

aid a recognition engine in differentiating structures or features of internal organs during surgery, and

the recognition engine identifying one or more features such as tumors because such features fail to conform to the person'"'"'s tissue-specific map.