Metric based recognition, systems and methods

• US 9,659,033 B2
• Filed: 08/19/2014
• Issued: 05/23/2017
• Est. Priority Date: 08/19/2013
• Status: Active Grant

• Current Assignee: Nant Holdings IP, LLC (NantWorks, LLC)
• Original Assignee: Nant Holdings IP, LLC (NantWorks, LLC)
• Inventors: Siddiqui, Matheen
• Primary Examiner(s): Seth, Manav
• Application Number:
  US14/463,617
• Publication Number:
  US 20150049939A1
• Time in Patent Office: 1,008 Days
• CPC Class Codes: G06F 16/3331 Query processing
• CPC Class Codes: G06F 16/51 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06F 16/7335 Graphical querying, e.g. query-by-region, query-by-sketch, query-by-trajectory, GUIs for designating a person/face/object as a query predicate end-user interface involving hot spots associated with the video H04N21/4725; end-user interface for selecting a Region of Interest H04N21/4728
• CPC Class Codes: G06F 16/785 using colour or luminescence
• CPC Class Codes: G06F 16/907 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06V 10/56 relating to colour
• CPC Class Codes: G06V 30/422 Technical drawings; Geographical maps
• CPC Class Codes: H04N 1/40012 Conversion of colour to monochrome

Abstract

Apparatus, methods and systems of object recognition are disclosed. Embodiments of the inventive subject matter generates map-altered image data according to an object-specific metric map, derives a metric-based descriptor set by executing an image analysis algorithm on the map-altered image data, and retrieves digital content associated with a target object as a function of the metric-based descriptor set.

Claims

1. An object recognition apparatus comprising:

a memory configured to store at least one object-specific metric map that maps an image color space to set of metric values related to a target object; and

a processor coupled with the memory and configured to operate as a recognition engine by executing the steps of;

obtaining the at least one object-specific metric map;

obtaining a digital representation of a scene and including image data;

generating map-altered image data according to the object-specific metric map executed on the image data;

deriving a metric-based descriptor set by executing an image analysis algorithm on the map-altered image data; and

retrieving digital content associated with the target object as a function of the metric-based descriptor set.

2. The apparatus of claim 1, wherein the object-specific metric map is associated with a class of target objects that includes the target object.

3. The apparatus of claim 2, wherein the object-specific metric map is associated with a hierarchy of target object classes.

4. The apparatus of claim 1, wherein the at least one object-specific metric map is configured to discriminate among target objects with respect to the image analysis algorithm.

5. The apparatus of claim 1, wherein the at least one object-specific metric map comprises a non-linear mapping of the image color space to the set of metric values.

6. The apparatus of claim 1, wherein the at least one object-specific metric map comprises a mapping that compresses a range of colors from the image color space to the set of metric values.

7. The apparatus of claim 1, wherein the at least one object-specific metric map comprises a mapping that stretched a range of colors from the image color space to the set of metric values.

8. The apparatus of claim 1, wherein the at least one object-specific metric map comprises a user defined mapping of the image color space to the set of metric values.

9. The apparatus of claim 1, wherein the step of obtaining the at least one object-specific metric map includes determining a contextual relevance of the at least one object-specific metric map based on the digital representation.

10. The apparatus of claim 9, wherein a contextual relevance is derived based on at least one of the following types of data within the digital representation:

a time, a location, a position, an orientation, a user preference, a news event, a motion, a gesture, an acceleration, a biometric, and an object attribute of the target object.

11. The apparatus of claim 1, wherein the object-specific metric map comprises pixel-level mapping of a RGB value to a metric value.

12. The apparatus of claim 1, wherein the image data comprises at least one of the following:

a still image, a video frame, a video frame delta, a video, a rendered image, a computer generated image, a projection, printed matter, and on-screen image.

13. The apparatus of claim 1, wherein the map-altered image data comprises at least one of the following:

a new image, a portion of a video, an overwritten image, and a modified image of the image data.

14. The apparatus of claim 1, wherein the map-altered image data comprises pixel-level data.

15. The apparatus of claim 14, wherein the pixel-level data is associated with a portion of the image data.

16. A method generating a metric-based recognition map comprising:

configuring a computing device to operate as an image processing engine;

receiving, by the image processing engine, image data representative of an object;

compiling, by the image processing engine, an initial object-specific metric map from at least a portion of the image data where the portion represents at least a portion of the object;

generating, by the image processing engine, a metric-based descriptor set by executing a feature identify algorithm on the portion of the image data as a function of the initial object-specific metric map; and

storing the metric-based descriptor set in a object recognition database.

17. The method of claim 16, further comprising adjusting the initial object-specific metric map to generate a new object-specific metric map by tuning metric values in a manner effective to enhance differentiation of descriptors generated by the feature identification algorithm as executed on the portion of the image data.

18. The method of claim 17, wherein adjusting the initial object-specific metric map includes accepting user input that alters at least some metric values within the initial object-specific metric map.

19. The method of claim 17, wherein adjusting the initial object-specific metric map includes the image processing engine recommending at least one a metric value that increases a confidence of a descriptor.

20. The method of claim 17, wherein adjusting the initial object-specific metric map includes the image processing engine automatically adjusting metric values of the initial object-specific metric map.

21. The method of claim 17, wherein the new metric-based map comprises a non-linear mapping from metric values within the initial metric-based map.

22. The method of claim 16, further comprising generating an object-specific color map based the object-specific metric map.

23. The method of claim 22, further comprising storing the object-specific color map as part of the metric-based descriptor set.

24. The method of claim 16, further comprising identifying at least one of a position and an orientation of an imaging device configure to capture the image data.

25. The method of claim 24, further comprising storing the at least one of the position and the orientation with the metric-based descriptor set.

26. The method of claim 16, further comprising removing specularity from the image data.

27. The method of claim 16, wherein the object-specific metric map comprises a pixel-level metric map.

28. The method of claim 16, wherein the metric-based descriptor set comprises lighting invariant descriptors.

29. The method of claim 16, wherein the metric-based descriptor set comprises metric-based invariant descriptors.

30. The method of claim 29, wherein the metric-based descriptors comprise metric-based scale invariant descriptors.

31. The method of claim 16, wherein the object comprises a physical object.

32. The method of claim 16, further comprising rendering the image data from a virtual object.

33. The method of claim 16, further comprising storing a key frame bundle that includes the metric-based descriptor set.