Metric-based recognition, systems and methods

• US 9,824,292 B2
• Filed: 04/13/2017
• Issued: 11/21/2017
• Est. Priority Date: 08/19/2013
• Status: Active Grant

• Current Assignee: Nant Holdings IP, LLC (NantWorks, LLC)
• Original Assignee: Nant Holdings IP, LLC (NantWorks, LLC)
• Inventors: Siddiqui, Matheen
• Primary Examiner(s): Seth, Manav
• Application Number:
  US15/487,031
• Publication Number:
  US 20170220893A1
• Time in Patent Office: 222 Days
• CPC Class Codes: G06F 16/3331 Query processing
• CPC Class Codes: G06F 16/51 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06F 16/7335 Graphical querying, e.g. query-by-region, query-by-sketch, query-by-trajectory, GUIs for designating a person/face/object as a query predicate end-user interface involving hot spots associated with the video H04N21/4725; end-user interface for selecting a Region of Interest H04N21/4728
• CPC Class Codes: G06F 16/785 using colour or luminescence
• CPC Class Codes: G06F 16/907 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06V 10/56 relating to colour
• CPC Class Codes: G06V 30/422 Technical drawings; Geographical maps
• CPC Class Codes: H04N 1/40012 Conversion of colour to monochrome

Abstract

Apparatus, methods and systems of object recognition are disclosed. Embodiments of the inventive subject matter generates map-altered image data according to an object-specific metric map, derives a metric-based descriptor set by executing an image analysis algorithm on the map-altered image data, and retrieves digital content associated with a target object as a function of the metric-based descriptor set.

Claims

1. An object recognition apparatus comprising:

a memory configured to store at least one object-specific metric map that maps an image color space to a set of metric values related to a target object; and

a processor coupled with the memory and configured to operate as a recognition engine by executing the steps of;

obtaining the at least one object-specific metric map;

obtaining a digital representation of a scene including image data;

generating map-altered image data according to the object-specific metric map executed on the image data;

deriving a metric-based descriptor set by executing an image analysis algorithm on the map-altered image data; and

using the metric-based descriptor set to retrieve digital content, comprising one or more virtual reality objects, augmented reality objects, or mixed reality objects, associated with the target object, and

wherein at least a portion of the digital content is superimposed over an image of a real-world object.

2. The apparatus of claim 1, wherein the object-specific metric map is associated with a class of target objects that includes the target object.

3. The apparatus of claim 2, wherein the object-specific metric map is associated with a hierarchy of target object classes.

4. The apparatus of claim 1, wherein the at least one object-specific metric map is configured to discriminate among target objects with respect to the image analysis algorithm.

5. The apparatus of claim 1, wherein the at least one object-specific metric map comprises a non-linear mapping of the image color space to the set of metric values.

6. The apparatus of claim 1, wherein the at least one object-specific metric map comprises a mapping that compresses a range of colors from the image color space to the set of metric values.

7. The apparatus of claim 1, wherein the at least one object-specific metric map comprises a mapping that stretched a range of colors from the image color space to the set of metric values.

8. The apparatus of claim 1, wherein the at least one object-specific metric map comprises a user defined mapping of the image color space to the set of metric values.

9. The apparatus of claim 1, wherein the step of obtaining the at least one object-specific metric map comprises using contextual data corresponding to the digital representation to select the object-specific metric map based on contextual relevance.

10. The apparatus of claim 9, wherein the contextual data includes at least one of the following types of data:

a time, a location, a position, an orientation, a user preference, a news event, a motion, a gesture, an acceleration, a biometric, and an object attribute of the target object.

11. The apparatus of claim 1, wherein the object-specific metric map comprises pixel-level mapping of a RGB value to a metric value.

12. The apparatus of claim 1, wherein the image data comprises at least one of the following:

a still image, a video frame, a video frame delta, a video, a rendered image, a computer generated image, a projection, printed matter, and on-screen image.

13. The apparatus of claim 1, wherein the map-altered image data comprises at least one of the following:

a new image, a portion of a video, an overwritten image, and a modified image of the image data.

14. The apparatus of claim 1, wherein the map-altered image data comprises pixel-level data.

15. The apparatus of claim 14, wherein the pixel-level data is associated with a portion of the image data.

16. The apparatus of claim 1, further configured to adjust the object-specific metric map to generate a new object-specific metric map by tuning metric values in a manner effective to enhance differentiation of descriptors generated by the image analysis algorithm as executed on the image data.

17. The apparatus of claim 16, further configured to accept user input that alters at least some metric values within the initial object-specific metric map.

18. The apparatus of claim 1, further configured to generate an object-specific color map based on the object-specific metric map.

19. The apparatus of claim 18, further configured to store the object-specific color map as part of the metric-based descriptor set.

20. The apparatus of claim 1, further configured to store a key frame bundle that includes the metric-based descriptor set.