Object recognition trait analysis systems and methods

• US 10,013,612 B2
• Filed: 03/16/2017
• Issued: 07/03/2018
• Est. Priority Date: 02/28/2014
• Status: Active Grant

• Current Assignee: Nant Holdings IP, LLC (NantWorks, LLC)
• Original Assignee: Nant Holdings IP, LLC (NantWorks, LLC)
• Inventors: McKinnon, David, Wiacek, John, Sudol, Jeremi, Wnuk, Kamil, Song, Bing
• Primary Examiner(s): Mehta, Bhavesh
• Assistant Examiner(s): Youssef, Menatoallah
• Application Number:
  US15/461,197
• Publication Number:
  US 20170200049A1
• Time in Patent Office: 474 Days
• CPC Class Codes: G06F 16/50 of still image data
• CPC Class Codes: G06F 16/51 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06V 20/35 Categorising the entire scene, e.g. birthday party or wedding scene
• CPC Class Codes: G06V 30/242 Division of the character sequences into groups prior to recognition; Selection of dictionaries

Abstract

A system for analyzing scene traits in an object recognition ingestion ecosystem is presented. In some embodiment, a trait analysis engine analyzes a digital representation of a scene to derive one or more features. The features are compiled into sets of similar features with respect to a feature space. The engine attempts to discover which traits of the scene (e.g., temperature, lighting, gravity, etc.) can be used to distinguish the features for purposes of object recognition. When such distinguishing traits are found, an object recognition database is populated with object information, possibly indexed according to the similar features and their corresponding distinguishing traits.

Claims

1. A computerized object recognition scene trait analysis system for enhancing identification of objects in a scene capture, the scene capture comprising (i) a digital representation of an image capture of a scene by an image sensor and (ii) one or more scene trait values captured by one or more scene trait sensors at the time of the image capture;

the system comprising;

a scene variance analyzer comprising one or more computers configured to;

compute scene trait variances, the scene trait variances corresponding to variances of scene trait values across a plurality of scene captures; and

determine distinguishing scene traits based on scene trait variances that are above a threshold value for the plurality of scene captures;

and

a weight processor comprising one or more computers configured to, for each of the plurality of scene captures;

determine relative weighting for the distinguishing scene traits; and

generate metadata using the relative weighting;

wherein the system is configured to associate generated metadata corresponding to a particular scene capture with features derived from a digital representation corresponding to the particular scene capture, transmit content information including the generated metadata and features to an object recognition system comprising an object recognition database, the object recognition system recognizing objects based on content information including the generated metadata and features, and store object recognition results from the object recognition system thereby improving accuracy and reducing latency of digital image searches.

2. The system of claim 1 wherein the scene variance analyzer is configured to identify as a distinguishing trait a first trait having a greater variance across a plurality of scene captures than a second trait.

3. The system of claim 1 wherein the scene variance analyzer is configured to identify as distinguishing traits two or more traits that have strong covariance across a plurality of scene captures, strong covariance being defined as being a covariance that is higher than a threshold value.

4. The system of claim 1 further comprising:

a feature set analyzer comprising one or more computers configured to identify similarity feature sets from a plurality of features derived from digital representations of a plurality of image captures, the plurality of features being represented in a feature space corresponding to a recognition algorithm and the similarity feature sets being selected based on a similarity measure.

5. The system of claim 4, wherein the similarity measure includes at least one of the following:

a Euclidean distance, a Mahalanobis distance, a Hamming distance and a feature space variance.

6. The system of claim 4 wherein the recognition algorithm is an image processing algorithm including at least one of the following:

SIFT, BRISK, SURF, FERNS, DAISY, GLOH, FAST, OCR, FREAK, HOG BRIEF, Harris Corners detector, EOG, TILT and facial recognition.

7. The system of claim 4 configured such that the scene variance analyzer analyzes only scene traits associated with scene captures corresponding to digital representations from which features in the similarity feature sets were derived.

8. The system of claim 1 coupled to communicate the generated metadata and associated features to an object recognition database.

9. A computer program product embedded in a non-transitory computer readable medium comprising instructions executable by a computer processor to perform object recognition scene trait analysis to enhance identification of objects in a scene capture, the scene capture comprising (i) a digital representation of an image capture of a scene by an image sensor and (ii) one or more scene trait values captured by one or more scene trait sensors at the time of the image capture;

the instructions being executable by a computer processor to execute processing comprising;

computing scene trait variances, the scene trait variances corresponding to variances of scene trait values across a plurality of scene captures;

identifying distinguishing scene traits based on scene trait variances that are above a threshold value for the plurality of scene captures;

determining relative weighting for the distinguishing scene traits;

generating metadata using the relative weighting; and

associating generated metadata corresponding to a particular scene capture with features derived from a digital representation corresponding to the particular scene capture, transmitting content information including the generated metadata and features to an object recognition system comprising an object recognition database, the object recognition sytem recognizing objects based on content information including the generated metadata and features, and receiving and storing object recognition results from the object recognition system, thereby improving accuracy and reducing latency of digital image searches.

10. The computer program product of claim 9 wherein the instructions are executable by the processor to identify as a distinguishing trait a first trait having a greater variance across a plurality of scene captures than a second trait.

11. The computer program product of claim 9 wherein the instructions are executable by the processor to identify as distinguishing traits two or more traits that have strong covariance across a plurality of scene captures, strong covariance being defined as higher than a threshold value.

12. The computer program product of claim 9 wherein the instructions are executable by the processor to identify similarity feature sets from a plurality of features derived from digital representations of a plurality of image captures, the plurality of features being represented in a feature space corresponding to a recognition algorithm, and the similarity feature sets being selected based on a similarity measure.

13. The computer program product of claim 12 wherein the similarity measure includes at least one of the following:

a Euclidean distance, a Mahalanobis distance, a Hamming distance and a feature space variance.

14. The computer program product of claim 12 wherein the recognition algorithm is an image processing algorithm including at least one of the following:

SIFT, BRISK, SURF, FERNS, DAISY, GLOH, FAST, OCR, FREAK, HOG BRIEF, Harris Corners detector, EOG, TILT and facial recognition.

15. The computer program product of claim 12 wherein the instructions are executable such that only variances for scene traits associated with scene captures corresponding to digital representations from which features in the similarity feature sets were derived are analyzed.

16. The computer program product of claim 9 further comprising instructions for communicating the generated metadata and associated features to an object recognition database.

17. An object recognition trait identification system comprising:

a computerized scene trait database storing a plurality of scene traits having values representative of a scene, the scene traits existing within a scene attribute space;

a trait analysis engine coupled to the scene trait database comprising;

an object recognition processor comprising circuitry configured by executable instruction code to accept at least a portion of a digital representation as input and derive a plurality of features by applying a recognition algorithm to the digital representation of an object in the scene, the plurality of features existing in a feature space;

a feature set analyzer comprising circuitry configured by executable instruction code to generate trait variances with respect to the scene traits in the scene attribute space for features within at least one similarity feature set for a plurality of scenes, wherein a feature corresponds to at least one scene trait;

a trait variance analyzer comprising circuitry configured by executable instruction code to select at least one distinguishing scene trait from the scene traits as a function of the trait variances where the at least one distinguishing scene trait differentiates features within the at least one similarity feature set; and

a weight processor comprising circuitry configured to;

(1) determine relative weighting for the at least one distinguishing scene trait; and

(2) generate metadata using the relative weighting;

and

an object recognition system comprising an object recognition database coupled to the trait analysis engine and storing each feature in the similarity feature set along with metadata that includes the at least one distinguishing scene trait, wherein the trait analysis engine transmits content information including the generated metadata and features to the object recognition system, and the object recognition system recognizes objects based on content information including the generated metadata and features and transmits object recognition results to the trait analysis engine, thereby improving accuracy and reducing latency of digital image searches.

18. The system of claim 17 wherein the trait variance analyzer is configured to identify as a distinguishing trait a first trait having a greater variance across a plurality of scene captures than a second trait.

19. The system of claim 17 wherein the trait variance analyzer is configured to identify as distinguishing traits two or more traits that have strong covariance across a plurality of scene captures, strong covariance being defined as being a covariance that is higher than a threshold value.

20. The system of claim 17 wherein the similarity feature set comprises a plurality of features derived from digital representations of a plurality of image captures, the plurality of features being represented in a feature space corresponding to a recognition algorithm and the similarity feature sets being selected based on a similarity measure.