OBJECT RECOGNITION TRAIT ANALYSIS SYSTEMS AND METHODS

• US 20150254510A1
• Filed: 03/02/2015
• Published: 09/10/2015
• Est. Priority Date: 02/28/2014
• Status: Active Grant

• Current Assignee: Nant Holdings IP, LLC (NantWorks, LLC)
• Original Assignee: Nant Vision, Inc, NantMobile, LLC (NantWorks, LLC), NantWorks, LLC
• Inventors: McKinnon, David, Wiacek, John, Sudol, Jeremi, Wnuk, Kamil, Song, Bing
• Granted Patent: US 9,633,042 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/50 of still image data
• CPC Class Codes: G06F 16/51 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06V 20/35 Categorising the entire scene, e.g. birthday party or wedding scene
• CPC Class Codes: G06V 30/242 Division of the character sequences into groups prior to recognition; Selection of dictionaries

Abstract

A system for analyzing scene traits in an object recognition ingestion ecosystem is presented. In some embodiment, a trait analysis engine analyzes a digital representation of a scene to derive one or more features. The features are compiled into sets of similar features with respect to a feature space. The engine attempts to discover which traits of the scene (e.g., temperature, lighting, gravity, etc.) can be used to distinguish the features for purposes of object recognition. When such distinguishing traits are found, an object recognition database is populated with object information, possibly indexed according to the similar features and their corresponding distinguishing traits.

Claims

1. An object recognition trait identification system comprising:

a computerized scene trait database storing a plurality of scene traits having values representative of a scene, the scene traits existing within an scene attribute space; and

a trait analysis engine coupled to the scene trait database and configured to;

derive a plurality of features by applying a recognition algorithm a digital representation of an object in a scene, the plurality of features existing in a feature space;

generate trait variances in the scene attribute space for similar features within the at least one similarity feature set and with respect to the scene traits;

select at least one distinguishing scene trait from the scene traits as a function of the trait variances where the at least one distinguishing scene trait differentiates features within the at least one similarity feature set; and

store each feature in the similarity feature set in an object recognition database along with metadata that includes the at least one distinguishing scene trait.

2. The system of claim 1 wherein the trait analysis engine is further configured to compile at least one similarity feature set comprising features of the plurality of features considered similar according to a similarity measure within the feature space.

3. The system of claim 1, wherein at least some of the scene traits are representative of a controlled environmental setting as the scene.

4. The system of claim 3, wherein the at least some of the scene traits comprise adjustable parameters with the controlled environment setting.

5. The system of claim 1, wherein at least some of the scene traits are representative of ambient sensor data of the scene.

6. The system of claim 3, wherein ambient sensor data is representative of external scene sensor data.

7. The system of claim 1, wherein the digital representation comprises at least one of the following data modalities:

image data, video data, audio data, 3D data, sensor data, location data, time data, biometric data, and game data.

8. The system of claim 1, wherein the digital representation of the object includes a representation of an object class.

9. The system of claim 1, wherein the digital representation of the object includes a representation at least one of the following:

a toy, a vehicle, a craft, an animal, a plant, a human, an appliance, a device, a document, a financial instrument, a game component, a poster, a newspaper, a magazine, and on screen content.

10. The system of claim 1, wherein the plurality of features includes at least one of the following features:

a descriptor, a key point, a signature, and an envelope.

11. The system of claim 1, wherein the recognition algorithm includes an image processing algorithm.

12. The system of claim 11, wherein the image processing algorithm includes at least one of the following:

SIFT, BRISK, SURF, FERNS, DAISY, GLOH, FAST, OCR, FREAK, HOG BRIEF, Harris Corners detector, EOG, TILT and facial recognition.

13. The system of claim 1, wherein the recognition algorithm includes a sound processing algorithm.

14. The system of claim 1, wherein the similarity measure includes at least one of the following:

a Euclidean distance, a Mahalanobis distance, a Hamming distance and a feature space variance.

15. The system of claim 2, wherein the trait analysis engine is further configured to compile at least one similarity feature set comprising features that fall within a threshold boundary with respect to the similarity measure.

16. The system of claim 1, wherein the trait variances are variances with respect to the scene attribute space.

17. The system of claim 1, wherein the trait analysis engine is further configured to select at least one distinguishing scene trait from the scene traits based on the scene traits having variances greater than a threshold.

18. The system of claim 1, wherein the trait analysis engine is further configured to construct at least one constellation of features from the at least one similarity feature set and the distinguishing scene traits.

19. The system of claim 18, wherein the at least one constellation of features is specific to the distinguishing scene trait.

20. The system of claim 18, wherein the at least one constellation of features includes multiple constellations where each constellation includes trait metadata representing one or more distinguishing scene traits.

21. The system of claim 1, further comprise the object recognition database.

22. The system of claim 18, wherein the object recognition database is configured to filter stored features according the metadata that includes the at least one distinguishing scene trait.

23. The system of claim 18, wherein the object recognition database is configure to return results sets as feature constellations based on the at least one distinguishing scene trait.

24. The system of claim 1, wherein the metadata further includes at least one of the following beyond the at least one distinguishing scene trait:

location data, context data, time data, profile data, identity data, manufacturing data, model data, promotion data, and healthcare data.

25. The system of claim 1, wherein the distinguishing scene trait includes gravity.

26. The system of claim 1, wherein the distinguishing scene trait includes at least one of the following:

a wireless signal, an electro-magnetic field, a temperature, a gradient, a physical location, a relative position, an orientation, a restriction, and a time.

27. A computerized object recognition scene trait analysis system for enhancing identification of objects in a scene capture, the scene capture comprising (i) a digital representation of an image capture of a scene by an image sensor and (ii) one or more scene trait values captured by one or more scene trait sensors at the time of the image capture;

the system comprising;

a scene variance analyzer comprising one or more computers configured to;

compute scene trait variances, the scene trait variances corresponding to variances of scene trait values across a plurality of scene captures; and

determine distinguishing scene traits based on scene trait variances that are above a threshold value;

and

a weight processor comprising one or more computers configured to, for each of a plurality of scene captures;

determine relative weighting for the distinguishing scene traits; and

generate metadata using the relative weighting;

wherein the system is configured to associate generated metadata corresponding to a particular scene capture with features derived from a digital representation corresponding to the particular scene capture.

28. The system of claim 27 wherein the scene variance analyzer is configured to identify as a distinguishing trait a first trait having a greater variance across a plurality of scene captures than a second trait.

29. The system of claim 27 wherein the scene variance analyzer is configured to identify as distinguishing traits two or more traits that have strong covariance across a plurality of scene captures, strong covariance being defined as being a covariance that is higher than a threshold value.

30. The system of claim 27 further comprising:

a feature set analyzer comprising one or more computers configured to identify similarity feature sets from a plurality of features derived from digital representations of a plurality of image captures, the plurality of features being represented in a feature space corresponding to a recognition algorithm and the similarity feature sets being selected based on a similarity measure.

31. The system of claim 30, wherein the similarity measure includes at least one of the following:

a Euclidean distance, a Mahalanobis distance, a Hamming distance and a feature space variance.

32. The system of claim 30 wherein the recognition algorithm is an image processing algorithm including at least one of the following:

SIFT, BRISK, SURF, FERNS, DAISY, GLOH, FAST, OCR, FREAK, HOG BRIEF, Harris Corners detector, EOG, TILT and facial recognition.

33. The system of claim 30 configured such that the variance analyzer analyzes only scene traits associated with scene captures corresponding to digital representations from which features in the similarity feature sets were derived.

34. The system of claim 27 coupled to communicate the generated metadata and associated features to an object recognition database.

35. A computer program product embedded in a non-transitory computer readable medium comprising instructions executable by a computer processor to perform object recognition scene trait analysis to enhance identification of objects in a scene capture, the scene capture comprising (i) a digital representation of an image capture of a scene by an image sensor and (ii) one or more scene trait values captured by one or more scene trait sensors at the time of the image capture;

the instructions being executable by a computer processor to execute processing comprising;

computing scene trait variances, the scene trait variances corresponding to variances of scene trait values across a plurality of scene captures;

identifying distinguishing scene traits based on scene trait variances that are above a threshold value;

determining relative weighting for the distinguishing scene traits;

generating metadata using the relative weighting; and

associating generated metadata corresponding to a particular scene capture with features derived from a digital representation corresponding to the particular scene capture.

36. The computer program product of claim 35 wherein the instructions are executable by the processor to identify as a distinguishing trait a first trait having a greater variance across a plurality of scene captures than a second trait.

37. The computer program product of claim 35 wherein the instructions are executable by the processor to identify as distinguishing traits two or more traits that have strong covariance across a plurality of scene captures, strong covariance being defined as higher than a threshold value.

38. The computer program product of claim 35 wherein the instructions are executable by the processor to identify similarity feature sets from a plurality of features derived from digital representations of a plurality of image captures, the plurality of feature being represented in a feature space corresponding to an recognition algorithm, and the similarity feature sets being selected based on a similarity measure.

39. The computer program product of claim 38 wherein the similarity measure includes at least one of the following:

a Euclidean distance, a Mahalanobis distance, a Hamming Distance and a feature space variance.

40. The computer program product of claim 38 wherein the recognition algorithm is an image processing algorithm including at least one of the following:

SIFT, BRISK, SURF, FERNS, DAISY, GLOH, FAST, OCR, FREAK, HOG BRIEF, Harris Corners detector, EOG, TILT and facial recognition.

41. The computer program product of claim 38 wherein the instructions are executable such that only variances for scene traits associated with scene captures corresponding to digital representations from which features in the similarity feature sets were derived are analyzed.

42. The computer program product of claim 35 further comprising instructions for communicating the generated metadata and associated features to an object recognition database.