FEATURE DENSITY OBJECT CLASSIFICATION, SYSTEMS AND METHODS

• US 20190012562A1
• Filed: 09/11/2018
• Published: 01/10/2019
• Est. Priority Date: 12/09/2013
• Status: Active Grant

• Current Assignee: Nant Holdings IP, LLC (NantWorks, LLC)
• Original Assignee: Nant Holdings IP, LLC (NantWorks, LLC)
• Inventors: Jaber, Mustafa, Sudol, Jeremi M., Song, Bing
• Granted Patent: US 10,671,879 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 18/23 Clustering techniques
• CPC Class Codes: G06T 2210/41 Medical
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/50 Depth or shape recovery
• CPC Class Codes: G06V 10/42 Global feature extraction by analysis of the whole pattern, e.g. using frequency domain transformations or autocorrelation
• CPC Class Codes: G06V 10/462 Salient features, e.g. scale invariant feature transforms [SIFT]
• CPC Class Codes: G06V 10/467 Encoded features or binary features, e.g. local binary patterns [LBP]
• CPC Class Codes: G06V 20/63 Scene text, e.g. street names

Abstract

A system capable of determining which recognition algorithms should be applied to regions of interest within digital representations is presented. A preprocessing module utilizes one or more feature identification algorithms to determine regions of interest based on feature density. The preprocessing modules leverages the feature density signature for each region to determine which of a plurality of diverse recognition modules should operate on the region of interest. A specific embodiment that focuses on structured documents is also presented. Further, the disclosed approach can be enhanced by addition of an object classifier that classifies types of objects found in the regions of interest.

Claims

1-37. -37. (canceled)

38. An object data processing system comprising:

at least one processor configured to execute;

a plurality of diverse recognition units stored on at least one non-transitory computer-readable storage medium, each recognition unit comprising at least one recognition algorithm and having feature density selection criteria; and

a data preprocessing unit executed by at least one processor, the data preprocessing unit comprising an invariant feature identification algorithm and configured to;

obtain a digital representation of a scene, the scene comprising one or more textual media;

generate a set of invariant features by applying the invariant feature identification algorithm to the digital representation;

cluster the set of invariant features into regions of interest in the digital representation of the scene, each region of interest having a region feature density;

classify, by a region classifier, at least one of the regions of interest according to object type as a function of attributes derived from the region feature density and the digital representation, wherein the at least one of the classified regions of interest corresponds to text; and

use a classification result corresponding to the at least one of the regions of interest to classify another of the regions of interest according to object type, wherein the another of the regions of interest corresponds to a region of interest for images.

39. The system of claim 38, wherein the preprocessing unit, based on the feature density selection criteria, determines that an OCR unit is applicable to the text, and that other recognition units are applicable to aspects of the photographs and to logos.

40. The system of claim 38, wherein a user creates a user profile for a camera-equipped smartphone that includes the information that the user is visually impaired, which causes prioritized execution of the OCR unit such that a text reader program begins reading the text to the user as quickly as possible.

41. The system of claim 40, further comprising an audio or tactile feedback mechanism that helps the user to position the smart phone relative to the text.

42. The system of claim 41, further comprising a “

hold still”

audio feedback signal that is sent to the user when the text is at the center of the captured scene.

43. The system of claim 38, wherein the digital representation comprises at least one of the following types of digital data:

image data, video data, and audio data.

44. The system of claim 38, wherein invariant feature identification algorithm comprises at least one of the following feature identification algorithms:

FAST, SIFT, FREAK, BRISK, Harris, DAISY, and MSER.

45. The system of claim 38, wherein the invariant feature identification algorithm includes at least one of the following:

edge detection algorithm, corner detection algorithm, saliency map algorithm, curve detection algorithm, a texton identification algorithm, and wavelets algorithm.

46. The system of claim 38, wherein at least one region of interest represents at least one physical object in the scene.

47. The system of claim 38, wherein at least one region of interest represents at least one textual media in the scene.

48. The system of claim 47, wherein the region of interest represents a document as the textual media.

49. The system of claim 48, wherein the region of interest represents a financial document.

50. The system of claim 48, wherein the region of interest represents a structured document.

51. The system of claim 38, wherein the plurality of diverse recognition algorithms include at least one of the following:

a template driven algorithm, a face recognition algorithm, an optical character recognition algorithm, a speech recognition algorithm, and an object recognition algorithm.

52. The system of claim 38, wherein the data preprocessing unit is further configured to assign each region of interest at least one recognition unit as a function of a scene context derived from the digital representation.

53. The system of claim 52, wherein the scene context includes at least one of the following types of data:

a location, a position, a time, a user identity, a news event, a medical event, and a promotion.

54. The system of claim 38, further comprising a mobile device comprising the plurality of diverse recognition units and the data preprocessing unit.

55. The system of claim 54, wherein the mobile device comprises at least one of the following:

a smart phone, a tablet, wearable glass, a toy, a vehicle, a computer, and a phablet.

56. The system of claim 38, further comprising a network-accessible server device comprising the plurality of diverse recognition units and the data preprocessing unit.

57. The system of claim 38, wherein the object type includes at least one of the following:

a face, an animal, a vehicle, a document, a plant, a building, an appliance, clothing, a body part, and a toy.

58. An object data processing system comprising:

at least one processor configured to execute;

a plurality of diverse recognition units stored on at least one non-transitory computer-readable storage medium, each recognition unit comprising at least one recognition algorithm and having feature density selection criteria; and

a data preprocessing unit executed by at least one processor, the data preprocessing unit comprising an invariant feature identification algorithm and configured to;

obtain a digital representation of a scene, the scene comprising one or more textual media;

generate a set of invariant features by applying the invariant feature identification algorithm to the digital representation;

cluster the set of invariant features into regions of interest in the digital representation of the scene, each region of interest having a region feature density;

classify, by a region classifier, at least one of the regions of interest according to object type as a function of attributes derived from the region feature density and the digital representation;

wherein the at least one of the classified regions of interest corresponds to text; and

use a classification result corresponding to the at least one of the regions of interest to classify another of the regions of interest according to object type, wherein the another of the regions of interest corresponds to a region of interest for images;

assign each region of interest at least one recognition unit from the plurality of diverse recognition units as a function of the region feature density of each region of interest and the feature density selection criteria of the plurality of diverse recognition units; and

configure the assigned recognition units to process their respective regions of interest, wherein the preprocessing unit, based on the feature density selection criteria, determines that an OCR unit is applicable to the text, and that other recognition units are applicable to aspects of the photographs and to logos.

59. A device comprising:

at least one processor configured to execute;

a plurality of diverse recognition units stored on at least one non-transitory computer-readable storage medium, each recognition unit comprising at least one recognition algorithm and having feature density selection criteria; and

a data preprocessing unit executed by at least one processor, the data preprocessing unit comprising an invariant feature identification algorithm and configured to;

obtain a digital representation of a scene, the scene comprising one or more textual media;

generate a set of invariant features by applying the invariant feature identification algorithm to the digital representation;

cluster the set of invariant features into regions of interest in the digital representation of the scene, each region of interest having a region feature density; and

classify, by a region classifier, at least one of the regions of interest according to object type as a function of attributes derived from the region feature density and the digital representation, wherein the at least one of the classified regions of interest corresponds to text; and

use a classification result corresponding to the at least one of the regions of interest to classify another of the regions of interest according to object type, wherein the another of the regions of interest corresponds to a region of interest for images.