Image classification and search

• US 8,200,025 B2
• Filed: 12/07/2007
• Issued: 06/12/2012
• Est. Priority Date: 12/07/2007
• Status: Active Grant

• Current Assignee: University of Ottawa
• Original Assignee: University of Ottawa
• Inventors: Woodbeck, Kris
• Primary Examiner(s): Carter, Aaron W
• Application Number:
  US11/952,643
• Publication Number:
  US 20090148068A1
• Time in Patent Office: 1,649 Days
• Field of Search: 382/159, 382224-228, 706/20
• US Class Current: 382/224
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06V 10/454 Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
• CPC Class Codes: G06V 30/242 Division of the character sequences into groups prior to recognition; Selection of dictionaries

Abstract

An electronic image classification and search system and method are provided. Images are processed to determine a plurality of simple feature descriptors based upon characteristics of the image itself. The simple feature descriptors are grouped into complex features based upon the orientation of the simple feature descriptors. End-stopped complex feature descriptors and complex feature descriptors at multiple orientations are grouped into hypercomplex feature descriptors. Hypercomplex resonant feature descriptor clusters are generated by linking pairs of hypercomplex feature descriptors. Feature hierarchy classification can then be performed by adaptive resonance on feature descriptors and classifier metadata associated with the image can then be generated to facilitate indexing and searching of the image within a hierarchical image database.

Claims

1. A method classifying an image comprising the steps of:

determining a plurality of simple feature descriptors based upon characteristics of the image;

grouping simple feature descriptors into complex features wherein the simple feature descriptors are grouped based upon orientation;

grouping end-stopped complex feature descriptors and complex feature descriptors at multiple orientations into hypercomplex feature descriptors;

generating a hypercomplex resonant feature descriptor cluster by linking pairs of hypercomplex feature descriptors;

performing feature hierarchy classification by adaptive resonance on feature descriptors; and

generating classifier metadata associated with the image.

2. The method of claim 1 further comprising the steps of:

determining redundant feature descriptors which overlap; and

removing the redundant feature descriptors.

3. The method of claim 1 wherein the simple feature descriptors are determined by applying Gabor filters to the image.

4. The method of claim 3 wherein the step of determining the feature descriptors is repeatedly performed to determine optimal feature descriptor acquisition parameters of the Gabor filters.

5. The method of claim 1 further comprising the step of:

performing hierarchical feature resonance against images already classified for a given object class, and utilizing the determined feature descriptors prior to the step of hierarchical classification.

6. The method of claim 5 further comprising the steps of:

storing classification metadata in an image database; and

storing index information for identifying the location of the image.

7. The method of claim 6 further comprising the step of storing image label information in the image database wherein the image label comprises keywords associated with the content of the image.

8. The method of claim 1 further comprising retrieving images from an image database having classifiers of the same classification metadata for display to a user.

9. The method of claim 1 wherein the classifiers are grouped into branch classifiers and dependent leaf classifiers in an image hierarchy database.

10. The method of claim 9 wherein when a classifier exceeds a pre-defined number of referenced images, performing the steps of:

isolating multiple principal feature descriptor components of the original leaf classifier into a new branch classifier; and

generating new leaf classifiers based upon the differential feature descriptors of the original leaf classifier.

11. The method of claim 1 wherein each of the plurality of feature descriptors is determined independently by a processing unit operating capable of Single Instruction Multi Data (SIMD) processing.

12. A system for image classification and searching comprising:

a processor;

a memory containing instructions for;

a feature extractor module for;

determining a plurality of feature descriptors based upon characteristics of the image;

grouping feature descriptors into complex features wherein the feature descriptors are grouped based upon orientation;

grouping end-stopped complex features and complex features at multiple orientations into hypercomplex features;

generating a hypercomplex resonant feature cluster by linking pairs of hypercomplex features;

a resonant classifier module for;

performing feature hierarchy classification by adaptive resonance on feature descriptors;

generating feature image classifiers based upon the adaptive resonance classifiers;

an indexing module for indexing the image within the classifier hierarchy;

a storage device containing an image database comprising;

classifier metadata associated with images;

indexing data comprising image location data.

13. The system of claim 12 wherein the storage device further comprises image labels comprising keywords associated with content of the images.

14. The system of claim 12 wherein a Single Instruction Multi Data (SIMD) processor and the feature extractor module operates by utilizing processing units for determining features in parallel.

15. The system of claim 12 wherein the feature extractor module further comprises performing hierarchical feature resonance over all visual fields for a given object class.

16. The system of claim 12 wherein the classifier metadata is stored in a branch/leaf hierarchy, each branch associated with images having similar principal components based upon the feature descriptors and the leaf classifiers reference images based upon differential feature descriptors relative to the branch.

17. The system of claim 16 wherein when a classifier exceeds a pre-defined number of referenced images the hierarchy is expanded by:

isolating multiple principal feature descriptor components of the original leaf classifier into a new branch classifier; and

generating new leaf classifiers based upon the differential feature descriptors of the original leaf classifier.

18. A system for image classification and searching comprising:

a processor;

a memory containing instructions comprising;

determining a plurality of simple feature descriptors based upon characteristics of the image;

grouping simple feature descriptors into complex features wherein the simple feature descriptors are grouped based upon orientation;

grouping end-stopped complex feature descriptors and complex feature descriptors at multiple orientations into hypercomplex feature descriptors;

generating a hypercomplex resonant feature descriptor cluster by linking pairs of hypercomplex feature descriptors;

performing feature hierarchy classification by adaptive resonance on feature descriptors; and

generating classifier metadata associated with the image.

19. A non-transitory computer readable medium containing instructions for image classification, the instructions which when executed on a processor perform the steps of:

determining a plurality of simple feature descriptors based upon characteristics of the image;

grouping simple feature descriptors into complex feature descriptors wherein the simple feature descriptors are grouped based upon orientation;

grouping end-stopped complex feature descriptors and complex feature descriptors at multiple orientations into hypercomplex feature descriptors;

generating a hypercomplex resonant feature descriptor cluster by linking pairs of hypercomplex feature descriptors;

performing feature hierarchy classification by adaptive resonance on feature descriptors; and

generating classifier metadata associated with the image.