Method and apparatus for object recognition

• US 6,038,337 A
• Filed: 03/29/1996
• Issued: 03/14/2000
• Est. Priority Date: 03/29/1996
• Status: Expired due to Term

• Current Assignee: NEC Corporation
• Original Assignee: NEC Research Institute, Inc. (NEC Corporation)
• Inventors: Lawrence, Stephen Robert, Giles, C. Lee
• Primary Examiner(s): Couso, Jose L.
• Assistant Examiner(s): Dang, Duy M.
• Application Number:
  US08/625,028
• Time in Patent Office: 1,446 Days
• Field of Search: 382/156, 382/118, 382/115, 382/155, 382/157, 382/159, 382/190, 382/195, 382/181, 382/224, 382/226, 382/205, 382/299, 382/298, 382/203, 395/21-27
• US Class Current: 382/156
• CPC Class Codes: G06V 10/454 Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]

Abstract

A hybrid neural network system for object recognition exhibiting local image sampling, a self-organizing map neural network, and a hybrid convolutional neural network. The self-organizing map provides a quantization of the image samples into a topological space where inputs that are nearby in the original space are also nearby in the output space, thereby providing dimensionality reduction and invariance to minor changes in the image sample, and the hybrid convolutional neural network provides for partial invariance to translation, rotation, scale, and deformation. The hybrid convolutional network extracts successively larger features in a hierarchical set of layers. Alternative embodiments using the Karhunen-Loeve transform in place of the self-organizing map, and a multi-layer perceptron in place of the convolutional network are described.

Claims

1. A method for recognizing an object having an object having a set of features, the method comprising the steps of:

sampling a plurality of local regions of an image of the object by stepping a local window in discrete steps over the image to generate a set of sample vectors;

reducing the dimensionality of the sample vectors, including (a) training a self-organizing map on the set of sample vectors, (b) stepping, in discrete steps, the local window over any images contained in any training and any test sets, thereby producing a resultant local image sample at each one of the discrete steps, and (c) generating a new training set in an output space of the self-organizing map by passing the resulting local image samples through the self-organizing map;

extracting the features of the object from the dimensionally reduced sample vectors, including training a convolutional neural network on the newly generated training set; and

classifying the object into a known classification using the extracted features.

2. The method according to claim 1, wherein the reducing step further comprises a Karhunen-Loeve transform.

3. An apparatus for recognizing an object, the apparatus comprising:

means for local image sampling which samples a plurality of local regions of an image of the object by stepping a local window in discrete steps over the image to generate a set of sample vectors;

means for dimensionality reduction which trains a self-organizing map on the set of sample vectors, and steps, in discrete steps, the local window over any images contained in any training and any test sets, thereby producing a resultant local image sample at each one of the discrete steps, and generates a new training set in an output space of the self-organizing map by passing the resulting local image samples through the self-organizing map;

means for feature extraction which extracts features of the object from the sample vectors and trains a convolutional neural network on the newly generated training set;

means for classification which classifies the object into a known classification using the extracted features.

4. The object recognition apparatus according to claim 3, wherein the feature extraction means successively transforms the output of the local image sampling means into a plurality of new representations which successively correspond to a set of higher level features of the image.

5. The object recognition apparatus according to claim 3, wherein the dimensionality reduction means further comprises means for performing a Karhunen-Loeve transform.

6. The object recognition apparatus according to claim 3, wherein the classification means further estimates a probability that the image contains an individual member of a group based on the extracted features, and wherein the apparatus further comprises means for confidence estimation which estimates a confidence that the individual member of the group is contained within the image.

7. The object recognition apparatus according to claim 6, wherein the confidence estimation means generates the confidence according to y_m (y_m -y_2m) where y_m is a first maximum output and y_2m is a second maximum output of the classification means.

8. The object recognition apparatus according to claim 3, wherein the classification means is a multi layer perceptron style classifier.

9. The object recognition apparatus according to claim 3, wherein the classification means is a nearest neighbor system.

10. An apparatus for recognizing an object, the apparatus comprising:

means for local image sampling which samples a plurality of local regions of an image of the object by stepping a local window in discrete steps over the image to generate a set of sample vectors;

means for dimensionality reduction wherein the set of sample vectors is mapped into a space of lower dimension wherein similar points in the input space are mapped to similar points in the output space;

means for feature extraction which extracts features from the sample vectors;

means for classification which estimates a probability that the image contains an individual member of a group based on the extracted features;

means for confidence estimation which estimates a confidence that the individual member of the group is contained within the image;

wherein the feature extraction means successively transforms the output of the local image sampling means into a plurality of new representations which successively correspond to a set of higher level features of the image; and

wherein the confidence estimation means generates the confidence according to y_m (y_m -y_2m), where y_m is a first maximum output and y_2m is a second maximum output of the classification means.