Trainable system to search for objects in images

• US 6,421,463 B1
• Filed: 03/31/1999
• Issued: 07/16/2002
• Est. Priority Date: 04/01/1998
• Status: Expired due to Fees

• Current Assignee: Massachusetts Institute of Technology
• Original Assignee: Massachusetts Institute of Technology
• Inventors: Oren, Michael, Papageorgiou, Constatine P., Sinha, Pawan, Poggio, Tomaso
• Primary Examiner(s): Mancuso, Joseph
• Assistant Examiner(s): Dastouri, Mehrdad
• Application Number:
  US09/282,742
• Time in Patent Office: 1,203 Days
• Field of Search: 382/128, 382/132, 382/224, 382/209, 382/217, 382/218, 382/279, 382/190, 382/195, 375/240.11
• US Class Current: 382/224
• CPC Class Codes: G06V 10/431 Frequency domain transformation; Autocorrelation
• CPC Class Codes: G06V 2201/11 Technique with transformation invariance effect
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation

Abstract

A trainable object detection system and technique for detecting objects such as people in static or video images of cluttered scenes is described. The described system and technique can be used to detect highly non-rigid objects with a high degree of variability in size, shape, color, and texture. The system learns from examples and does not rely on any a priori (hand-crafted) models or on motion. The technique utilizes a wavelet template that defines the shape of an object in terms of a subset of the wavelet coefficients of the image. It is invariant to changes in color and texture and can be used to robustly define a rich and complex class of objects such as people. The invariant properties and computational efficiency of the wavelet template make it an effective tool for object detection.

Claims

1. A system for processing an image, the system comprising:

an image database which includes a set of example images at least some of which include objects of interest; and

a classifier, coupled to said image database, to receive an image from said database and to process said image, said classifier including a wavelet template generator, said wavelet template generator comprising;

(1) a wavelet scale selector to select at least one wavelet scale which corresponds to at least one feature of the object of interest;

(2) a wavelet coefficient processor for computing wavelet coefficients at each of the at least one wavelet scales; and

(3) a normalization processor to receive the wavelet coefficients and to normalize the wavelet coefficients such that an average value of the wavelet coefficients of a random pattern is a predetermined value.

2. The system of claim 1 further comprising:

an image preprocessor coupled to receive images from said image database and to provide at least a portion of an image to said classifier, said image pre-processor for moving a window across an image selected from the database; and

a resizing preprocessor for scaling an image from a first size to a second size at a predetermined increment and for providing each of the scaled images to said classifier.

3. The system of claim 1 further comprising a training system coupled to said classifier, said training system comprising:

an image database containing a first plurality of positive example images and a second plurality of negative example images; and

a quadratic programming solver wherein said training system provides negative example images to said classifier and any object detected by said classifier in the negative example images are identified as false positive images and are added to the second plurality of negative example images.

4. The system of claim 3 further comprising:

an image retrieval device coupled to said image database for retrieving images from said image database;

a relationship processor, coupled to said image retrieval device, said relationship processor for identifying relationships between image characteristics of images retrieved from said database;

a wavelet template generator, coupled to said relationship processor, said wavelet template generator for encoding relationships between characteristics which are consistent between images retrieved from said database as a wavelet image template; and

an image detector for applying the wavelet image template to images in said image database to detect images belonging to a particular class of images stored in said image database.

5. The image processing system of claim 4 wherein said image detector detects novel images having relative relationships between selected image regions thereof which are consistent with the relative relationships encoded in the wavelet image template.

6. The image processing system of claim 5 wherein said wavelet template generator comprises:

a wavelet scale selector to select at least one wavelet scale which corresponds to at least one feature of the object of interest;

a wavelet coefficient processor for computing wavelet coefficients at each of the least one wavelet scales; and

a normalization processor to receive the wavelet coefficients and to normalize the wavelet coefficients such that an average value of the wavelet coefficients of a random pattern is a predetermined value.

7. A method of generating a model for use in an image processing system, the method comprising the steps of:

(a) providing a set of example images at least some of which include objects of interest to a classifier;

(b) selecting at least one wavelet scale which corresponds to at least one feature of the object of interest;

(c) computing wavelet coefficients at each of the least one wavelet scales;

(d) normalizing the wavelet coefficients such that an average value of coefficients of a random pattern is a predetermined value.

8. The Method of claim 7 wherein the step of normalizing the wavelet coefficients includes the steps of:

(d1) computing a class average value for each wavelet coefficient;

(d2) normalizing each wavelet coefficient by its class average; and

(d3) averaging the normalized coefficients over example images to provide a normalized average value for each coefficient.

9. The method of claim 8 wherein the step of averaging the normalized coefficients over example images includes the step of averaging the normalized coefficients over the entire set of example images.

10. The method of claim 7 further comprising the steps of:

comparing normalized wavelet coefficient values; and

selecting coefficient values which capture one or more significant characteristics of the object of interest.

11. The method of claim 10 wherein the step of selecting coefficient values includes the step of selecting a number of coefficients which is less than the total number of coefficients.

12. The method of claim 11 wherein the wavelet coefficients correspond to vertical, horizontal and diagonal wavelet coefficients.

13. The method of claim 10 wherein the step of selecting at least one wavelet scale which corresponds to at least one feature of the object of interest includes the step of selecting a plurality of wavelet scales, each of the wavelet scales selected to correspond to a corresponding plurality of characteristics of the object.

14. The method of claim 9 wherein the example images correspond to grey-scale example images.