Method and apparatus for image-based object detection and tracking

• US 6,434,254 B1
• Filed: 10/30/1996
• Issued: 08/13/2002
• Est. Priority Date: 10/31/1995
• Status: Expired due to Term

• Current Assignee: SRI International, Inc.
• Original Assignee: Sarnoff Corporation (SRI International, Inc.)
• Inventors: Wixson, Lambert Ernest
• Primary Examiner(s): Johns, Andrew W.
• Assistant Examiner(s): Le, Hieu
• Application Number:
  US08/742,432
• Time in Patent Office: 2,113 Days
• Field of Search: 382/103, 382/107, 382/173, 382/270, 348/142, 348/152, 348/154, 348/155, 348/169, 348/416, 348/701, 348/700, 348/699, 348/122, 348/143
• US Class Current: 382/103
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20016 Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 2207/30236 Traffic on road, railway or crossing
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: G06V 10/255 Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
• CPC Class Codes: G06V 20/54 of traffic, e.g. cars on the road, trains or boats
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G08B 13/19604 involving reference image or background adaptation with time to compensate for changing conditions, e.g. reference image update on detection of light level change
• CPC Class Codes: G08B 13/19608 Tracking movement of a target, e.g. by detecting an object predefined as a target, using target direction and or velocity to predict its new position
• CPC Class Codes: G08G 1/04 using optical or ultrasonic detectors

Abstract

A method and apparatus for detecting objects from a sequence of images of a scene containing an object by using two distinct methods for object detection. One is suited for well-lit scenes (e.g., daytime), while the other is suitable for poorly-lit scenes (e.g., nighttime) where the objects have lights mounted on them. Further, the method and apparatus are adaptive to the image statistics of the objects being detected, and can be programmed to filter out “weak” detections that lie below a certain percentile in the observed statistical distribution of image measures. The specific percentile is determined based on whether the scene has been determined to be well- or poorly-lit and whether it contains shadows or not.

Claims

1. A method of identifying and tracking objects within a scene represented by a sequence of images comprising the steps of:

generating a reference image containing background information of the scene;

selecting an image from said sequence of images as a two dimensional object image;

converting said two dimensional object image into a one-dimensional strip of object image values;

converting said reference image into a one-dimensional reference strip of reference values;

producing a difference strip of difference values by comparing the reference values in the reference strip to the object image values in the one-dimensional strip; and

processing said difference strip to identify moving objects in said scene.

2. The method of claim 1 wherein said processing step further comprises the steps of:

classifying each difference value in said difference strip as either background or non-background.

3. The method of claim 1 wherein said producing step comprises the steps of:

generating a brightness difference value for each object image value in said one-dimensional strip by computing a reference brightness value for each reference value in said reference strip, computing an object image brightness value for each object image value in said one dimensional strip, and comparing each of said reference brightness values to each corresponding object image brightness value;

generating an energy difference value for each object image value in said one-dimensional strip by computing a reference energy value for each reference value in said reference strip, computing an object image energy value for each object image value in said one dimensional strip, and comparing each said reference energy value to each corresponding object image energy value; and

comparing the brightness difference value to a brightness difference threshold value and the energy difference value to a energy difference threshold value to classify each object image value as either background or non-background values.

4. The method of claim 3 further comprising the steps of:

determine each bright non-background object image value in said one dimensional strip having a brightness maxima;

identifying a brightness region of object image values adjacent to each object image value being a brightness maxima;

determining a brightness change within a subregion of each region, if a change in object image value brightness between adjacent object image values that is greater than a predefined threshold occurs, label the subregion values an object.

5. The method of claim 4 further comprising the step of detecting a lack of motion of a previously detected object to identify a falsely detected object.

6. The method of claim 3 wherein said producing step further comprises the steps of:

for each object image value that is classified as a non-background value, comparing the brightness difference value to a second brightness threshold value to classify the object image value as either bright or dark.

7. The method of claim 6 further comprising the steps of:

comparing the object image energy value of each of the bright non-background object image values to an energy threshold value;

discarding each bright non-background object image value that has an object image energy value that is less than the energy threshold value; and

identifying remaining pixels as object pixels.

8. The method of claim 7 further comprising the steps of:

tracking the object using predictive estimation of object position.

9. The method of claim 7 wherein the energy threshold value is determined by computing a statistical distribution of energy levels for the object image energy values.

10. The method of claim 7 further comprising the step of detecting a lack of motion of a previously detected object to identify a falsely detected object.

11. The method of claim 8 further comprising the step of detecting a lack of motion of a previously detected object to identify a falsely detected object.

12. A method of identifying and tracking objects within a scene represented by a sequence of images comprising the steps of:

generating a reference image containing background information of the scene;

selecting an image from said sequence of images as an object image;

generating a brightness difference value for each pixel in said object image by computing a reference brightness value for each reference pixel in said reference image, computing an image brightness value for each pixel in said object image, and comparing each of said reference brightness values to each corresponding image brightness value;

generating an energy difference value for each pixel in said object image by computing a reference energy value for each reference pixel in said reference image, computing an image energy value for each pixel in said object image, and comparing each said reference energy value to each corresponding image energy value;

comparing the brightness difference value to a brightness difference threshold value and the energy difference value to a energy difference threshold value to classify each pixel in said object image as either background or non-background values; and

for each object image value that is classified as a non-background value, comparing the brightness difference value to a second brightness threshold value to classify each non-background pixel as bright or dark; and

processing each bright non-background pixel to determine if the pixel is an object.

13. The method of claim 12 further comprising the steps of:

comparing an energy value of each of the bright non-background pixels to an energy threshold value;

discarding each bright non-background pixel that does not meet the energy threshold value; and

identifying remaining pixels as object pixels.

14. The method of claim 12 wherein said reference image is a one dimensional reference strip and said object image is a one dimensional object image strip.

15. Apparatus for identifying and tracking objects within a scene represented by a sequence of images comprising:

a reference image generator for generating a reference image containing background information of the scene;

an image processor, coupled to said reference image generator, for selecting an image from said sequence of images as an object image for generating a brightness difference value for each pixel in said object image by computing a reference brightness value for each reference pixel in said reference image, computing an image brightness value for each pixel in said object image, and comparing each of said reference brightness values to each corresponding image brightness value, generating an energy difference value for each pixel in said object image by computing a reference energy value for each reference pixel in said reference image, computing an image energy value for each pixel in said object image, and comparing each said reference energy value to each corresponding image energy value, comparing the brightness difference value to a brightness difference threshold value and the energy difference value to a energy difference threshold value to classify each pixel in said image as either background or non-background values comparing an energy value of each of the non-background pixels to an energy threshold;

discarding each non-background pixel that does not meet the threshold; and

an object detector, coupled to said image processor, for processing each bright non-background pixel to determine if the pixel is an object.

16. The apparatus of claim 15 wherein the threshold is determined by computing a statistical distribution of energy levels.

17. The apparatus of claim 15 wherein each image in said sequence of images is a one dimensional strip representation of a two-dimensional image, and said reference image is a one-dimensional reference image.