Method for detecting moving cast shadows object segmentation

• US 6,349,113 B1
• Filed: 10/22/1998
• Issued: 02/19/2002
• Est. Priority Date: 11/03/1997
• Status: Expired due to Term

• Current Assignee: AT&T Corporation (AT&T, Inc.)
• Original Assignee: AT&T Corporation (AT&T, Inc.)
• Inventors: Ostermann, Joern, Stauder, Jurgen, Mech, Roland
• Primary Examiner(s): RAO, ANAND SHASHIKANT
• Application Number:
  US09/177,686
• Time in Patent Office: 1,216 Days
• Field of Search: 382/236, 382/107, 382/199, 382/194, 382/171, 375/240.12, 375/240.13, 375/240.14, 375/240.15, 375/240.17, 375/240.16, 375/240.08
• US Class Current: 375/240.08
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: G06V 10/273 removing elements interfering with the pattern to be recognised

Abstract

An image region changed by a moving cast shadow from a first image to a second image is detected. For each pixel within a change detection mask (a binary mask indicating image areas of difference between the first image and the second image) and a set of neighboring pixels, the following steps are performed. Whether the pixel and the set of neighboring pixels include a static background edge or no edge is determined. Whether the pixel and the set of neighboring pixels include an edge with a spatial signal step width greater than a threshold is determined. Whether the pixel and the set of neighboring pixels have a uniform temporal change of illumination is determined. The pixel is classified as being changed by a moving cast shadow when at least two of the above-mentioned determinations succeed for the pixel or when at least one determination from the above-mentions determinations succeed for a majority of the set of neighboring pixels. In another embodiment of the present invention, detected image regions are used to estimate the two-dimensional shape of moving objects in image sequences even in the presence of moving case shadows. In another embodiment, detected image regions are temporarily integrated to represent the total of moving cast shadows.

Claims

1. A method for detecting a static background edge within a first image, comprising:

(a) detecting a first plurality of edges in the first image;

(b) detecting a second plurality of edges in a second image;

(c) subtracting the first image from the second image to produce a frame difference;

(d) applying a high frequency filter to the frame difference to produce a high frequency frame difference;

(e) comparing the high frequency frame difference to a threshold; and

(f) classifying, for each pixel from the first plurality of edges and from the second plurality of edges, a static edge when the edge from the high frequency frame difference is below the threshold.

2. The method of claim 1, wherein the threshold is adaptively calculated based on the high frequency frame difference.

3. A method for detecting a uniform temporal change of illumination in a pixel between a first image and a second image, comprising:

(a) determining a frame ratio within a change detection mask based on a first image and a second image, the change detection mask corresponding to a plurality of image regions of large difference between the first image and the second image;

(b) comparing a local spatial variance of the frame ratio for each pixel with an illumination threshold; and

(c) classifying each pixel as having a uniform temporal change of illumination when the local spatial variance is below the illumination threshold for that pixel.

4. The method of claim 3, wherein said classifying step (c) is performed only when the frame ratio for a subset of pixels neighboring the pixel have a frame ratio uniformly greater than or less than one.

5. A method for detecting borders of a moving cast shadow within a change detection mask having a border, the change detection mask corresponding to a difference between a first image and a second image, comprising:

(a) obtaining a plurality of moving cast shadow border candidates from the border of the change detection mask;

(b) performing, for each moving cast shadow border candidate from the plurality of moving cast shadow border candidates, the following substeps;

(i) evaluating a spatial signal step width; and

(ii) classifying the moving cast shadow border candidate as a moving cast shadow border when the moving cast shadow border candidate has a spatial signal step width exceeding a width threshold.

6. The method of claim 5, wherein the spatial signal step width evaluated in said evaluating step (b)(i) corresponds to the frame difference between the first image and the second image.

7. The method of claim 5, wherein the width threshold is related to an aperture of a camera providing the first image and the second image.

8. The method of claim 5, wherein the spatial signal step width evaluated in said evaluating step (b) for each moving cast shadow border candidate is determined along a line perpendicular to the moving cast shadow border candidate, the perpendicular line being determined by linear regression of a plurality of positions of neighboring moving cast shadow border candidates.

9. The method of claim 5, wherein the spatial signal step width evaluated in said evaluating step (b) for each moving cast shadow border candidate corresponds to a spatial signal step height divided by a spatial signal step gradient.

10. The method of claim 5, wherein the plurality of moving cast shadow candidates obtained in said obtaining step (a) includes the substep of:

(i) relocating at least one moving cast shadow candidate from the plurality of moving cast shadow candidates, the moving cast shadow candidate being relocated perpendicular to a border of the change detection mask to a position of highest spatial gradient of the difference between the first image and the second image.

11. A method for detecting an image region changed by a moving cast shadow from a first image to a second image, comprising:

(a) performing, for each pixel within a change detection mask, the change detection mask corresponding to a difference between the first image and the second image, the following substeps;

(i) determining whether the pixel and a plurality of neighboring pixels include a static background edge;

(ii) determining whether the pixel and the plurality of neighboring pixels are close to an edge with a spatial signal step width greater than a threshold, the signal step width being the difference between the first image and the second image;

(iii) determining whether the pixel and the plurality of neighboring pixels have a uniform temporal change of illumination; and

(iv) classifying the pixel as being changed by a moving cast shadow when at least two determinations from the group of determinations in steps (a)(i), (a)(ii) and (a)(iii) succeed for the pixel and the plurality of neighboring pixels.

12. The method of claim 11, wherein said determining substep (a)(i) further includes the following substeps:

(1) detecting a first plurality of edges in the first image;

(2) detecting a second plurality of edges in a second image;

(3) subtracting the first image from the second image to produce a frame difference;

(4) applying a high frequency filter to the frame difference to produce a high frequency frame difference;

(5) comparing the high frequency frame difference to a threshold; and

(6) classifying, for each pixel from the first plurality of edges and from the second plurality of edges, a static edge when the edge from the high frequency frame difference is below the threshold.

13. The method of claim 11, wherein said determining substep (a)(ii) further includes the following substeps:

(1) obtaining a plurality of moving cast shadow border candidates from the border of the change detection mask;

(2) performing, for each moving cast shadow border candidate from the plurality of moving cast shadow border candidates, the following substeps;

(A) evaluating a spatial signal step width; and

(B) classifying the moving cast shadow border candidate as a moving cast shadow border when the moving cast shadow border candidate has the spatial signal step width exceeding a width threshold.

14. The method of claim 11, wherein said determining step (a)(iii) further includes the following substeps:

(1) determining a frame ratio within the change detection mask based on a first image and a second image, the change detection mask corresponding to a difference between a first image and a second image;

(2) comparing a local spatial variance of the frame ratio for the pixel with an illumination threshold; and

(3) classifying the pixel as having a uniform temporal change of illumination when the local spatial variance is below the illumination threshold for the pixel.

15. A method for segmenting a moving object in front of a rigid background having a moving cast shadow, comprising:

(a) determining a change detection mask based on a first image and a second image;

(b) detecting an elimination portion of the change detection mask changed by the moving cast shadow to produce a remaining portion of the change detection mask;

(c) combining the remaining portion of the change detection mask with an object mask of the first image, if available, to produce an object change detection mask; and

(d) eliminating from the object change detection mask a portion corresponding to background uncovered by motion of the moving object to produce a mask of the moving object in the second image.

16. The method of claim 15, wherein said detecting step (b) further includes, for each pixel within a change detection mask and a plurality of neighboring pixels, the following substeps:

(i) determining whether the pixel and the plurality of neighboring pixels include a static background edge;

(ii) determining whether the pixel and the plurality of neighboring pixels are close to an edge with a spatial signal step width exceeding a threshold, the signal step width being the difference between the first image and the second image;

(iii) determining whether the pixel and the plurality of neighboring pixels has a uniform temporal change of illumination; and

(iv) classifying the pixel as being changed by a moving cast shadow when at least two determinations from the group of determinations in steps (b)(i), (b)(ii) and (b)(iii) succeed for the pixel and for the plurality of neighboring pixels.

17. The method of claim 15, wherein said eliminating step (d) further includes the substep of:

(i) detecting the background uncovered by motion of the moving object within the change detection mask.

18. A method for detecting uncovered background within a change detection mask, the change detection mask corresponding to a large difference between a first image and a second image, comprising:

(a) estimating a displacement vector field having one vector for each pixel of the second image, the vector for each pixel of the second image pointing from a corresponding image position of the first image to the pixel of the second image; and

(b) categorizing a portion of the change detection mask as uncovered background where pixels of the second image have a vector from the displacement vector field with an origin outside the change detection mask.

19. A method for segmenting and temporally tracking a plurality of moving cast shadows on a static background for a plurality of sequential image pairs, each image pair having a first image and a second image, comprising:

(a) setting each pixel of a mask to a first value, the first value indicating no moving cast shadow;

(b) performing, for each image pair from the plurality of sequential image pairs, the following substeps;

(1) detecting a plurality of image regions changed by the moving cast shadows;

(2) calculating a frame ratio by dividing each pixel of the second image by the corresponding pixel in the first image;

(3) classifying each image region detected in said detecting step (1) into;

(i) a first portion when the image region has a frame ratio greater than or equal to one, and

(ii) a second portion when the image region has a frame ratio smaller than one;

(4) setting to the first value in the mask each image region classified as the first portion in said classifying step (b)(1); and

(5) setting to a second value in the mask each image region classified as the second portion in said classifying step (b)(2).

20. The method of claim 19, wherein said performing step (b) further comprises the substep of:

(6) eliminating small regions having the first value and small regions having the second value in the mask.