FEATURE- AND CLASSIFIER-BASED VEHICLE HEADLIGHT/SHADOW REMOVAL IN VIDEO

• US 20150278616A1
• Filed: 03/27/2014
• Published: 10/01/2015
• Est. Priority Date: 03/27/2014
• Status: Active Grant

• Current Assignee: Conduent Business Services, LLC (Conduent, Inc.)
• Original Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Inventors: Li, Qun, Bernal, Edgar A., Loce, Robert P., Shreve, Matthew
• Granted Patent: US 9,275,289 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 18/2411 based on the proximity to a decision surface, e.g. support vector machines
• CPC Class Codes: G06T 2207/10048 Infrared image
• CPC Class Codes: G06T 2207/20032 Median filtering
• CPC Class Codes: G06T 2207/20076 Probabilistic image processing
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 2207/30242 Counting objects in image
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06T 7/215 Motion-based segmentation
• CPC Class Codes: G06T 7/251 involving models
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 20/46 Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
• CPC Class Codes: G06V 20/584 of vehicle lights or traffic lights

Abstract

A method for removing false foreground image content in a foreground detection process performed on a video sequence includes, for each current frame, comparing a feature value of each current pixel against a feature value of a corresponding pixel in a background model. The each current pixel is classified as belonging to one of a candidate foreground image and a background based on the comparing. A first classification image representing the candidate foreground image is generated using the current pixels classified as belonging to the candidate foreground image. The each pixel in the first classification image is classified as belonging to one of a foreground image and a false foreground image using a previously trained classifier. A modified classification image is generated for representing the foreground image using the pixels classified as belonging to the foreground image while the pixels classified as belonging to the false foreground image are removed.

Claims

1. A method for removing false foreground image content in a foreground detection process performed on a video sequence, the method comprising:

receiving video data from a sequence of frames taken from an associated image capture device monitoring a scene of interest;

for a current frame, comparing a current feature value of each current pixel against a feature value of a corresponding pixel in a background model of the scene of interest;

classifying the each current pixel as belonging to one of a candidate foreground image and a background based on the comparing;

generating a first classification image representing the candidate foreground image using current pixels classified as belonging to the candidate foreground image;

classifying the each pixel in the first classification image as belonging to one of a foreground image and a false foreground image using a previously trained classifier; and

,

generating a modified classification image representing the foreground image using the pixels classified as belonging to the foreground image and removing pixels classified as belonging to the false foreground image.

2. The method of claim 1, wherein the classifying the each pixel as belonging to the one of the foreground image and background includes:

performing a comparison of the current feature value in the current frame relative to the corresponding feature value in the background model.

3. The method of claim 1, wherein the false foreground detection includes at least one of a shadow and a light cast from a moving object.

4. The method of claim 1, wherein the classifying the each current pixel in the first classification image as belonging to one of a foreground image and a false foreground detection includes:

segmenting the first classification image into pixel blocks;

for each pixel block, calculating a first feature vector for the current frame;

calculating a second feature vector for a current background estimate;

comparing the second feature vector with the first feature vector to obtain a difference feature vector corresponding to the pixel block;

inputting the resulting difference feature vector to the previously trained classifier; and

,

outputting a classification decision for the each pixel block.

5. The method of claim 4, wherein the classifying the each pixel in the first classification image as belonging to the one of the foreground image and the false foreground detection further includes:

generating a coarse scale classification image based on the binary decision of the each pixel block;

classifying the each pixel in the first classification image as belonging to the one of the foreground image and the false foreground detection based on the classification decision.

6. The method of claim 5, further comprising:

comparing the coarse scale classification image to the first classification image and removing redundant pixels in the blocks.

computing a convex hull on a resulting image to generate the foreground image.

7. The method of claim 1, further comprising training the classifier before receiving the video data, the training including:

locating an object in a sample frame;

computing a feature of at least one segmented block in the sample frame corresponding to the object;

computing a difference between the current feature value and a corresponding feature value for a corresponding one of the at least one segmented block in a background estimate;

labeling each difference as corresponding to one of a foreground object and a false foreground object; and

,

training the classifier using the labeled samples and corresponding difference features.

8. The method of claim 7, wherein the current feature value represents one of a texture and color.

9. The method of claim 7, wherein the current feature value represents a feature that is invariant to monotonic transformations of color intensities.

10. A system for removing false foreground image content in a video sequence, the system comprising a false foreground image detection device including a memory and a processor in communication with the processor configured to:

receive video data from a sequence of frames taken from an associated image capture device monitoring a scene of interest;

for a current frame, compare a current feature value of each current pixel against a feature value of a corresponding pixel in a background model of the scene of interest;

classify the each current pixel as belonging to one of a candidate foreground image and a background based on the comparing;

generate a first classification image representing the candidate foreground image using current pixels classified as belonging to the candidate foreground image;

classify the each pixel in the first classification image as belonging to one of a foreground image and a false foreground image using a previously trained classifier; and

,

generate a modified classification image representing the foreground image using the pixels classified as belonging to the foreground image and removing pixels classified as belonging to the false foreground image.

11. The system of claim 10, wherein the processor is further configured to:

perform a comparison of the current feature value in the current frame relative to a corresponding feature value in the background model for the classifying of the each pixel as belonging to the one of the foreground image and background.

12. The system of claim 10, wherein the false foreground detection includes at least one of a shadow and a light cast from a moving object.

13. The system of claim 10, wherein the processor is further configured to:

segment the first classification image into pixel blocks;

for each pixel block, calculate a first feature vector for the current frame;

calculate a second feature vector for a current background estimate;

compare the second feature vector with the first feature vector to obtain a difference feature vector corresponding to the pixel block;

input the resulting difference feature vector to the previously trained classifier; and

,

output a classification decision for the each pixel block.

14. The system of claim 13, wherein the processor is further configured to:

generate a coarse scale classification image based on the binary decision of the each pixel block;

classify the each pixel in the first classification image as belonging to the one of the foreground image and the false foreground detection based on the classification decision.

15. The system of claim 14, wherein the processor is further configured to:

compare the coarse scale classification image to the first classification image and removing redundant pixels in the blocks.

compute a convex hull on a resulting image to generate the foreground image.

16. The system of claim 10, wherein the processor is further configured to:

locate an object in a sample frame;

compute a feature of at least one segmented block in the sample frame corresponding to the object;

compute a difference between the current feature value and a corresponding feature value for a same block in a background estimate;

label each difference as corresponding to one of a foreground object and a false foreground object; and

,

train a classifier using the labeled samples and corresponding difference features.

17. The system of claim 16, wherein the current feature value represents one of a texture and color.

18. The system of claim 16, wherein the current feature value represents a feature that is invariant to monotonic transformations of color intensities.