System and method for detecting a passing vehicle from dynamic background using robust information fusion

• US 20050196020A1
• Filed: 02/10/2005
• Published: 09/08/2005
• Est. Priority Date: 02/19/2004
• Status: Active Grant

• Current Assignee: Siemens Corp. (Siemens AG)
• Original Assignee: Siemens Corporate Research Incorporated (Siemens AG)
• Inventors: Zhu, Ying, Comaniciu, Dorin, Kohler, Thorsten, Pellkofer, Martin
• Granted Patent: US 7,248,718 B2
• Time in Patent Office: Days
• US Class Current: 382/104
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads

Abstract

A system and method for detecting a passing vehicle is disclosed. A video sequence comprising a plurality of image frames is received. Image intensity is measured and image motion is estimated in each image frame. A hypothesis model describing background dynamics is formulated. The measured image intensity and motion estimation is used to determine if the background dynamics has been violated in a given image frame. If the background dynamics has been violated, motion coherency is used to determine whether the violation of the background dynamics is caused by a passing vehicle.

Claims

1. A method of detecting a passing vehicle comprising the steps of:

receiving a video sequence comprised of a plurality of image frames;

measuring image intensity and estimating image motion in each image frame;

formulating a hypothesis model describing background dynamics;

using the measured image intensity and motion estimation to determine if the background dynamics has been violated in a given image frame; and

if the background dynamics has been violated, using motion coherency to determine whether the violation of the background dynamics is caused by a passing vehicle.

2. The method of claim 1 wherein the hypothesis model describing background dynamics comprises hypothesis on image intensity and hypothesis on image motion.

3. The method of claim 1 further comprising the steps of:

determining the background dynamics is violated if the difference between measured image intensity and predicted image intensity is above a threshold.

4. The method of claim 3 wherein the predicted image intensity is determined by the hypothesis model describing background dynamics.

5. The method of claim 3 wherein the predicted image motion is determined by ego-motion and camera parameters.

6. The method of claim 3 wherein the measured image motion is obtained through a robust motion estimation method.

7. The method of claim 3 further comprising the steps of:

determining if the measured motion associated with the violated background dynamics has motion coherency; and

if there is motion coherency, determining that the violation is due to a passing vehicle.

8. The method of claim 3wherein the step of determining if the measured motion associated with the violated background dynamics has motion coherency further comprises the steps of:

applying a first subwindow and a second subwindow to the image frame having the violated background dynamics to determine the sequence of motion causing the violation;

determining the trajectory of motion of the violated background dynamics by determining which subwindow first indicates a change in image motion; and

if the first subwindow experiences a change in image motion first, determining the violation of background dynamics to be caused by a passing vehicle.

9. The method of claim 8 wherein the motion in the first and second subwindows is measured using the robust motion estimation method.

10. The method of claim 9 wherein robust motion estimation is done by calculating image flows and their covariances at distributed sample points in an analysis window associated with multiple image frames.

11. The method of claim 9 wherein robust motion estimation is done by exponentially weighting image flows and their covariances at distributed sample points.

12. The method of claim 9 wherein the robust motion estimation is done through joint spatial temporal filtering on image flows at distributed sample points.

13. The method of claim 12 wherein the spatial temporal filtering is done using variable density bandwidth fusion and multiscale mean shift.

14. The method of claim 13 wherein variable density bandwidth fusion and multiscale mean shift discard outliers and find the most significant sample estimate.

15. The method of claim 13 wherein the measured motion in the first and second analysis windows are determined by the most significant sample estimate obtained by variable density bandwidth fusion and multiscale mean shift.

16. The method of claim 1 wherein the video sequence is received from a camera mounted to an ego vehicle.

17. The method of claim 1 wherein trajectory of motion is determined using vehicle speed of the ego vehicle and camera parameters.

18. A system for detecting a passing vehicle comprising:

at least one camera for capturing a video sequence of image frames of background dynamics;

a processor associated with the at least one camera, the processor performing the following steps;

i). measuring image intensity and estimating image motion in each image frame;

ii). formulating a hypothesis model describing the background dynamics;

iii). using the measured image intensity and motion estimation to determine if the background dynamics has been violated in a given image frame; and

iv). if the background dynamics has been violated, using motion coherency to determine whether the violation of the background dynamics is caused by a passing vehicle.

19. The system of claim 18 wherein the hypothesis model describing background dynamics comprises hypothesis on image intensity and hypothesis on image motion.

20. The system of claim 18 wherein the processor further performs the step of

determining the background dynamics is violated if the difference between measured image intensity and predicted image intensity is above a threshold.

21. The system of claim 20 wherein the predicted image intensity is determined by the hypothesis model describing background dynamics.

22. The system of claim 20 wherein the predicted image motion is determined by ego-motion and camera parameters.

23. The system of claim 20 wherein the measured image motion is obtained through a robust motion estimation method.

24. The system of claim 20 wherein the processor further performs the steps of:

determining if the measured motion associated with the violated background dynamics has motion coherency; and

if there is motion coherency, determining that the violation is due to a passing vehicle.

25. The system of claim 20 wherein the step of determining if the measured motion associated with the violated background dynamics has motion coherency further comprises the steps of:

applying a first subwindow and a second subwindow to the image frame having the violated background dynamics to determine the sequence of motion causing the violation;

determining the trajectory of motion of the violated background dynamics by determining which subwindow first indicates a change in image motion; and

if the first subwindow experiences a change in image motion first, determining the violation of background dynamics to be caused by a passing vehicle.

26. The system of claim 25 wherein the motion in the first and second subwindows is measured using the robust motion estimation method.

27. The system of claim 26 wherein robust motion estimation is done by calculating image flows and their covariances at distributed sample points in an analysis window associated with multiple image frames.

28. The system of claim 26 wherein robust motion estimation is done by exponentially weighting image flows and their covariances at distributed sample points.

29. The system of claim 26 wherein the robust motion estimation is done through joint spatial temporal filtering on image flows at distributed sample points.

30. The system of claim 29 wherein the spatial temporal filtering is done using variable density bandwidth fusion and multiscale mean shift.

31. The system of claim 30 wherein variable density bandwidth fusion and multiscale mean shift discard outliers and find the most significant sample estimate.

32. The system of claim 30 wherein the measured motion in the first and second analysis windows are determined by the most significant sample estimate obtained by variable density bandwidth fusion and multiscale mean shift.

33. The system of claim 18 wherein the video sequence is received from a camera mounted to an ego vehicle.

34. The system of claim 18 wherein trajectory of motion is determined using vehicle speed of the ego vehicle and camera parameters.