VISION BASED REAL TIME TRAFFIC MONITORING

• US 20100322476A1
• Filed: 12/05/2008
• Published: 12/23/2010
• Est. Priority Date: 12/13/2007
• Status: Active Grant

• Current Assignee: Clemson University
• Original Assignee: Clemson University
• Inventors: Sarasua, Wayne A., Kanhere, Neeraj Krantiveer, Birchfield, Stanley T.
• Granted Patent: US 8,379,926 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06F 18/2321 using statistics or function optimisation, e.g. modelling of probability density functions
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30236 Traffic on road, railway or crossing
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 7/248 involving reference images or patches
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: G06T 7/55 from multiple images
• CPC Class Codes: G06V 10/763 Non-hierarchical techniques, e.g. based on statistics of modelling distributions
• CPC Class Codes: G06V 20/54 of traffic, e.g. cars on the road, trains or boats
• CPC Class Codes: G08G 1/0175 by photographing vehicles, e.g. when violating traffic rules

Abstract

A system and method for detecting and tracking one or more vehicles using a system for obtaining two-dimensional visual data depicting traffic flow on a road is disclosed. In one exemplary embodiment, the system and method identifies groups of features for determining traffic data. The features are classified as stable features or unstable features based on whether each feature is on the frontal face of a vehicle close to the road plane. In another exemplary embodiment, the system and method identifies vehicle base fronts as a basis for determining traffic data. In yet another exemplary embodiment, the system and method includes an automatic calibration procedure based on identifying two vanishing points

Claims

1. A method for detecting and tracking one or more vehicles using a system for obtaining two-dimensional visual data depicting traffic flow on a road, the two-dimensional visual data comprising a plurality of image frames, the method comprising:

filtering the two-dimensional visual data using background subtraction to yield a foreground mask;

identifying a plurality of feature points from the foreground mask using a feature detector;

determining the plumb line projection of the feature points;

estimating three-dimensional world coordinates of the feature points using the plumb line projection of the feature points;

categorizing each of the feature points as either a stable feature or an unstable feature based on the estimated three-dimensional world coordinates of the feature points;

grouping stable features to form groups of associated stable features;

associating unstable features with groups of associated stable features to create segmented feature groups;

outputting data based on the segmented feature groups.

2. The method of claim 1, wherein the method further comprises classifying a segmented feature group as either a car or a truck based on the number of unstable features associated with the segmented feature group.

3. The method of claim 1, wherein the method comprises tracking segmented feature groups over the plurality image frames.

4. The method of claim 3, wherein the method comprises associating a segmented feature group already being tracked with a segmented feature group in a current image frame by computing the number of stable features shared between the segmented feature group already being tracked and the segmented feature group in the current image frame.

5. The method of claim 4, wherein the method further comprises initializing a segmented feature group in a current image frame that has not been associated with a segmented feature group already being tracked as a new vehicle.

6. The method of claim 1, wherein the method comprises calibrating the system for obtaining two-dimensional visual data.

7. The method of claim 1, wherein the system for obtaining two-dimensional visual data is calibrated using an automatic calibration procedure.

8. The method of claim 1, wherein the system for obtaining two-dimensional visual data comprises a camera that is mounted at a low angle and off-centered with respect to the road.

9. A method for detecting and tracking one or more vehicles using a system for obtaining two-dimensional visual data depicting traffic flow on a road, the two-dimensional visual data comprising a plurality of image frames, the method comprising:

filtering the two-dimensional visual data using background subtraction to yield a foreground mask;

detecting a vehicle base front from the foreground mask;

tracking the vehicle base front through the plurality of image frames;

outputting data based on the tracking of the vehicle base front.

10. The method of claim 9, wherein the method comprises calibrating the system for obtaining two-dimensional visual data.

11. The method of claim 10, wherein the system for obtaining two-dimensional visual data is calibrated using an automatic calibration procedure.

12. The method of claim 9, wherein the method further comprises:

detecting features in a base-constrained region associated with the vehicle base front;

estimating displacement of a vehicle base front using the detected features in the base-constrained region associated with the vehicle base front.

13. The method of claim 9, wherein the method further comprises filtering shadows from the two-dimensional visual data.

14. The method of claim 9, wherein the system for obtaining two-dimensional visual data comprises a camera that is mounted at a low angle and off-centered with respect to the road.

15. A method for automatically calibrating a system for detecting and tracking one or more vehicles based on two-dimensional visual data depicting traffic flow, the two-dimensional visual data comprising a plurality of image frames, the two-dimensional visual data defining a direction of travel and a direction perpendicular to the direction of travel, the method comprising:

detecting and tracking a vehicle based on the two-dimensional visual data to determine vehicle tracks of the vehicle;

estimating a first vanishing point in the direction of travel using the vehicle tracks of the detected vehicle;

estimating a second vanishing point in the direction perpendicular to the direction of travel using strong image gradients found on the detected vehicle;

calibrating the system for detecting and tracking one or more vehicles based on two-dimensional visual data using the first and second vanishing points.

16. The method of claim 15, wherein a vehicle is detected using a boosted cascade vehicle detector.

17. The method of claim 16, wherein the boosted cascade vehicle detector performs a pattern classification procedure.

18. The method of claim 15, wherein the method comprises using a Random Sample Consensus (RANSAC) procedure to eliminate outliers resulting from noise and/or image compression.

19. The method of claim 15, wherein a hinge point is used to estimate the second vanishing point.

20. The method of claim 15, wherein the system for detecting and tracking one or more vehicles comprises a pan-tilt-zoom camera, the method comprising determining the focal length, tilt angle, pan angle, or height of the pan-tilt-zoom camera using the first and second vanishing points.