Vision based real time traffic monitoring

• US 8,379,926 B2
• Filed: 12/05/2008
• Issued: 02/19/2013
• Est. Priority Date: 12/13/2007
• Status: Active Grant

• Current Assignee: Clemson University
• Original Assignee: Clemson University
• Inventors: Kanhere, Neeraj Krantiveer, Birchfield, Stanley T., Sarasua, Wayne A.
• Primary Examiner(s): Tabatabai, Abolfazl
• Application Number:
  US12/747,260
• Publication Number:
  US 20100322476A1
• Time in Patent Office: 1,537 Days
• Field of Search: 382/103, 382/104, 340/906, 340/907, 340/910, 340/917, 340/919, 340/931, 340/995.13, 342/454, 342/456, 701/117, 701/118, 701/120, 701/414, 701/423
• US Class Current: 382/104
• CPC Class Codes: G06F 18/2321 using statistics or function optimisation, e.g. modelling of probability density functions
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30236 Traffic on road, railway or crossing
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 7/248 involving reference images or patches
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: G06T 7/55 from multiple images
• CPC Class Codes: G06V 10/763 Non-hierarchical techniques, e.g. based on statistics of modelling distributions
• CPC Class Codes: G06V 20/54 of traffic, e.g. cars on the road, trains or boats
• CPC Class Codes: G08G 1/0175 by photographing vehicles, e.g. when violating traffic rules

Abstract

A system and method for detecting and tracking one or more vehicles using a system for obtaining two-dimensional visual data depicting traffic flow on a road is disclosed. In one exemplary embodiment, the system and method identifies groups of features for determining traffic data. The features are classified as stable features or unstable features based on whether each feature is on the frontal face of a vehicle close to the road plane. In another exemplary embodiment, the system and method identifies vehicle base fronts as a basis for determining traffic data. In yet another exemplary embodiment, the system and method includes an automatic calibration procedure based on identifying two vanishing points.

Claims

1. A method for detecting and tracking one or more vehicles using a system for obtaining two-dimensional visual data depicting traffic flow on a road, the two-dimensional visual data comprising a plurality of image frames, the method comprising:

filtering the two-dimensional visual data using background subtraction to yield a foreground mask;

identifying a plurality of feature points from the foreground mask using a feature detector;

determining the plumb line projection of the feature points;

estimating three-dimensional world coordinates of the feature points using the plumb line projection of the feature points;

categorizing each of the feature points as either a stable feature or an unstable feature based on the estimated three-dimensional world coordinates of the feature points;

grouping stable features to form groups of associated stable features;

associating unstable features with groups of associated stable features to create segmented feature groups;

outputting data based on the segmented feature groups.

2. The method of claim 1, wherein the method further comprises classifying a segmented feature group as either a car or a truck based on the number of unstable features associated with the segmented feature group.

3. The method of claim 1, wherein the method comprises tracking segmented feature groups over the plurality image frames.

4. The method of claim 3, wherein the method comprises associating a segmented feature group already being tracked with a segmented feature group in a current image frame by computing the number of stable features shared between the segmented feature group already being tracked and the segmented feature group in the current image frame.

5. The method of claim 4, wherein the method further comprises initializing a segmented feature group in a current image frame that has not been associated with a segmented feature group already being tracked as a new vehicle.

6. The method of claim 1, wherein the method comprises calibrating the system for obtaining two-dimensional visual data.

7. The method of claim 1, wherein the system for obtaining two-dimensional visual data is calibrated using an automatic calibration procedure.

8. The method of claim 1, wherein the system for obtaining two-dimensional visual data comprises a camera that is mounted at a low angle and off-centered with respect to the road.

9. A method for detecting and tracking one or more vehicles using a system for obtaining two-dimensional visual data depicting traffic flow on a road, the two-dimensional visual data comprising a plurality of image frames, the method comprising:

calibrating the system for obtaining two-dimensional visual data;

filtering the two-dimensional visual data using background subtraction to yield a foreground mask;

detecting a vehicle base front from the foreground mask, the vehicle base front representative of the front side of a base of a vehicle;

tracking the vehicle base front through the plurality of image frames; and

outputting data based on the tracking of the vehicle base front.

10. The method of claim 9, wherein the system for obtaining two-dimensional visual data is calibrated using an automatic calibration procedure.

11. The method of claim 9, wherein the method further tracking the vehicle base front through the plurality of image frames comprises:

detecting features in a base-constrained region associated with the vehicle base front; and

estimating displacement of a vehicle base front using the detected features in the base-constrained region associated with the vehicle base front.

12. The method of claim 9, wherein the method further comprises filtering shadows from the two-dimensional visual data.

13. The method of claim 9, wherein the system for obtaining two-dimensional visual data comprises a camera that is mounted at a low angle and off-centered with respect to the road.

14. The computer-implemented method of claim 9, wherein the method comprises detecting a vehicle base front from the foreground mask using a difference operator in the vertical direction.

15. A computer-implemented method for automatically calibrating a system for detecting and tracking one or more vehicles based on two-dimensional visual data depicting traffic flow, the two-dimensional visual data comprising a plurality of image frames, the two-dimensional visual data defining a direction of travel and a direction perpendicular to the direction of travel, the computer-implemented method comprising:

detecting and tracking, with a computing device, a vehicle based on the two-dimensional visual data to determine vehicle tracks of the vehicle;

estimating, with the computing device, a first vanishing point in the direction of travel using the vehicle tracks of the detected vehicle;

estimating, with the computing device, a second vanishing point in the direction perpendicular to the direction of travel using strong image gradients found on the detected vehicle; and

calibrating the system for detecting and tracking one or more vehicles based on two-dimensional visual data using the first and second vanishing points.

16. The computer-implemented method of claim 15, wherein a vehicle is detected using a boosted cascade vehicle detector.

17. The computer-implemented method of claim 16, wherein the boosted cascade vehicle detector performs a pattern classification procedure.

18. The computer-implemented method of claim 15, wherein the method comprises using a Random Sample Consensus (RANSAC) procedure to eliminate outliers resulting from noise and/or image compression.

19. The computer-implemented method of claim 15, wherein a hinge point is used to estimate the second vanishing point.

20. The computer-implemented method of claim 15, wherein the system for detecting and tracking one or more vehicles comprises a pan-tilt-zoom camera, the method comprising determining the focal length, tilt angle, pan angle, or height of the pan-tilt-zoom camera using the first and second vanishing points.