Traffic violation detection

• US 9,679,203 B2
• Filed: 02/02/2016
• Issued: 06/13/2017
• Est. Priority Date: 05/15/2014
• Status: Active Grant

• Current Assignee: Conduent Business Services, LLC (Conduent, Inc.)
• Original Assignee: Conduent Business Services, LLC (Conduent, Inc.)
• Inventors: Burry, Aaron M., Loce, Robert P., Bulan, Orhan
• Primary Examiner(s): PARK, SOO JIN
• Application Number:
  US15/013,354
• Publication Number:
  US 20160148058A1
• Time in Patent Office: 497 Days
• Field of Search: None
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 2207/30236 Traffic on road, railway or crossing
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 20/54 of traffic, e.g. cars on the road, trains or boats
• CPC Class Codes: G08G 1/0175 by photographing vehicles, e.g. when violating traffic rules

Abstract

A method for detecting a vehicle running a stop signal positioned at an intersection includes acquiring a sequence of frames from at least one video camera monitoring an intersection being signaled by the stop signal. The method includes defining a first region of interest (ROI) including a road region located before the intersection on the image plane. The method includes searching the first ROI for a candidate violating vehicle. In response to detecting the candidate violating vehicle, the method includes tracking at least one trajectory of the detected candidate violating vehicle across a number of frames. The method includes classifying the candidate violating vehicle as belonging to one of a violating vehicle and a non-violating vehicle based on the at least one trajectory.

Claims

1. A method to detect a traffic violation, the method comprising:

acquiring a sequence of image frames from at least one image source recording vehicle traffic;

defining a first region of interest (ROI) on an image plane of at least one of the image frames in the sequence of image frames;

searching the defined first ROI in at least one of the image frames in the sequence of image frames for a candidate violating vehicle that may be committing the traffic violation;

in response to detecting the candidate violating vehicle in the defined first ROI in at least one image frame in the sequence of image frames, tracking a location of the detected candidate violating vehicle in at least one other of the image frames in the sequence of image frames;

generating at least one trajectory of the detected candidate violating vehicle using the location of the detected candidate violating vehicle in the at least one image frame and in the at least one other of the image frames;

defining a second ROI on an image plane of at least one of the image frames in the sequence of image frames; and

in response to determining the at least one trajectory of the detected candidate violating vehicle includes a trajectory component in the defined first ROI and the defined second ROI, classifying the candidate violating vehicle as a violating vehicle, wherein one of the first and second ROIs includes a road region located before one of a stop signal and an intersection on an image plane and another of the first and second ROIs includes a different road region located after the one of the stop signal and the intersection on the image plane.

2. The method of claim 1 wherein the defined second ROI is adjacent the defined first ROI.

3. The method of claim 1 wherein the defined second ROI is displaced from the defined first ROI.

4. The method of claim 1 wherein the defined first ROI includes a road portion located before an intersection and the defined second ROI includes a road portion located distal to a location of the first ROI.

5. The method of claim 1 wherein the classifying further includes:

identifying a stop signal;

in response to the stop signal being a traffic light, determining a color of at least one signal lamp displayed on the traffic light as belonging to one of red, yellow, and green; and

classifying the candidate violating vehicle as belonging to a violating vehicle based on the at least one trajectory and the color of the at least one signal lamp displayed being red.

6. The method of claim 5, wherein the determining the color of the signal includes:

defining at least two lamp ROIs, each lamp ROI including a location of a signal lamp displayed on the traffic light;

calculating a measure of intensity for pixels located at the at least one signal lamp within the lamp ROIs;

determining a lamp ROI corresponding to a highest measured intensity;

identifying the signal lamp corresponding to the determined lamp ROI; and

classifying the displayed traffic light color based on the identified signal lamp.

7. The method of claim 1, wherein the searching of the defined first ROI for the candidate violating vehicle includes:

performing foreground detection within the defined first ROI via one of motion estimation and background estimation with subsequent subtraction.

8. The method of claim 1, wherein the tracking includes:

extracting a set of feature descriptors from pixels from at least one image frame in the sequence of image frames of the detected candidate violating vehicle;

determining if the detected candidate violating vehicle corresponds with one of a tracked vehicle in a previous image frame and a new vehicle of interest in a current image frame using the extracted set of feature descriptors;

in response to the detected candidate violating vehicle corresponding with the new vehicle of interest, adding the extracted set of feature descriptors to a pool of current feature descriptors; and

tracking the pool of current feature descriptors across the sequence of image frames.

9. The method of claim 8, wherein the feature descriptors are selected from a group consisting of:

color attributes of at least one pixel from at least one image frame in the sequence of image frames of the detected candidate violating vehicle;

geometric attributes of the detected candidate violating vehicle;

shape attributes of the detected candidate violating vehicle; and

a combination of the above.

10. The method of claim 8, wherein the determining if the detected candidate violating vehicle corresponds with the tracked vehicle in the previous image frame includes:

computing a number of currently tracked features of the detected candidate violating vehicle;

comparing the number of tracked features to a predetermined threshold;

in response to the number of tracked features not meeting and exceeding the predetermined threshold, classifying the detected candidate violating vehicle as belonging to the new vehicle of interest; and

in response to the number of tracked features meeting and exceeding the predetermined threshold, classifying the detected candidate violating vehicle as belonging to the tracked vehicle in the previous image frame.

11. The method of claim 1, wherein the sequence of image frames is acquired from a group consisting of:

video captured in relation to when the candidate violating vehicle triggers a sensor in a road;

video captured in relation to when a red signal light is detected;

video captured in relation to when a signal is transmitted to a signal light for turning the signal light red; and

a combination of the above.

12. The method of claim 1, wherein the acquiring the sequence of image frames includes:

triggering the at least one image source to capture the sequence of image frames when a signal lamp on an image plane is red.

13. A system to detect a traffic violation, the system including a memory and a processor in communication with the memory, the processor configured to:

acquire a sequence of image frames from at least one image source recording vehicle traffic;

define a first region of interest (ROI) on an image plane of at least one of the image frames in the sequence of image frames;

search the defined first ROI in at least one of the image frames in the sequence of image frames for a candidate violating vehicle that may be committing the traffic violation;

in response to detecting the candidate violating vehicle in the defined first ROI in at least one image frame in the sequence of image frames, track a location of the detected candidate violating vehicle in at least one other of the image frames in the sequence of image frames;

generate at least one trajectory of the detected candidate violating vehicle using the location of the detected candidate violating vehicle in the at least one image frame and in the at least one other of the image frames;

define a second ROI on an image plane of at least one of the image frames in the sequence of image frames; and

classify the detected candidate violating vehicle as a violating vehicle if the at least one trajectory of the detected candidate violating vehicle includes a trajectory component in the defined first ROI and the defined second ROI,

wherein the processor is further configured to;

define one of the first and second ROIs to include a road region located before one of a stop signal and an intersection on an image plane and define another of the first and second ROIs to include a different road region located after the one of the stop signal and the intersection on the image plane.

14. The system of claim 13, wherein the processor is further configured to:

identify a stop signal;

in response to the stop signal being a traffic light, determine a color of a signal lamp displayed on the traffic light as belonging to one of red, yellow, and green; and

classify the candidate violating vehicle as belonging to a violating vehicle based on the at least one trajectory and the color of the signal lamp displayed being red.

15. The system of claim 14, wherein the processor is further configured to:

define at least two lamp ROIs, each lamp ROI including a location of a signal lamp displayed on the traffic light;

calculate a measure of intensity for pixels located at the each signal lamp within the lamp ROIs;

determine a lamp ROI corresponding to a highest measured intensity;

identify the signal lamp corresponding to the determined lamp ROI; and

,

classify the displayed traffic light color based on the identified signal lamp.

16. The system of claim 13, wherein the processor is further configured to:

perform foreground detection within the defined first ROI via one of motion estimation and background estimation with subsequent subtraction.

17. The system of claim 13, wherein the processor is further configured to:

extract a set of feature descriptors from pixels corresponding to a region including the detected candidate violating vehicle;

determine if the detected candidate violating vehicle corresponds with one of a tracked vehicle in a previous image frame and a new vehicle of interest in a current image frame using the extracted set of feature descriptors;

in response to the detected candidate violating vehicle corresponding with the new vehicle of interest, add the extracted set of feature descriptors to a pool of current feature descriptors; and

track the pool of current feature descriptors across the sequence of image frames.

18. The system of claim 17, wherein the feature descriptors are selected from a group consisting of:

color attributes of the region including the detected candidate violating vehicle;

geometric attributes of the detected candidate violating vehicle;

shape attributes extracted from the region including the detected candidate violating vehicle; and

a combination of the above.

19. The system of claim 17, wherein the processor is further configured to:

compute a number of currently tracked features in the region including the detected candidate violating vehicle;

compare the number of tracked features to a predetermined threshold;

in response to the number of tracked features not meeting and exceeding the predetermined threshold, classify the detected candidate violating vehicle as belonging to the new vehicle of interest; and

in response to the number of tracked features meeting and exceeding the predetermined threshold, classify the detected candidate violating vehicle as belonging to the tracked vehicle in the previous image frame.

20. The system of claim 13, wherein the processor is further configured to acquire the sequence of image frames from a group consisting of:

video captured in relation to when the candidate violating vehicle triggers a sensor in a road;

video captured in relation to when a red signal light is detected;

video captured in relation to when a signal is transmitted to a signal light for turning the signal light red; and

a combination of the above.

21. The system of claim 13, wherein the processor is further configured to:

trigger the at least one camera to capture the sequence of image frames when a signal lamp on an image plane is red.

22. A method to detect a movement violation, the method comprising:

acquiring a sequence of image frames from at least one image source recording object movement;

defining a first region of interest (ROI) on an image plane of at least one of the image frames in the sequence of image frames;

searching the defined first ROI in at least one of the image frames in the sequence of image frames for a candidate violating object that may be committing the movement violation;

in response to detecting the candidate violating object in the defined first ROI in at least one image frame in the sequence of image frames, tracking a location of the detected candidate violating object in at least one other of the image frames in the sequence of image frames;

generating at least one trajectory of the detected candidate violating object using the location of the detected candidate violating object in the at least one image frame and in the at least one other of the image frames;

defining a second ROI on an image plane of at least one of the image frames in the sequence of image frames; and

classifying the detected candidate violating object as a violating object if the at least one trajectory of the detected candidate violating object includes a trajectory component in the defined first ROI and the defined second ROI, wherein the defined first ROI includes a road portion located before one of a stop signal and an intersection on an image plane and the defined second ROI includes a different road region located after the one of the stop signal and the intersection on the image plane.