Method for stop sign law enforcement using motion vectors in video streams

• US 10,018,703 B2
• Filed: 09/13/2012
• Issued: 07/10/2018
• Est. Priority Date: 09/13/2012
• Status: Active Grant

• Current Assignee: Conduent Business Services, LLC (Conduent, Inc.)
• Original Assignee: Conduent Business Services, LLC (Conduent, Inc.)
• Inventors: Bernal, Edgar A., Bulan, Orhan, Loce, Robert P.
• Primary Examiner(s): Ustaris, Joseph
• Assistant Examiner(s): Adrovel, William
• Application Number:
  US13/613,174
• Publication Number:
  US 20140071286A1
• Time in Patent Office: 2,126 Days
• Field of Search: None
• CPC Class Codes: G01S 3/7864 T.V. type tracking systems
• CPC Class Codes: G06F 18/23 Clustering techniques
• CPC Class Codes: G06T 2207/30236 Traffic on road, railway or crossing
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/215 Motion-based segmentation
• CPC Class Codes: G06T 7/223 using block-matching
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/269 using gradient-based methods
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 20/54 of traffic, e.g. cars on the road, trains or boats
• CPC Class Codes: G08G 1/0175 by photographing vehicles, e.g. when violating traffic rules
• CPC Class Codes: G08G 1/04 using optical or ultrasonic detectors
• CPC Class Codes: G08G 1/096716 where the received information does not generate an automatic action on the vehicle control
• CPC Class Codes: G08G 1/096741 where the source of the transmitted information selects which information to transmit to each vehicle
• CPC Class Codes: G08G 1/096783 where the origin of the information is a roadside individual element
• CPC Class Codes: H04N 19/139 Analysis of motion vectors, e.g. their magnitude, direction, variance or reliability
• CPC Class Codes: H04N 19/513 Processing of motion vectors

Abstract

A method for enforcing traffic signal compliance includes acquiring a series of temporal related image frames including a target area. Each image frame includes pixel data representative of the target area. The method includes generating one or more motion vectors between two or more of the image frames. The motion vectors are the type produced by compressing the pixel data associated with the two or more image frames. The method includes associating a cluster of motion vectors with a vehicle. The method further includes tracking a position of the vehicle across the two or more image frames. Using the tracking results, the method includes determining whether the vehicle stops in the target area. For the vehicle being determined as not stopping, the method includes signaling an occurrence of noncompliance.

Claims

1. A vector-based stop detection method for enforcing traffic signal compliance, the method comprising:

receiving video data from a surveillance camera;

defining a target area in the received video data;

dividing a current frame into a number of blocks;

computing a displacement for a block between a reference frame and the current frame and describing the displacement of the block as a motion vector;

defining a cluster as a group of one or more connected motion vectors having a length meeting or exceeding a first vector length threshold and classifying a block having a motion vector meeting or exceeding the first vector length as an active motion block;

in response to a number of active motion blocks in the cluster meeting or exceeding a cluster size threshold, classifying the cluster of motion vectors as belonging to a vehicle;

tracking a position of the cluster across two or more video frames;

monitoring the length of motion vectors of the cluster in the two or more video frames and reclassifying any motion vector that no longer exceeds the first vector length threshold as not an active motion block;

in response to the cluster transitioning from having the number of active motion blocks meeting or exceeding the cluster size threshold to having fewer than a second number of active motion blocks, classifying the cluster as being inactive;

in response to the inactivity of the cluster persisting for a predetermined duration, classifying the vehicle as being stationary;

in response to the activity of the cluster persisting in the target area, classifying the vehicle as not being stationary and transmitting a signal and compressed pixel data to a second device.

2. The method of claim 1 wherein the motion vectors are a type produced in the process of compressing pixel data associated with the two or more video frames.

3. The method of claim 2, wherein the video data is compressed video data, and the motion vectors are extracted from the compressed video data.

4. The method of claim 2, wherein the received video data includes a live uncompressed video stream, and wherein the motion vectors are generated during video compression.

5. The method of claim 1 wherein the motion vectors are one of those extracted from optical flow data between the two or more video frames and those establishing correspondences between frames across a plurality of video frames.

6. The method of claim 1, further comprising:

defining the target area in image coordinates as including one or more of a polygon, a circle, an ellipse, a closed curve, an open curve, and a line.

7. The method of claim 1, wherein the target area includes a stationary stop indicator.

8. The method of claim 7, wherein the stop indicator is selected from a group consisting of:

an intersection;

a stop line;

a crosswalk;

a stop signal;

a stop sign; and

, a combination of the above.

9. The method of claim 7, wherein the target area extends at least one vehicle length behind the stop indicator.

10. The method of claim 1 further comprising:

counting a number of consecutive frames across which the cluster persists;

comparing the number of consecutive frames to a threshold; and

,

for the number of consecutive frames meeting and exceeding the threshold, associating the vehicle as an oncoming vehicle.

11. The method of claim 1, further comprising:

determining whether the vehicle stops in the target area by;

determining a first frame in the series corresponding to when the cluster disappears and determining a second frame in the series corresponding to when the cluster reappears;

counting a number of frames between the first and second frames;

comparing the number of frames to a threshold; and

for the number of frames not meeting the threshold, associating the vehicle as not stopping.

12. A computer program product comprising a non-transitory computer-readable medium which encodes instructions for performing the method of claim 1.

13. A system for enforcing traffic signal compliance, the system comprising:

a video capture device comprising memory which stores instructions for performing the method of claim 1 and a processor, in communication with the memory for executing the instructions.

14. A vector-based stop detection system for enforcing traffic signal compliance, the system comprising:

a video capture device including;

a video capture module configured to acquire a series of temporal related image frames including a target area, each image frame including pixel data representative of the target area;

a motion vector extraction module adapted to;

divide a current frame into a number of blocks,

compute a displacement for a block between a reference frame and the current frame,

describe the displacement of the block as a motion vector,

define a cluster as a group of one or more connected motion vectors having a length meeting or exceeding a first vector length threshold,

in response to a number of blocks in the cluster meeting or exceeding a cluster size threshold, characterize the cluster of motion vectors as belonging to

a vehicle; and

,

a violation determination module configured to;

in response to the cluster transitioning from having the number of active motion blocks meeting or exceeding the cluster size threshold to having fewer than a second number of active motion blocks, classifying the cluster as being inactive;

in response to the inactivity of the cluster persisting for a predetermined duration, characterize the vehicle as being stationary; and

in response to the vehicle not stopping in the target area, transmit at least one of a signal and the compressed pixel data to a second device.

15. The system of claim 14, wherein the motion vector extraction module is further configured to:

in a current frame, define the target area in image coordinates as including one or more of a polygon, a circle, an ellipse, a closed curve, an open curve, and a line.

16. The system of claim 15, wherein the target area includes a stationary stop indicator selected from a group consisting:

an intersection;

a stop line;

a crosswalk;

a stop signal;

a stop sign; and

, a combination of the above.

17. The system of claim 15, wherein the target area extends at least one vehicle length behind the stop indicator.

18. The system of claim 14, wherein the motion vector extraction module is further configured to:

count a number of consecutive frames across which the cluster persists;

compare the number of consecutive frames to a third threshold; and

,

for the number of consecutive frames meeting and exceeding a third threshold, associate the vehicle as an oncoming vehicle.

19. The system of claim 14, wherein the violation determination module is further configured to:

determine a first frame in the series corresponding to when the cluster disappears and determine a second frame in the series corresponding to when the cluster reappears;

count a number of frames between the first and second frames;

compare the number of frames to a stop time threshold; and

for the number of frames not meeting the stop time threshold, associate the vehicle as not complying with a stop indicator.

20. The system of claim 14, wherein the violation determination module is included in one of the image capture device and a second, remote device.