Single camera video-based speed enforcement system with a secondary auxiliary RGB traffic camera

• US 9,183,746 B2
• Filed: 03/12/2013
• Issued: 11/10/2015
• Est. Priority Date: 03/12/2013
• Status: Active Grant

• Current Assignee: Conduent Business Services, LLC (Conduent, Inc.)
• Original Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Inventors: Wu, Wencheng, Bernal, Edgar A., Loce, Robert P., Wade, Thomas F., Islam, Abu
• Primary Examiner(s): Czekaj, Dave
• Assistant Examiner(s): Beck, Leron
• Application Number:
  US13/795,744
• Publication Number:
  US 20140267733A1
• Time in Patent Office: 973 Days
• Field of Search: 348/149
• US Class Current: 1/1
• CPC Class Codes: G08G 1/0175 by photographing vehicles, e.g. when violating traffic rules
• CPC Class Codes: G08G 1/054 photographing overspeeding vehicles

Abstract

When performing video-based speed enforcement a main camera and a secondary RGB traffic camera are employed to provide improved accuracy of speed measurement and improved evidentiary photo quality compared to single camera approaches. The RGB traffic camera provides sparse secondary video data at a lower cost than a conventional stereo camera. The sparse stereo processing is performed using the main camera data and the sparse RGB camera data to estimate a height of one or more tracked vehicle features, which in turn is used to improve speed estimate accuracy. By using secondary video, spatio-temporally sparse stereo processing is enabled specifically for estimating the height of a vehicle feature above the road surface.

Claims

1. A computer-implemented method for video-based speed estimation, comprising:

acquiring traffic video data from a primary camera and acquiring one or more image frames from a secondary camera;

preprocessing the video data acquired from the primary camera;

detecting at least one vehicle in video data acquired from the primary camera;

tracking the at least one vehicle of interest by identifying and tracking a location of one or more vehicle features across a plurality of video frames in video data acquired from the primary camera;

performing sparse stereo processing using video data of one or more tracked features within a predetermined region in the video frames from the primary camera and the one or more image frames from the secondary camera;

estimating a height of the one or more tracked features relative to a reference plane;

estimating vehicle speed as a function of camera calibration information and estimated feature height associated with at least one of the one or more tracked features;

wherein sparse stereo processing comprises performing height estimation by;

identifying a least square solution that is a function of camera calibration and orientation information;

estimating the feature height multiple times using a plurality of stereo feature pairs; and

processing the estimated heights statistically by computing one or more of an average height, a median height, a mean height, and a truncated mean height.

2. The method according to claim 1, further comprising preparing a violation package including a citation for a vehicle having an estimated speed that is greater than or equal to a predetermined speed threshold.

3. The method according to claim 2, further comprising transmitting the violation package to a law enforcement entity for validation.

4. The method according to claim 1, wherein the secondary camera is one of a red-green-blue (RGB) camera and a black and white camera.

5. The method according to claim 4, wherein the secondary camera is a video camera, and the one or more image frames are extracted from video captured by the secondary camera.

6. The method according to claim 1, wherein detecting at least one vehicle in the video data acquired from the primary camera further comprises submitting at least one frame of video data to a vehicle identification module that identifies the at least one vehicle.

7. The method according to claim 1, wherein the one or more tracked features of each vehicle comprises a license plate of the vehicle.

8. The method according to claim 7, further comprising identifying a given vehicle by the license plate of the vehicle, and including vehicle license plate information in a violation package that is transmitted to a law enforcement entity for use in issuing a citation to an owner of the identified vehicle.

9. The method according to claim 1, wherein the one or more tracked features comprises one or more of a scale invariant feature transform (SIFT), speeded up robust features (SURF), a gradient location and orientation histogram (GLOH), Harris corners, a histogram of oriented gradients (HOG), and local binary patterns (LBP).

10. A processor configured to execute computer-executable instructions for performing the method of claim 1, the instructions being stored on a non-transitory computer-readable medium.

11. A system that facilitates video-based speed enforcement, comprising:

a primary camera that captures video of vehicle;

a secondary camera that concurrently captures one or more image frames of the vehicle; and

a processor configured to;

acquire traffic video data from the primary camera and acquire the one or more image frames from a secondary camera;

preprocess the video data acquired from the primary camera;

detect at least one vehicle in video data acquired from the primary camera;

track the at least one vehicle of interest by identifying and tracking a location of one or more vehicle features across a plurality of video frames in video data acquired from the primary camera;

perform sparse stereo processing using video data of one or more tracked features within a predetermined region in the video frames from the primary camera and the one or more image frames from the secondary camera;

estimate a height of the one or more tracked features relative to a reference plane;

estimate vehicle speed as a function of camera calibration information and estimated feature height associated with at least one of the one or more tracked features;

wherein the processor is further configured to perform the sparse stereo processing and height estimation by;

identifying a least square solution that is a function of camera calibration and orientation information;

estimating the feature height multiple times using a plurality of stereo feature pairs; and

processing the estimated heights statistically by computing one or more of an average height, a median height, a mean height, and a truncated mean height.

12. The system according to claim 11, wherein the processor is further configured to prepare a violation package including a citation for a vehicle having an estimated speed that is greater than or equal to a predetermined speed threshold.

13. The system according to claim 12, wherein the processor is further configured to transmit the violation package to a law enforcement entity for validation.

14. The system according to claim 11, wherein the secondary camera is one of a red-green-blue (RGB) camera and a black and white camera.

15. The system according to claim 11, wherein the secondary camera is a video camera, and the one or more image frames are extracted from video captured by the secondary camera.

16. The system of claim 11, further comprising a vehicle identification module to which the processor submits at least one frame of video data to a vehicle identification module that identifies the at least one vehicle in order to detect at least one vehicle in the video data acquired from the primary camera.

17. The system according to claim 11, wherein the one or more tracked features of each vehicle comprises a license plate of the vehicle.

18. The system according to claim 17, wherein the processor identifies a given vehicle by the license plate of the vehicle, and includes vehicle license plate information in a violation package that is transmitted to a law enforcement entity for use in issuing a citation to an owner of the identified vehicle.

19. The system according to claim 11, wherein the one or more tracked features comprises one or more of a scale invariant feature transform (SIFT), speeded up robust features (SURF), a gradient location and orientation histogram (GLOH), Harris corners, a histogram of oriented gradients (HOG), and local binary patterns (LBP).

20. A non-transitory computer-readable medium having stored thereon computer-executable instructions for video-based speed estimation, the instructions comprising:

acquiring traffic video data from a primary camera and acquiring one or more image frames from a secondary camera;

preprocessing the video data acquired from the primary camera;

detecting at least one vehicle in video data acquired from the primary camera;

tracking the at least one vehicle of interest by identifying and tracking a location of one or more vehicle features across a plurality of video frames in video data acquired from the primary camera;

performing sparse stereo processing using video data of one or more tracked features within a predetermined region in the video frames from the primary camera and the one or more image frames from the secondary camera;

estimating a height of the one or more tracked features relative to a reference plane; and

estimating vehicle speed as a function of camera calibration information and estimated feature height associated with at least one of the one or more tracked features;

wherein sparse stereo processing comprises performing height estimation by;

identifying a least square solution that is a function of camera calibration and orientation information;

estimating the feature height multiple times using a plurality of stereo feature pairs; and

processing the estimated heights statistically by computing one or more of an average height, a median height, a mean height, and a truncated mean height.

21. The computer-readable medium of claim 20, further comprising preparing a violation package including a citation for the vehicle having an estimated speed that is greater than or equal to a predetermined speed threshold.

22. The computer-readable medium of claim 20, wherein the primary camera is a video camera and the secondary camera is one of a red-green-blue (RGB) camera and a black and white camera.