System and method for detecting, tracking and estimating the speed of vehicles from a mobile platform

• US 9,070,289 B2
• Filed: 05/10/2013
• Issued: 06/30/2015
• Est. Priority Date: 05/10/2013
• Status: Active Grant

• Current Assignee: Conduent Business Services, LLC (Conduent, Inc.)
• Original Assignee: Palo Alto Research, Inc.
• Inventors: Saund, Eric, Paulson, Christopher, Burton, Gregory, Peeters, Eric
• Primary Examiner(s): Black, Thomas G
• Assistant Examiner(s): Nolan, Peter D
• Application Number:
  US13/891,765
• Publication Number:
  US 20140336848A1
• Time in Patent Office: 781 Days
• Field of Search: 701/117, 701/119, 340/936, 340/937, 340/436
• US Class Current: 1/1
• CPC Class Codes: B64U 2101/30 for imaging, photography or videography
• CPC Class Codes: G05D 1/0094 involving pointing a payload, e.g. camera, weapon, sensor, towards a fixed or moving target
• CPC Class Codes: G06T 2207/10032 Satellite or aerial image; Remote sensing
• CPC Class Codes: G06T 2207/30236 Traffic on road, railway or crossing
• CPC Class Codes: G06T 7/215 Motion-based segmentation
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06V 10/255 Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
• CPC Class Codes: G06V 20/13 Satellite images
• CPC Class Codes: G06V 20/17 taken from planes or by drones
• CPC Class Codes: G06V 20/54 of traffic, e.g. cars on the road, trains or boats
• CPC Class Codes: G06V 2201/08 Detecting or categorising vehicles
• CPC Class Codes: G08G 1/0175 by photographing vehicles, e.g. when violating traffic rules
• CPC Class Codes: G08G 1/054 photographing overspeeding vehicles
• CPC Class Codes: G08G 5/00 Traffic control systems for aircraft , e.g. air-traffic control [ATC]

Abstract

A method and system for measurement of ground based vehicle speed includes a movable platform that includes an unmanned aerial vehicle (UAV) located in proximity to a roadway, the UAV operates under control and navigation of a UAV control unit, and the UAV also carries camera and monitoring equipment, the camera and monitoring equipment including an onboard computing system, and a camera with a wide angle lens and a camera with a telephoto lens, the cameras being mounted on a pan/tilt device. An algorithm operated by the on-board computing system is used to detect and track vehicles moving on a roadway. The algorithm is configured to detect and track the vehicles despite motion created by movement of the UAV. The cameras mounted on the pan/tilt device are moved under the direction of the computer vision algorithm to maintain a target vehicle of the detected moving vehicles in view, and the speed of the target vehicle is measured.

Claims

1. A method for measuring ground vehicle speed comprising:

positioning an unmanned aerial vehicle (UAV) in proximity to a roadway, the UAV flying under control and navigation of a UAV control unit, and further carrying camera and monitoring equipment, the camera and monitoring equipment including an onboard computing system, and a camera with a wide angle lens and a camera with a telephoto lens, the cameras mounted on a pan/tilt device;

using a computer vision algorithm to detect vehicles moving on a roadway despite motion created by movement of the UAV, wherein the computer vision algorithm is operated by the on-board computing system;

moving the cameras mounted on the pan/tilt device under the direction of the computer vision algorithm to maintain a target vehicle of the detected moving vehicles in view; and

measuring a speed of the target vehicle,

wherein the UAV control unit uses Global Positioning System (GPS) coordinates and camera geometry, altitude, and location of the target vehicle in the image to measure the target vehicle speed.

2. The method according to claim 1 wherein the computer vision algorithm automatically selects the target vehicle to track.

3. The method according to claim 2 wherein the tracking of the target vehicle is by active control of the pan/tilt device holding the cameras.

4. The method according to claim 1 wherein the cameras are approximately axis-aligned cameras, wherein the camera with the wide angle lens is used to track the target vehicle and the camera with the telephoto lens is used to capture a high resolution snapshot while the target vehicle is centered in the wide angle image frame.

5. The method according to claim 1 wherein a timing of the target vehicle passing locally calibrated landmarks is used to measure the target vehicle speed.

6. The method according to claim 1 wherein the computer vision algorithm tracks a vehicle by detecting the vehicle in every camera image while the vehicle is within a view of the camera with the wide angle lens.

7. The method according to claim 1 wherein the computer vision algorithm employs Point-Motion-Pairs to establish correspondences among keypoint features of vehicles.

8. A method for measuring ground vehicle speed comprising:

positioning an unmanned aerial vehicle (UAV) in proximity to a roadway, the UAV flying under control and navigation of a UAV control unit, and further carrying camera and monitoring equipment, the camera and monitoring equipment including an onboard computing system, and a camera with a wide angle lens and a camera with a telephoto lens, the cameras mounted on a pan/tilt device;

using a computer vision algorithm to detect vehicles moving on a roadway despite motion created by movement of the UAV, wherein the computer vision algorithm is operated by the on-board computing system;

moving the cameras mounted on the pan/tilt device under the direction of the computer vision algorithm to maintain a target vehicle of the detected moving vehicles in view; and

measuring a speed of the target vehicle, including following the target vehicle for at least a kilometer or half a minute to measure the vehicle'"'"'s speed by a distance the UAV has travelled as measured by a Global Positioning System (GPS).

9. A method for detecting, tracking and estimating speeds of vehicles on a roadway, the method comprising:

positioning an unmanned aerial vehicle (UAV) in proximity to a roadway, the UAV flying under control and navigation of a UAV control unit, and further carrying camera and monitoring equipment, the camera and monitoring equipment including an onboard computing system, and a camera with a wide angle lens and a camera with a telephoto lens, the cameras mounted on a pan/tilt device;

directing the pan/tilt controlled cameras to view the roadway; and

engaging a moving vehicle monitoring procedure which includes;

detecting moving vehicles from a video feed of the camera with the wide-angle-lens;

estimating vehicle speed for each moving vehicle in the video feed;

determining a target vehicle to continue to track from among the vehicles in the video feed;

tracking the target vehicle;

recording video evidence of the target vehicle by the camera with the wide-angle lens;

serving the pan/tilt device to center the target vehicle in the frame of the video feed;

measuring the target vehicle speed;

taking a zoomed-in snapshot of the target vehicle license plate with the camera with the telephoto lens when the target vehicle license plate is centered in the frame of the camera with the wide angle lens; and

recording the target vehicle license plate when it is determined the target vehicle license plate is readable and that sufficient speed evidence of the target vehicle has been collected.

10. The method of claim 9 wherein the tracking algorithm is a computer vision algorithm.

11. The method of claim 10 wherein the computer vision algorithm detects moving vehicles despite motion created by movement of the UAV.

12. The method according to claim 9 wherein the computer vision algorithm automatically picks out the target vehicle to track

13. The method according to claim 9 wherein the tracking of the target vehicle is by active control of the pan tilt controlled cameras.

14. The method according to claim 9 wherein the cameras are arranged as approximately axis-aligned cameras on the pan/tilt device mounted on the UAV to obtain high-resolution images of vehicles from the camera with the telephoto lens while guiding the pan/tilt device under visual tracking control of the camera with the wide angle lens.

15. The method according to claim 9 wherein the UAV is used to measure the speed of a ground vehicle by use of Global Positioning System (GPS) coordinates of the aircraft and the geometry of direction of a pan/tilt controlled cameras while maintaining visual tracking of the vehicle.

16. A speed measurement system to detect, track and determine a speed of a vehicle on a roadway, the system comprising:

an unmanned aerial vehicle (UAV);

a UAV control unit positioned on the UAV and configured to control the operation and navigation of the UAV;

a camera and monitoring system carried on the UAV and controlled by the UAV control unit, and including;

a camera with a wide angle lens and a camera with a telephoto lens, the cameras mounted on a pan/tilt device connected to the UAV by a mounting bracket, and

an on-board computing system configured to operate a moving vehicle monitoring procedure (MVMP) which includes a tracking algorithm that actively controls movement of the pan/tilt device, to coordinate movement of the camera with the wide angle lens and the camera with the telephoto lens to obtain an image of a license plate from a target vehicle being tracked by the MVMP procedure and to determine a speed of the target vehicle,

wherein the UAV control uses Global Positioning System (GPS) coordinates and camera geometry, altitude, and location of the target vehicle in the image to measure the target vehicle speed.

17. The system according to claim 16 wherein the cameras are used as approximately axis-aligned cameras on a pan/tilt device mounted on the UAV to obtain high-resolution images of vehicles from the camera with the telephoto lens while guiding the pan/tilt device under visual tracking control of the camera of the wide angle lens.

18. The system according to claim 16 wherein the MVMP includes a computer vision algorithm used to detect moving vehicles on a roadway despite motion created by movement of the UAV, wherein the computer vision algorithm is operated by the on-board computing system.

19. The system according to claim 18 wherein the computer vision algorithm is configured to automatically select the target vehicle to track.