Method and apparatus for estimating the velocity vector of multiple vehicles on non-level and curved roads using a single camera

• US 7,920,959 B1
• Filed: 04/28/2006
• Issued: 04/05/2011
• Est. Priority Date: 05/01/2005
• Status: Active Grant

• Current Assignee: Christopher Reed Williams
• Original Assignee: Christopher Reed Williams
• Inventors: Williams, Christopher Reed
• Primary Examiner(s): Keith; Jack
• Assistant Examiner(s): Nguyen; Chuong P
• Application Number:
  US11/414,602
• Time in Patent Office: 1,803 Days
• Field of Search: 701/1, 701/28, 701/33, 701/117, 702/141, 702/142, 702/150, 702/152, 702/153, 345/419, 345611-615, 348135-139, 348/145, 348/148, 348/149, 382190-195, 382276-286
• US Class Current: 701/117
• CPC Class Codes: G06T 17/00 Three dimensional [3D] modelling, e.g. data description of 3D objects
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30236 Traffic on road, railway or crossing
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: G08G 1/0104 Measuring and analyzing of parameters relative to traffic conditions
• CPC Class Codes: G08G 1/056 with provision for distinguishing direction of travel

Abstract

A method and an apparatus is provided for determining the velocity vector, speed, and direction of moving vehicles traveling on roadways which change elevation and have curves. A camera mounted above a roadway observes at least one vehicles traveling in different directions on different roads. Each pixel in the two-dimensional optical image is mapped into a three-dimensional real world location to enable the moving vehicles to be analyzed in the three-dimensional real world as they travel on roads, which may change elevation and may have curves. The estimated vehicle velocities may be displayed on an image near the moving vehicle. Individual images with vehicle velocities may be posted on an Internet site. The sequence of optical images, or streaming video, with the displayed vehicle velocities may also be used for television news programs that show traffic moving on the roadways.

Claims

1. A system for determining velocity in three dimensions of at least one vehicle, from a two-dimensional image of the at least one vehicle in a roadway, comprising:

a camera configured to acquire two-dimensional images of a three-dimensional roadway system, the roadway system comprising at least one road, the roadway system including at least one vehicle moving relative to a background scene,

a processor, coupled to the camera, configured to receive and store an image sequence of the three-dimensional roadway system, said processor configured to record the image sequence as a series of frames, each frame including image data forming a two-dimensional representation including a plurality of image regions depicting the moving vehicles and the background scene at an associated point in time, the processor configured to map each pixel of the two-dimensional background image to a three-dimensional model of the three-dimensional roadway system to produce a conversion map for converting two dimensional pixel locations to three-dimensional real-world locations, the processor configured to convert pixels assigned in the two-dimensional image to each identified vehicle to a three-dimensional real world location using the conversion map, the processor configured to identify a moving vehicle of the plurality of moving vehicles by comparing a plurality of frames of the image sequence and identifying a image region of the image sequence including the moving vehicle, the processor configured to calculate a location of the moving vehicle in the three-dimensional roadway system in each of the plurality of frames containing the image region including the moving vehicle, and the processor configured to calculate a velocity vector consisting of a speed and a direction of the moving vehicle in the three-dimensional roadway system in each of the plurality of frames containing the image region including the moving vehicle.

2. The system of claim 1, wherein the processor is configured to generate a background image by constructing a long-term average of collected optical images, and is configured to identify regions of the background image predetermined to contain moving vehicles, the processor is configured to measure a difference between the long-term average background image and a current image to produce a difference image, the processor is configured to process the predetermined regions in the difference image to produce regions of difference, and the processor is configured to process the regions of difference to separate single vehicles from multiple vehicles identify individual vehicles;

wherein the processor is configured to determine a location of an individual vehicle in each frame of the two-dimensional image by calculating the centroid of the pixels associated with each vehicle and assigning a pixel in each frame of the two-dimensional image to each identified vehicle.

3. The system of claim 2, wherein the processor is configured to measure a difference between determined three-dimensional real world locations of a vehicle for different frames of the two-dimensional image to determine a distance traveled by the vehicle in the three-dimensional real world between frames, and the processor is configured to divide the difference by a time between frames to determine velocity of the vehicle.

4. The system of claim 1, wherein the processor is configured to map each pixel of the two-dimensional background image to a three-dimensional model of the three-dimensional roadway system to produce a conversion map, the processor is configured to accept a survey of at least a portion of the at least one road in the three-dimensional roadway system, the processor configured to produce a three-dimensional data map of the roadway system comprising a plurality of road location data points;

the processor configured to accept a survey of a plurality of three-dimensional reference points within a field of view of the camera; and

the processor configured to correlate the plurality of three-dimensional reference points within the field of view of the camera to two-dimensional pixels in the background image to convert the plurality of road location data points to pixels in the background image.

5. The system of claim 4, wherein the processor is configured to map each pixel of the two-dimensional background image to a three-dimensional model by being configured to locate the reference points in the video image after the camera is moved;

and being configured to re-correlate the plurality of three-dimensional reference points within the field of view of the camera to two-dimensional pixels in the background image to convert the plurality of road location data points to pixels in the background image.

6. The system of claim 1, further comprising:

an image processor, coupled to the processor, the image processor configured to display velocity of the at least one vehicle on each of the plurality of frames near the image region including the at least one vehicle.

7. The system of claim 1, further comprising:

an image processor, coupled to the processor, the image processor configured to display velocity of the at least one vehicle on each of the plurality of frames in a fixed location in the frame.

8. The system of claim 1, further comprising:

an image processor coupled to the processor, the image processor configured to display velocity of the at least one vehicle on a frame and a communications means for displaying said frame on an Internet.

9. The system of claim 4, wherein

the processor is configured to measure camera tilt and pan angle after camera movement to calculate a difference in camera tilt and pan angle from a previous camera position;

and

the processor is configured to update the plurality of road location data points converted to pixels in the background image by applying the difference in camera tilt and pan angle to mathematically calculate new values for corresponding pixels in the background image.