Camera-based speed estimation and system calibration therefor

• US 10,015,394 B2
• Filed: 10/06/2015
• Issued: 07/03/2018
• Est. Priority Date: 10/06/2015
• Status: Active Grant

• Current Assignee: Genetec Incorporated (Noritsu Koki Co., Ltd.)
• Original Assignee: Genetec Incorporated (Noritsu Koki Co., Ltd.)
• Inventors: Blais-Morin, Louis-Antoine
• Primary Examiner(s): Pontius, James
• Application Number:
  US14/876,016
• Publication Number:
  US 20170098304A1
• Time in Patent Office: 1,001 Days
• CPC Class Codes: F16M 11/06 allowing pivoting
• CPC Class Codes: F16M 11/28 Undercarriages for supports with one single telescoping pillar
• CPC Class Codes: G06T 2200/24 involving graphical user interfaces [GUIs]
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/10048 Infrared image
• CPC Class Codes: G06T 2207/30236 Traffic on road, railway or crossing
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
• CPC Class Codes: G08G 1/052 with provision for determining speed or overspeed speed measuring in general G01P
• CPC Class Codes: H04N 23/60 Control of cameras or camera modules
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof

Abstract

Provided is a visual vehicle speed estimation system based on camera output and calibration for such a vehicle speed estimation system. The calibration allows use of the system where the absolute camera position is unknown. Calibration determines an absolute-position-independent relationship between an image space placement and a physical space position relative to the camera. The calibration is done on the basis of camera output using vehicle features of known dimensions and some assumed physical constraints related thereto to provide a conversion relationship between image coordinates and physical space coordinates in a physical space defined in relation to the camera. This relationship is then used to estimate vehicle speeds based only on the visual information provided by the camera. Abstract is not to be interpreted as limiting.

Claims

1. A method for calibrating a vehicle speed estimation system that employs a camera having a set of camera parameters to estimate the speed of vehicles based on images received from the camera, the images defined in an image space and the vehicles and camera each belonging to a physical space and having an absolute position therein, the method comprising:

a) receiving at a controller from the camera a calibration image stream comprising a first image of a vehicle captured at a first time and a second image of the vehicle captured at a second time, the first and second images having the same image space;

b) at a controller executing controller logic to identify in each of the first and second image a vehicle feature having a known dimension in the physical space, the identifying comprising determining a placement of the vehicle feature in the image space at each of the first and second times;

c) at the controller executing controller logic to define on the basis of the placement of the vehicle feature in the image space at each of the first and second times an absolute-position-independent relationship between an image space placement and a physical space position relative to the camera;

d) at the controller, using controller logic, applying the absolute-position-independent relationship between an image space placement and a physical space position relative to the camera in a speed estimation algorithm to determine the speed of a vehicle captured by the camera.

2. The method of claim 1, wherein executing controller logic to define on the basis of the placement of the vehicle feature in the image space at each of the first and second times an absolute-position-independent relationship between an image space placement and a physical space position relative to the camera comprises:

assuming a set of physical constraints concerning the vehicle feature whereby the absolute-position-independent relationship between an image space placement and a physical space position relative to the camera is further defined on the basis of the set of physical constrains.

3. The method of claim 2, wherein the set of physical constraints comprises at least one of:

a) a known dimension of the feature;

b) that the vehicle feature is perpendicular to a direction of travel of the vehicle; and

c) that the vehicle feature maintains a constant vertical distance from the camera.

4. The method of claim 2, wherein the absolute-position-independent relationship between an image space placement and a physical space position relative to the camera is a selected absolute-position-independent relationship, and wherein the executing controller logic to define the selected absolute-position-independent relationship comprises:

a) assuming an assumed set of camera parameters;

b) defining a tentative absolute-position-independent relationship between an image space placement and a physical space position relative to the camera on the basis of the assumed set of camera parameters;

c) evaluating the validity of the tentative absolute-position-independent relationship and upon confirming the validity establishing the tentative absolute-position-independent relationship as the selected absolute-position-independent relationship.

5. The method of claim 4, wherein evaluating the validity of the tentative absolute-position-independent relationship comprises

d. using the tentative absolute-position-independent relationship computing a physical space location of at least a portion of the vehicle at each of the first and second times;

e. verifying compliance with the assumed set of physical constraints of the computed space location of the at least a portion of the vehicle at each of the first and second times.

6. The method of claim 5, wherein:

a) the set of camera parameters is a first set of camera parameters from among a plurality of sets of camera parameters, further comprising repeating steps a-e with each of the plurality of different sets of camera parameters;

b) verifying compliance with the set of physical constraints comprises using the tentative absolute-position-independent relationship and the set of physical constraints to compute a tentative image space placement for the vehicle feature and computing an error between the tentative image space placement for the vehicle feature and the placement of the vehicle feature in the image space at at least one of the first and second times; and

c) verifying compliance with the set of physical constraints comprises selecting a selected set of camera parameters from among the plurality of sets of camera parameters providing the lowest error.

7. The method of claim 4, wherein the assumed set of camera parameters comprises at least one of a) a tilt angle (θ

_t); and

b) a roll angle (θ

_r).

8. The method of claim 4, wherein the assumed set of camera parameters comprises tilt angle (θ

_t) and a roll angle (θ

_r), and wherein the absolute-position-independent relationship between an image space placement and a physical space position relative to the camera is a function of the matrix M where;

9. The method of claim 8, wherein the absolute-position-independent relationship between an image space placement and a physical space position relative to the camera comprises the function ƒ

_y(X,Y,z) for converting from image-space coordinates to physical-space coordinates where;

10. The method of claim 8, wherein the absolute-position-independent relationship between an image space placement and a physical space position relative to the camera comprises the functions ƒ

_X<sup>−

1</sup>(x,y,z) and ƒ

_X<sup>−

1</sup>(x,y,z) for converting from physical-space coordinates to image-space coordinates where;

11. A camera-based visual speed estimation system for estimating the speed of vehicles on the basis of video output from of a camera, the camera-based visual speed estimation system comprising:

a) a controller camera input for receiving from the video output of the camera comprising at least first and second images of a travelling vehicle captured at respective first and second times, the first and second images being defined in an image space and the travelling vehicle and camera each belonging to a physical space and having respective absolute positions therein;

b) a tangible controller in communication with the camera input and receiving therefrom the video data from the camera comprising at least the first and second images, the controller comprising calibration logic executable on the controller for causing the controller to;

i. identify in each of the first and second image a vehicle feature having a known dimension in the physical space, the identifying comprising determining a placement of the vehicle feature in the image space at each of the first and second times;

ii. define on the basis of the placement of the vehicle feature in the image space at each of the first and second times an absolute-position-independent relationship between an image space placement and a physical space position relative to the camera;

iii. apply the absolute-position-independent relationship between an image space placement and a physical space position relative to the camera in a speed estimation algorithm to derive speed estimation data indicative of the speed of a vehicle captured by the camera;

c) a controller output in communication with the controller for receiving from the controller the speed estimation data and outputting the speed estimation data.

12. The camera-based visual speed estimation system of claim 11, wherein the calibration logic executable on the controller for causing the controller to define on the basis of the placement of the vehicle feature in the image space at each of the first and second times an absolute-position-independent relationship between an image space placement and a physical space position relative to the camera comprises:

logic for causing the controller to assume a set of physical constraints concerning the vehicle feature whereby the absolute-position-independent relationship between an image space placement and a physical space position relative to the camera is further defined on the basis of the set of physical constrains.

13. The camera-based visual speed estimation system of claim 12, wherein the set of physical constraints comprises at least one of:

a) a known dimension of the feature;

b) that the vehicle feature is perpendicular to a direction of travel of the vehicle; and

c) that the vehicle feature maintains a constant vertical distance from the camera according to the physical space.

14. The camera-based visual speed estimation system of claim 12, wherein the absolute-position-independent relationship between an image space placement and a physical space position relative to the camera is a selected absolute-position-independent relationship, and wherein the calibration logic executable on the controller for causing the controller to define the selected absolute-position-independent relationship comprises logic executable on the controller for causing the controller to:

a) assume an assumed set of camera parameters;

b) define a tentative absolute-position-independent relationship between an image space placement and a physical space position relative to the camera on the basis of the assumed set of camera parameters;

c) evaluate the validity of the tentative absolute-position-independent relationship and upon confirming the validity establishing the tentative absolute-position-independent relationship as the selected absolute-position-independent relationship.

15. The camera-based visual speed estimation system of claim 14, wherein the logic executable on the controller for causing the controller to evaluate the validity of the tentative absolute-position-independent relationship comprises logic executable on the controller for causing the controller to:

f. using the tentative absolute-position-independent relationship compute a physical space location of at least a portion of the vehicle at each of the first and second times;

g. verify compliance with the assumed set of physical constraints of the computed space location of the at least a portion of the vehicle at each of the first and second times.

16. The camera-based visual speed estimation system of claim 15, wherein:

a) the set of camera parameters is a first set of camera parameters from among a plurality of sets of camera parameters, wherein the calibration logic further comprises logic executable on the controller for causing the controller to repeat steps a-e with each of the plurality of different sets of camera parameters;

b) the logic executable on the controller for causing the controller to verify compliance with the set of physical constraints comprises logic executable on the controller for causing the controller to using the tentative absolute-position-independent relationship and the set of physical constraints to compute a tentative image space placement for the vehicle feature and to compute an error between the tentative image space placement for the vehicle feature and the placement of the vehicle feature in the image space at at least one of the first and second times; and

c) the logic executable on the controller for causing the controller to verify compliance with the set of physical constraints comprises logic executable on the controller for causing the controller to select a selected set of camera parameters from among the plurality of sets of camera parameters providing the lowest error.

17. The camera-based visual speed estimation system of claim 14, wherein the assumed set of camera parameters comprises at least one of a) a tilt angle (θ

_t); and

b) a roll angle (θ

_r).

18. The camera-based visual speed estimation system of claim 14, wherein the assumed set of camera parameters comprises a tilt angle (θ

_t) and a roll angle (θ

_r), and wherein the absolute-position-independent relationship between an image space placement and a physical space position relative to the camera is a function of the matrix M where;

19. The camera-based visual speed estimation system of claim 18, wherein the absolute-position-independent relationship between an image space placement and a physical space position relative to the camera comprises the function ƒ

_y(X,Y,z) for converting from image-space coordinates to physical-space coordinates where;

20. The camera-based visual speed estimation system of claim 18, wherein the absolute-position-independent relationship between an image space placement and a physical space position relative to the camera comprises the functions ƒ

_X<sup>−

1</sup>(x,y,z) and ƒ

_X<sup>−

1</sup>(x,y,z) for converting from physical-space coordinates to image-space coordinates where;