Systems and methods for automatic camera calibration

• US 8,059,154 B1
• Filed: 10/06/2008
• Issued: 11/15/2011
• Est. Priority Date: 10/06/2008
• Status: Active Grant

• Current Assignee: Cognyte Technologies Israel Ltd. (Cognyte Software Ltd.)
• Original Assignee: Verint Systems Limited (Verint Systems Incorporated)
• Inventors: Horovitz, Itsik, Kiro, Shmuel
• Primary Examiner(s): Hannett; James
• Application Number:
  US12/245,836
• Time in Patent Office: 1,135 Days
• Field of Search: 348/184
• US Class Current: 348/184
• CPC Class Codes: G06T 2207/20076 Probabilistic image processing
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
• CPC Class Codes: H04N 17/002 for television cameras

Abstract

Camera calibration for a particular camera is automated based on line segments derived from historical data of images passing within that camera'"'"'s view. Objects having a stable history of orientation are selected for conversion to line segments. A model is derived that fits the way most of the objects behave. The model is specific for the camera supplying the historical data and is specific for the location and position of the camera in that location. The model is able to predict how an object would appear at different locations in the scene and can be applied to data collected by that camera in that location in the future. Outputs include internal camera parameters, a vertical vanishing point and horizontal vanishing line, an equation of the floor and height of objects touching the floor relative to the average person height in the calibration video data.

Claims

1. An automated calibration system comprising:

a video monitoring device;

a recorder that receives video data from a camera; and

an auto-calibration module that receives historical calibration video data from the video monitoring device and returns output to calibrate the video monitoring device, the output comprising internal parameters, vertical vanishing point, horizontal vanishing line, an equation for a floor, the output providing a model from which objects whose image is captured by the video monitoring device are analyzed.

2. The system of claim 1, further comprising the video monitoring device to be calibrated according to the internal parameters, the video monitoring device comprising a camera.

3. The system of claim 1, wherein the output to calibrate the video monitoring device is associated with the video monitoring device providing the historical calibration video data and a fixed location in which the video monitoring device is positioned when the historical calibration video data is collected.

4. The system of claim 1, wherein the auto-calibration module calculates the output by:

converting the historical video data into a set of object lines, each line of the set of object lines associated with a length, an orientation and an original object index and time.

5. The system of claim 1, wherein the auto-calibration module calculates the output by:

calculating a vanishing point using a RANSAC algorithm to remove outliers and generate a vanishing point.

6. The system of claim 1, wherein the auto-calibration module calculates the output by:

determining a first approximation of a horizontal vanishing line.

7. The system of claim 1, wherein the auto-calibration module calculates the output by:

determining a first approximation of internal parameters for the video monitoring device;

determining an equation of the floor; and

determining a final set of internal parameters and setting the internal parameters of the video monitoring device to the final set of internal parameters.

8. A method of calibrating a camera comprising:

receiving historical video data collected by a camera to be calibrated;

generating a model for the camera to be calibrated from the historical video data for a fixed location in which the camera was placed when the historical video data was collected;

converting an object of a plurality of moving objects into a set of object lines, wherein each object line of the set of object lines is associated with a length, an orientation, an original object index and time; and

calibrating the camera by applying the generated model to video data received from the camera at the fixed location.

9. The method of claim 8, further comprising:

calculating the length of an object line of the set of object lines by observing movement of the object line over time and taking measurements of the object line.

10. The method of claim 9, further comprising:

calculating the orientation of the object line using second order Moments.

11. The method of claim 10, further comprising:

calculating a vanishing point for the object line using a RANSAC algorithm wherein object lines not fitting the model are removed from the set of converted objects and a vanishing point and a subset of object lines fitting the model are returned.

12. The method of claim 8, further comprising:

generating a first approximation of a horizontal vanishing line by selecting a line intersecting end points of line pairs originating from an object of the plurality of moving objects and defining a line through intersection points using a least squares algorithm to determine the horizontal vanishing line;

determining a first approximation of camera internal parameters;

determining an equation of a floor; and

determining a final set of internal camera parameters.

13. A computer readable storage medium comprising computer executable program instructions that when executed:

receive historical video data for a video monitoring device to be calibrated, the video monitoring device placed in a fixed position in a fixed location;

generate a model for the video monitoring device to be calibrated from the historical video data for the fixed position in the fixed location in which the video monitoring device was placed when the historical video data was collected;

generate a first approximation of a horizontal vanishing line for an object by selecting a line intersecting end points of lines pairs originating from the object and defining a line through intersection points using a least squares algorithm to determine the first approximation of the horizontal vanishing line; and

calibrating the camera by applying the generated model to video data received from the video monitoring device at the fixed location.

14. The computer readable storage medium of claim 13, further comprising computer executable program instructions that when executed:

generate the model by;

converting an object of a plurality of moving objects into a set of object lines, wherein each object line of the set of object lines is associated with a length, an orientation, an original object index and time;

calculating the length of an object line of the set of object lines by observing movement of the object line over time and taking measurements of the object line; and

calculating a vanishing point for the object line using a RANSAC algorithm wherein object lines not fitting the model are removed from the set of converted objects and a vanishing point and a subset of object lines fitting the model are returned.

15. The computer readable storage medium of claim 13, comprising further computer executable program instructions that when executed:

determine a first approximation of internal parameters for the video monitoring device.

16. The computer readable storage medium of claim 15, comprising further computer executable program instructions that when executed:

determine an equation of a floor.

17. The computer readable storage medium of claim 13, comprising further computer executable program instructions that when executed:

determine a final set of internal parameters for the video monitoring device.

18. The computer readable storage medium of claim 17, comprising further computer executable program instructions that when executed:

analyze objects detected by the video monitoring device when placed in the fixed position in the fixed location using the generated model.