AUTOMATIC SPATIAL CALIBRATION OF CAMERA NETWORK

• US 20160012589A1
• Filed: 07/07/2015
• Published: 01/14/2016
• Est. Priority Date: 07/11/2014
• Status: Active Grant

• Current Assignee: AGT International GMBH
• Original Assignee: AGT International GMBH
• Inventors: SCHLATTMANN, Markus, HEREMANS, Roel, HAMER, Henning, SAWCHUK, Arsenii
• Granted Patent: US 9,928,594 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06T 17/05 Geographic models
• CPC Class Codes: G06T 2200/21 involving computational photography
• CPC Class Codes: G06T 2207/10004 Still image; Photographic image
• CPC Class Codes: G06T 2207/20008 Globally adaptive
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 2207/30256 Lane; Road marking
• CPC Class Codes: G06T 2210/61 Scene description
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
• CPC Class Codes: G08G 1/04 using optical or ultrasonic detectors
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

A method for automatic spatial calibration of a network of cameras along a road includes, processing a frame that is obtained from each camera of the network to automatically identify an image of a pattern of road markings that have a known spatial relationship to one another. The identified images are used to calculate a position of each camera relative to the pattern of road markings that is imaged by that camera. Geographical information is applied to calculate an absolute position of a field of view of each camera. A global optimization is applied to adjust the absolute position of the field of view of each camera of the camera network relative to an absolute position of the fields of view of other cameras of the camera network

Claims

1. A method for automatic spatial calibration of a network of cameras along a road, the method comprising:

processing a frame that is obtained from each camera of the network to automatically identify an image of a pattern of road markings that have a known spatial relationship to one another;

using the identified images to calculate a position of each camera relative to the pattern of road markings that is imaged by that camera;

applying geographical information to calculate an absolute position of a field of view of each camera; and

applying a global optimization to adjust the absolute position of the field of view of each camera of the camera network relative to the absolute positions of the fields of view of other cameras of the camera network.

2. The method of claim 1, wherein identifying the markings comprises applying a Hough transform, a scale invariant feature transform (SIFT), or a speeded up robust features (SURF) detection to the frame.

3. The method of claim 1, wherein the road markings comprise broken line lane separation markings.

4. The method of claim 1, wherein the pattern of road markings is two dimensional.

5. The method of claim 1, wherein using the identified images comprises using a measured position of the identified images within the frame to calculate the position of the camera relative to the pattern.

6. The method of claim 1, wherein the geographical information comprises a known geographical position of the camera.

7. The method of claim 1, wherein the geographical information comprises a known geographical position of the road or of the pattern.

8. The method of claim 1, wherein the field of view is defined relative to a global coordinate system.

9. The method of claim 1, wherein applying the global optimization comprises minimizing a value of a function that is indicative of a misalignment of the field of view of a camera of the camera network relative to the fields of view of the other cameras of the camera network.

10. The method of claim 1, wherein applying the global optimization comprises applying a rotation, translation, size adjustment, or shape modification to an absolute position of a field of view of a camera of the camera network.

11. The method of claim 10, wherein the rotation, translation or size adjustment is configured to align a projection into a horizontal plane of a marking image in the frame that is acquired by one of the cameras with a projection into the horizontal plane of a marking image in the frame that is acquired by a camera of said other cameras.

12. The method of claim 1, wherein the frame that is obtained from a camera of the camera network is obtained by application of background learning to a plurality of frames that are acquired by that camera.

13. A camera network system comprising:

a plurality of cameras that are positioned at different places along a road, the road including at least one pattern of road markings; and

a processing unit in communication with a computer readable medium, wherein the computer readable medium contains a set of instructions wherein the processing unit is designed to carry out the set of instructions to;

obtain a frame from each camera of said plurality of cameras;

identify in each obtained frame an image of a pattern of said at least one pattern;

calculate a position of each camera relative to said pattern of said at least one pattern;

apply geographical information to calculate an absolute position of a field of view of each camera; and

apply a global optimization to align the absolute position of the field of view of each camera of said plurality of cameras relative to the absolute position of the fields of view of other cameras of said plurality of cameras.

14. The system of claim 13, wherein said at least one pattern of road markings comprises at least two broken line lane separation markings.

15. The system of claim 13, wherein a geographic position of a camera of said plurality of cameras is known.

16. The system of claim 13, wherein a camera of said plurality of cameras is substantially stationary.

17. The system of claim 13, wherein the instructions to apply the global optimization comprise instructions to minimize a value of a function that is indicative of a misalignment of the field of view of a camera of said plurality of cameras relative to the fields of view of the other cameras of said plurality of cameras.

18. The system of claim 13, wherein the instructions to apply the global optimization comprise instructions to rotate, translate, adjust a size, or modify a shape of an absolute position of a field of view of a camera of said plurality of cameras.

19. The system of claim 13, wherein the absolute position of a field of view is defined relative to a global coordinate system.

20. A non-transitory computer readable storage medium having stored thereon instructions that when executed by a processor will cause the processor to perform the method of:

obtaining a frame from each camera of a network of cameras that are arranged along a road;

identifying an image of a pattern of road markings in each obtained frame;

calculating a position of each camera of the network relative to the pattern corresponding to the image in the frame that was obtained from that camera;

applying geographical information to calculate an absolute position of a field of view of each camera of the network; and

adjusting the absolute position of the field of view of a camera of the network in relative to the absolute position of the field of view of another camera of the network.