Method and device for determining external parameter of stereoscopic camera

• US 11,010,924 B2
• Filed: 09/30/2017
• Issued: 05/18/2021
• Est. Priority Date: 10/12/2016
• Status: Active Grant

• Current Assignee: Hangzhou Hikvision Digital Technology Company Limited
• Original Assignee: Hangzhou Hikvision Digital Technology Company Limited
• Inventors: Zhang, Di, Gong, Hui, Ren, Ye
• Primary Examiner(s): Hunter, Mishawn N.
• Application Number:
  US16/340,901
• Publication Number:
  US 20190236807A1
• Time in Patent Office: 1,326 Days
• CPC Class Codes: G06T 2200/04 involving 3D image data
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/20032 Median filtering
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06T 7/85 Stereo camera calibration
• CPC Class Codes: H04N 13/246 Calibration of cameras
• CPC Class Codes: H04N 23/60 Control of cameras or camera modules

Abstract

Embodiments of application discloses a method and device for determining external parameters of a stereo camera. The method comprises: acquiring ground surface data of a location of the stereo camera; generating, according to three-dimensional coordinates of each pixel in the ground surface data in a camera coordinate system corresponding to the stereo camera, a first plane equation in the camera coordinate system that conforms to the three-dimensional coordinates of the pixels; determining, according to a distance between the origin of the camera coordinate system and the ground surface, a setting height of the stereo camera; and generating, according to the first plane equation, a plane normal vector corresponding to the first plane equation; and determining, according to the plane normal vector, a pitch angle of the stereo camera and an inclination angle of the stereo camera. The external parameters of the stereo camera can be determined without using a reference point in a scene.

Claims

1. A method for determining external parameters of a stereo camera, comprising:

acquiring ground surface data of a location where the stereo camera is positioned, wherein the ground surface data includes a two-dimensional image and a disparity image of the ground surface or includes a two-dimensional image and a depth image of the ground surface;

generating, according to three-dimensional coordinates of each of pixels in the ground surface data in a camera coordinate system corresponding to the stereo camera, a first plane equation in the camera coordinate system that conforms to the three-dimensional coordinates of the pixels;

determining, according to a distance between an origin of the camera coordinate system and the ground surface, a setting height of the stereo camera;

generating, according to the first plane equation, a plane normal vector corresponding to the first plane equation; and

determining, according to the plane normal vector, the external parameters of the stereo camera without using a vertical edge perpendicular to the ground in a scene as a reference point, wherein the external parameters comprises a pitch angle of the stereo camera and an inclination angle of the stereo camera.

2. The method according to claim 1, further comprising:

performing filtering processing on the ground surface data when the ground surface data does not meet a preset condition;

wherein the filtering processing comprises median filtering in time domain and median filtering in spatial domain.

3. The method according to claim 1, wherein, generating, according to three-dimensional coordinates of each of pixels in the ground surface data in a camera coordinate system corresponding to the stereo camera, a first plane equation in the camera coordinate system that conforms to the three-dimensional coordinates of the pixels comprises:

selecting at least three pixels from pixels in the ground surface data, and using three-dimensional coordinates of the at least three pixels in the camera coordinate system corresponding to the stereo camera as a first sample set;

generating, according to the first sample set, a second plane equation that conforms to three-dimensional coordinates of pixels in the first sample set in the camera coordinate system;

determining, among the pixels in the ground surface data, all pixels that meet the second plane equation, and using the three-dimensional coordinates of the pixels that meet the second plane equation in the camera coordinate system as a second sample set; and

generating, according to the second sample set, the first plane equation in the camera coordinate system that conforms to the three-dimensional coordinates of the pixels in the ground surface data in the camera coordinate system.

4. The method according to claim 3, wherein, generating, according to the second sample set, the first plane equation in the camera coordinate system that conforms to the three-dimensional coordinates of the pixels in the ground surface data in the camera coordinate system comprises:

performing fitting processing on the second sample set by using a least squares method to generate the first plane equation that conforms to the three-dimensional coordinates of the pixels in the ground surface data in the camera coordinate system.

5. The method according to claim 3, wherein, selecting at least three pixels from pixels in the ground surface data comprises:

selecting at least three pixels from the pixels in the ground surface data by using a random sample consensus RANSAC algorithm.

6. The method according to claim 5, wherein, selecting at least three pixels from pixels in the ground surface data, and using three-dimensional coordinates of the at least three pixels in the camera coordinate system corresponding to the stereo camera as a first sample set and generating, according to the first sample set, a second plane equation that conforms to three-dimensional coordinates of pixels in the first sample set in the camera coordinate system comprises:

selecting at least three pixels from the pixels in the ground surface data by using a RANSAC algorithm, and using the three-dimensional coordinates of the at least three pixels in the camera coordinate system corresponding to the stereo camera as the first sample set;

generating, according to the first sample set, a current plane equation that conforms to the three-dimensional coordinates of pixels in the first sample set in the camera coordinate system;

determining whether the current plane equation satisfies a preset condition;

using the current plane equation as the second plane equation when the current plane equation satisfies the preset condition;

increasing the number of iterations by 1 when the current plane equation does not satisfy the preset condition and returning to perform the steps of selecting at least three pixels from pixels in the ground surface data, and using three-dimensional coordinates of the at least three pixels in the camera coordinate system corresponding to the stereo camera as a first sample set;

wherein an initial value of the number of iterations is 0.

7. The method according to claim 6, wherein, the preset condition comprises:

the number of pixels, among the pixels in the ground surface data, that meet the current plane equation reaches a preset first threshold, or the number of iterations corresponding to the current plane equation reaches a preset second threshold.

8. The method according to claim 1, wherein, determining, according to the plane normal vector, a pitch angle of the stereo camera and an inclination angle of the stereo camera comprises:

determining, in the camera coordinate system, a first angle by which a corresponding axis z_c rotates about an axis x in a clockwise direction when the stereo camera rotates from a standard attitude to a current attitude, and using the first angle as the pitch angle of the stereo camera; and

determining, in the camera coordinate system, a second angle by which a corresponding axis x_c rotates about an axis z in a clockwise direction when the stereo camera rotates from the standard attitude to the current attitude, and using the second angle as the inclination angle of the stereo camera;

wherein a three-dimensional coordinate system defined by the axis x_c, the axis z_c, and an axis y_c is the camera coordinate system;

a three-dimensional coordinate system defined by the axis x, the axis z, and an axis y is a world coordinate system;

when directions of the axis x_c and the axis z_c coincide with directions of the axis x and the axis z, and when direction of the axis y_c is opposite to direction of the axis y, the stereo camera is in the standard attitude.

9. A non-transitory storage medium, which is used for storing executable program codes, wherein the executable program codes, when being executed, are configured to implement the method for determining external parameters of a stereo camera according to claim 1.

10. A device for determining external parameters of a stereo camera, comprising:

an acquiring unit, configured for acquiring ground surface data of a location where the stereo camera is positioned, wherein the ground surface data includes a two-dimensional image and a disparity image of the ground surface or includes a two-dimensional image and a depth image of the ground surface;

a first generating unit, configured for generating, according to three-dimensional coordinates of each of pixels in the ground surface data in a camera coordinate system corresponding to the stereo camera, a first plane equation in the camera coordinate system that conforms to the three-dimensional coordinates of the pixels;

a first determining unit, configured for determining, according to a distance between an origin of the camera coordinate system and the ground surface, a setting height of the stereo camera;

a second generating unit, configured for generating, according to the first plane equation, a plane normal vector corresponding to the first plane equation; and

a second determining unit, configured for determining, according to the plane normal vector, the external parameters of the stereo camera without using a vertical edge perpendicular to the ground in a scene as a reference point, wherein the external parameters comprises a pitch angle of the stereo camera and an inclination angle of the stereo camera.

11. The device according to claim 10, wherein, the device further comprises:

a filtering unit, configured for performing filtering processing on the ground surface data when the ground surface data does not meet a preset condition;

wherein the filtering processing comprises median filtering in time domain and median filtering in spatial domain.

12. The device according to claim 10, wherein, the first generating unit comprises:

a first selecting subunit, a first generating subunit, a first determining subunit, and a second generating subunit;

wherein

the first selecting subunit is configured for selecting at least three pixels from pixels in the ground surface data, and using three-dimensional coordinates of the at least three pixels in the camera coordinate system corresponding to the stereo camera as a first sample set;

the first generating sub-unit is configured for generating, according to the first sample set, a second plane equation that conforms to three-dimensional coordinates of pixels in the first sample set in the camera coordinate system;

the first determining subunit is configured for determining, among the pixels in the ground surface data, all pixels that meet the second plane equation, and using the three-dimensional coordinates of the pixels that meet the second plane equation in the camera coordinate system as a second sample set; and

the second generating sub-unit is configured for generating, according to the second sample set, the first plane equation in the camera coordinate system that conforms to the three-dimensional coordinates of the pixels in the ground surface data in the camera coordinate system.

13. The device according to claim 12, wherein, the first selecting subunit is further configured for selecting at least three pixels from the pixels in the ground surface data by using a random sample consensus RANSAC algorithm.

14. The device according to claim 12, wherein, the second generating subunit is further configured for performing fitting processing on the second sample set by using a least squares method to generate the first plane equation that conforms to the three-dimensional coordinates of the pixels in the ground surface data in the camera coordinate system.

15. The device according to claim 13, wherein, the first generating subunit comprises:

a second selecting subunit, a third generating subunit, a determining subunit, a second determining subunit, and a returning and performing subunit;

wherein,

the second selecting subunit is configured for selecting at least three pixels from the pixels in the ground surface data by using a RANSAC algorithm, and using the three-dimensional coordinates of the at least three pixels in the camera coordinate system corresponding to the stereo camera as the first sample set;

the third generating subunit is configured for generating, according to the first sample set, a current plane equation that conforms to the three-dimensional coordinates of pixels in the first sample set in the camera coordinate system;

the determining subunit is configured for determining whether the current plane equation satisfies a preset condition;

the second determining subunit is configured for using the current plane equation as the second plane equation when the current plane equation satisfies the preset condition;

the returning and performing subunit is configured for increasing the number of iterations by 1 when the current plane equation does not satisfy the preset condition and returning to perform the steps of selecting at least three pixels from pixels in the ground surface data, and using three-dimensional coordinates of the at least three pixels in the camera coordinate system corresponding to the stereo camera as a first sample set;

wherein an initial value of the number of iterations is 0.

16. The device according to claim 15, wherein, the preset condition comprises:

the number of pixels, among the pixels in the ground surface data, that meet the current plane equation reaches a preset first threshold, or the number of iterations corresponding to the current plane equation reaches a preset second threshold.

17. The device according to claim 10, wherein, the second determining unit comprises:

a third determining subunit and a fourth determining subunit;

wherein,

the third determining subunit is configured for determining, in the camera coordinate system, a first angle by which a corresponding axis z_c rotates about an axis x in a clockwise direction when the stereo camera rotates from a standard attitude to a current attitude, and using the first angle as the pitch angle of the stereo camera; and

the fourth determining sub-unit is configured for determining, in the camera coordinate system, a second angle by which a corresponding axis x_c rotates about an axis z in a clockwise direction when the stereo camera rotates from the standard attitude to the current attitude, and using the second angle as the inclination angle of the stereo camera;

wherein a three-dimensional coordinate system defined by the axis x_c, the axis z_c, and an axis y_c is the camera coordinate system;

a three-dimensional coordinate system defined by the axis x, the axis z, and an axis y is a world coordinate system;

when directions of the axis x_c and the axis z_c coincide with directions of the axis x and the axis z, and when direction of the axis y_c is opposite to direction of the axis y, the stereo camera is in the standard attitude.

18. A device for determining external parameters of a stereo camera, which comprises:

one or more controllers, one or more memories, wherein the one or more memories are configured to store executable program codes;

the executable program codes stored in the one or more memories are executed by the one or more controllers to perform;

acquiring ground surface data of a location where the stereo camera is positioned, wherein the ground surface data includes a two-dimensional image and a disparity image of the ground surface or includes a two-dimensional image and a depth image of the ground surface;

generating, according to three-dimensional coordinates of each of pixels in the ground surface data in a camera coordinate system corresponding to the stereo camera, a first plane equation in the camera coordinate system that conforms to the three-dimensional coordinates of the pixels;

determining, according to a distance between an origin of the camera coordinate system and the ground surface, a setting height of the stereo camera;

generating, according to the first plane equation, a plane normal vector corresponding to the first plane equation; and

determining, according to the plane normal vector, the external parameters of the stereo camera without using a vertical edge perpendicular to the ground in a scene as a reference point, wherein the external parameters comprises a pitch angle of the stereo camera and an inclination angle of the stereo camera.