VEHICLE VISION SYSTEM WITH CALIBRATION ALGORITHM

• US 20180197310A1
• Filed: 03/07/2018
• Published: 07/12/2018
• Est. Priority Date: 01/16/2015
• Status: Active Grant

• Current Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Original Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Inventors: Singh, Jagmal, Pliefke, Sebastian, Hess, Holger
• Granted Patent: US 10,235,775 B2
• Time in Patent Office: Days
• CPC Class Codes: G06T 2207/30208 Marker matrix
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 3/12 Panospheric to cylindrical image transformations
• CPC Class Codes: G06T 5/80 Geometric correction
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

A method for calibrating a camera of a vehicular vision system includes providing a camera and an image processor at the vehicle. A monoview noncoplanar three dimensional calibration pattern distributed on a semi-sphere is determined via processing of image data captured by the camera. Responsive to determination of the monoview noncoplanar three dimensional calibration pattern distributed on the semi-sphere, extrinsic parameters of the camera and intrinsic parameters of the camera are estimated. The system performs at least one of (i) a linear refinement of the estimated extrinsic parameters and intrinsic parameters and (ii) a non-linear refinement of the estimated extrinsic parameters and intrinsic parameters. Responsive to processing by the image processor of image data captured by the camera, the camera is calibrated at least in part by using the determined monoview noncoplanar three dimensional semi-spherical calibration pattern.

Claims

1. A method for calibrating a camera of a vehicular vision system, said method comprising:

providing a camera at a vehicle so as to have a field of view exterior of the vehicle;

wherein the camera comprises a pixelated imaging array having a plurality of photosensing elements;

providing an image processor at the vehicle, wherein the image processor is operable to process image data captured by the camera;

capturing image data via the camera;

processing image data captured by the camera;

determining, at least in part via processing of image data captured by the camera, a monoview noncoplanar three dimensional calibration pattern distributed on a semi-sphere;

responsive to determination of the monoview noncoplanar three dimensional calibration pattern distributed on the semi-sphere, estimating extrinsic parameters of the camera and estimating intrinsic parameters of the camera;

performing at least one of (i) a linear refinement of the estimated extrinsic parameters and intrinsic parameters and (ii) a non-linear refinement of the estimated extrinsic parameters and intrinsic parameters; and

responsive to processing by the image processor of image data captured by the camera, calibrating the camera at least in part by using the determined monoview noncoplanar three dimensional semi-spherical calibration pattern.

2. The method of claim 1, wherein calibrating the camera comprises using a Taylor model.

3. The method of claim 2, wherein values of distortion polynomial coefficients for the Taylor model are estimated using three dimensional calibration points distributed on the semi-sphere.

4. The method of claim 1, wherein providing a camera comprises providing a plurality of cameras at the vehicle so as to have respective fields of view exterior of the vehicle, wherein providing the plurality of cameras comprises (i) providing a forward viewing camera at the vehicle having a field of view at least forward of the vehicle, (ii) providing a rearward viewing camera at the vehicle having a field of view at least rearward of the vehicle, (iii) providing a driver side viewing camera at the vehicle having a field of view at least sideward of a driver side of the vehicle, and (iv) providing a passenger side viewing camera at the vehicle having a field of view at least sideward of a passenger side of the vehicle.

5. The method of claim 4, wherein the image processor is operable to process image data captured by the cameras of the plurality of cameras, and wherein the vision system calibrates each of the cameras of the plurality of cameras at least in part by using the monoview noncoplanar three dimensional semi-spherical calibration pattern.

6. The method of claim 1, comprising transferring image data captured by the camera to an electronic control unit via a communication link of the vehicle.

7. The method of claim 6, wherein the communication link comprises a vehicle network bus of the vehicle.

8. The method of claim 1, wherein calibrating the camera comprises detecting a position of a center of the camera using an iterative search algorithm by minimizing the sum of squared re-projection errors (SSRE).

9. The method of claim 1, wherein performing at least one of (i) a linear refinement of the estimated extrinsic parameters and intrinsic parameters and (ii) a non-linear refinement of the estimated extrinsic parameters and intrinsic parameters comprises performing a linear refinement of the estimated extrinsic parameters and intrinsic parameters.

10. The method of claim 1, wherein performing at least one of (i) a linear refinement of the estimated extrinsic parameters and intrinsic parameters and (ii) a non-linear refinement of the estimated extrinsic parameters and intrinsic parameters comprises performing a non-linear refinement of the estimated extrinsic parameters and intrinsic parameters.

11. A method for calibrating a camera of a vehicular vision system, said method comprising:

providing a camera at a vehicle so as to have a field of view exterior of the vehicle;

wherein the camera comprises a fish eye lens;

wherein the camera comprises a pixelated imaging array having a plurality of photosensing elements;

providing an image processor at the vehicle, wherein the image processor is operable to process image data captured by the camera;

capturing image data via the camera;

transferring image data captured by the camera to an electronic control unit of the vehicle, wherein the electronic control unit comprises the image processor;

processing image data captured by the camera;

determining, at least in part via processing of image data captured by the camera, a monoview noncoplanar three dimensional calibration pattern distributed on a semi-sphere;

responsive to determination of the monoview noncoplanar three dimensional calibration pattern distributed on the semi-sphere, estimating extrinsic parameters of the camera and estimating intrinsic parameters of the camera;

performing at least one of (i) a linear refinement of the estimated extrinsic parameters and intrinsic parameters and (ii) a non-linear refinement of the estimated extrinsic parameters and intrinsic parameters;

responsive to processing by the image processor of image data captured by the camera, calibrating the camera at least in part by using a Taylor model and the determined monoview noncoplanar three dimensional semi-spherical calibration pattern; and

wherein values of distortion polynomial coefficients for the Taylor model are estimated using three dimensional calibration points distributed on the semi-sphere.

12. The method of claim 11, wherein image data captured by the camera is transferred to the electronic control unit of the vehicle via a communication link of the vehicle, and wherein the communication link comprises a vehicle network bus of the vehicle.

13. The method of claim 11, wherein calibrating the camera comprises detecting a position of a center of the camera using an iterative search algorithm by minimizing the sum of squared re-projection errors (SSRE).

14. The method of claim 11, wherein performing at least one of (i) a linear refinement of the estimated extrinsic parameters and intrinsic parameters and (ii) a non-linear refinement of the estimated extrinsic parameters and intrinsic parameters comprises performing a linear refinement of the estimated extrinsic parameters and intrinsic parameters.

15. The method of claim 11, wherein performing at least one of (i) a linear refinement of the estimated extrinsic parameters and intrinsic parameters and (ii) a non-linear refinement of the estimated extrinsic parameters and intrinsic parameters comprises performing a non-linear refinement of the estimated extrinsic parameters and intrinsic parameters.

16. A method for calibrating at least some cameras of a vehicular vision system, said method comprising:

providing a plurality of cameras at a vehicle so as to have respective fields of view exterior of the vehicle;

wherein providing the plurality of cameras comprises (i) providing a forward viewing camera at the vehicle having a field of view at least forward of the vehicle, (ii) providing a rearward viewing camera at the vehicle having a field of view at least rearward of the vehicle, (iii) providing a driver side viewing camera at the vehicle having a field of view at least sideward of a driver side of the vehicle, and (iv) providing a passenger side viewing camera at the vehicle having a field of view at least sideward of a passenger side of the vehicle;

wherein each of the cameras comprises a pixelated imaging array having a plurality of photosensing elements;

providing an image processor at the vehicle, wherein the image processor is operable to process image data captured by the cameras;

capturing image data via the cameras;

processing image data captured by the cameras;

determining, via processing of image data captured by at least one of the cameras, a monoview noncoplanar three dimensional calibration pattern distributed on a semi-sphere;

responsive to determination of the monoview noncoplanar three dimensional calibration pattern distributed on the semi-sphere, estimating extrinsic parameters of at least one of the cameras and estimating intrinsic parameters of the at least one of the cameras;

performing at least one of (i) a linear refinement of the estimated extrinsic parameters and intrinsic parameters and (ii) a non-linear refinement of the estimated extrinsic parameters and intrinsic parameters; and

responsive to processing by the image processor of image data captured by the at least one of the cameras, calibrating the at least one of the cameras at least in part by using the determined monoview noncoplanar three dimensional semi-spherical calibration pattern.

17. The method of claim 16, wherein the vision system calibrates each camera of the plurality of cameras at least in part by using the monoview noncoplanar three dimensional semi-spherical calibration pattern.

18. The method of claim 16, wherein calibrating the at least one of the cameras comprises detecting a position of a center of the at least one of the cameras using an iterative search algorithm by minimizing the sum of squared re-projection errors (SSRE).

19. The method of claim 16, wherein performing at least one of (i) a linear refinement of the estimated extrinsic parameters and intrinsic parameters and (ii) a non-linear refinement of the estimated extrinsic parameters and intrinsic parameters comprises performing a linear refinement of the estimated extrinsic parameters and intrinsic parameters.

20. The method of claim 16, wherein performing at least one of (i) a linear refinement of the estimated extrinsic parameters and intrinsic parameters and (ii) a non-linear refinement of the estimated extrinsic parameters and intrinsic parameters comprises performing a non-linear refinement of the estimated extrinsic parameters and intrinsic parameters.