Vehicle vision system with calibration algorithm

• US 9,916,660 B2
• Filed: 01/15/2016
• Issued: 03/13/2018
• Est. Priority Date: 01/16/2015
• Status: Active Grant

• Current Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Original Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Inventors: Singh, Jagmal, Pliefke, Sebastian, Hess, Holger
• Primary Examiner(s): Adams, Eileen
• Application Number:
  US14/996,571
• Publication Number:
  US 20160210750A1
• Time in Patent Office: 788 Days
• Field of Search: None
• CPC Class Codes: G06T 2207/30208 Marker matrix
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 3/12 Panospheric to cylindrical image transformations
• CPC Class Codes: G06T 5/80 Geometric correction
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

A vision system of a vehicle includes a camera configured to be disposed at a vehicle so as to have a field of view exterior of the vehicle. The camera may include a fisheye lens. An image processor is operable to process image data captured by the camera. The vision system provides enhanced camera calibration using a monoview noncoplanar three dimensional calibration pattern. The system may include a plurality of cameras configured to be disposed at the vehicle so as to have respective fields of view exterior of the vehicle.

Claims

1. A vision system of a vehicle, said vision system comprising:

a camera configured to be disposed at a vehicle so as to have a field of view exterior of the vehicle;

wherein said camera comprises a pixelated imaging array having a plurality of photosensing elements;

an image processor operable to process image data captured by said camera;

wherein said camera calibration comprises an estimation of extrinsic parameters and an estimation of intrinsic parameters responsive to determination of the monoview noncoplanar three dimensional calibration pattern distributed on a semi-sphere;

wherein said vision system provides camera calibration using a monoview noncoplanar three dimensional semi-spherical calibration pattern; and

wherein values of distortion polynomial coefficients are estimated using three dimensional calibration points distributed on a semi-sphere by formulating the following;

2. The vision system of claim 1, wherein said camera comprises a fish eye lens.

3. The vision system of claim 1, wherein the values of distortion polynomial coefficients are estimated by reducing the formation to obtain:

v′

_j(r₃₁X_j+r₃₂Y_j+r₃₃Z_j+t₃)−

(a₀+a₂ρ

_j²+ . . . +a_Nρ

_j^N)(r₂₁X_j+r₂₂Y_j+r₂₃Z_j+t₂)=0

(a₀+a₂ρ

_j²+ . . . +a_Nρ

_j^N)(r₁₁X_j+r₁₂Y_j+r₁₃Z_j+t₁)−

u′

_j(r₃₁X_j+r₃₂Y_j+r₃₃Z_j+t₃)=0

u′

_j(r₂₁X_j+r₂₂Y_j+r₂₃Z_j+t₂)−

v′

_j(r₁₁X_j+r₁₂Y_j+r₁₃Z_j+t₁)=0

wherein r_ij and t_i are unknown parameters.

4. The vision system of claim 3, wherein the values of distortion polynomial coefficients are estimated by solving the equations to obtain a formulation for distortion polynomial coefficients:

5. The vision system of claim 4, wherein, by providing knowledge about t₃ will give an unambiguous solution for distortion polynomial, by setting t₃ to zero and reducing to following final equation:

6. A vision system of a vehicle, said vision system comprising:

a camera configured to be disposed at a vehicle so as to have a field of view exterior of the vehicle;

wherein said camera comprises a pixelated imaging array having a plurality of photosensing elements;

an image processor operable to process image data captured by said camera;

wherein said vision system provides camera calibration using a monoview noncoplanar three dimensional semi-spherical calibration pattern; and

wherein an equal distribution of calibration points having equal angular distance and constant distance to the camera is a preferred point for a value r₃ in;

7. The vision system of claim 6, comprising a plurality of cameras configured to be disposed at a vehicle so as to have respective fields of view exterior of the vehicle.

8. The vision system of claim 7, wherein said plurality of cameras comprises (i) a forward viewing camera disposed at the vehicle so as to have a forward field of view forward of the vehicle, (ii) a rearward viewing camera disposed at the vehicle so as to have a rearward field of view rearward of the vehicle, (iii) a driver side viewing camera disposed at the vehicle so as to have a sideward field of view sideward of a driver side of the vehicle, and (iv) a passenger side viewing camera disposed at the vehicle so as to have a sideward field of view sideward of a passenger side of the vehicle.

9. A vision system of a vehicle, said vision system comprising:

a camera configured to be disposed at a vehicle so as to have a field of view exterior of the vehicle, wherein said camera comprises a fish eye lens;

wherein said camera comprises a pixelated imaging array having a plurality of photosensing elements;

an image processor operable to process image data captured by said camera;

wherein said vision system provides camera calibration using a monoview noncoplanar three dimensional semi-spherical calibration pattern;

wherein said camera calibration comprises an estimation of extrinsic parameters and an estimation of intrinsic parameters responsive to determination of the monoview noncoplanar three dimensional calibration pattern distributed on a semi-sphere; and

wherein values of distortion polynomial coefficients are estimated using three dimensional calibration points distributed on a semi-sphere by formulating the following;

10. The vision system of claim 9, wherein the values of distortion polynomial coefficients are estimated by reducing the formation to obtain:

v′

_j(r₃₁X_j+r₃₂Y_j+r₃₃Z_j+t₃)−

(a₀+a₂ρ

_j²+ . . . +a_Nρ

_j^N)(r₂₁X_j+r₂₂Y_j+r₂₃Z_j+t₂)=0

(a₀+a₂ρ

_j²+ . . . +a_Nρ

_j^N)(r₁₁X_j+r₁₂Y_j+r₁₃Z_j+t₁)−

u′

_j(r₃₁X_j+r₃₂Y_j+r₃₃Z_j+t₃)=0

u′

_j(r₂₁X_j+r₂₂Y_j+r₂₃Z_j+t₂)−

v′

_j(r₁₁X_j+r₁₂Y_j+r₁₃Z_j+t₁)=0

wherein A_j is r₂₁X_j+r₂₂Y_j+r₂₃Z_j+t₂ of the equation of claim 9, and Cj is r₁₁X_j+r₁₂Y_j+r₁₃Z_j+t₁ of the equation of claim 9.

11. The vision system of claim 10, wherein the values of distortion polynomial coefficients are estimated by and solving the equations to obtain a formulation for distortion polynomial coefficients:

12. The vision system of claim 11, wherein, by providing knowledge about t₃ will give an unambiguous solution for distortion polynomial, by setting t₃ to zero and reducing to following final equation:

13. A vision system of a vehicle, said vision system comprising:

a camera configured to be disposed at a vehicle so as to have a field of view exterior of the vehicle, wherein said camera comprises a fish eye lens;

wherein said camera comprises a pixelated imaging array having a plurality of photosensing elements;

an image processor operable to process image data captured by said camera;

wherein said vision system provides camera calibration using a monoview noncoplanar three dimensional semi-spherical calibration pattern;

wherein said camera calibration comprises an estimation of extrinsic parameters and an estimation of intrinsic parameters responsive to determination of the monoview noncoplanar three dimensional calibration pattern distributed on a semi-sphere;

wherein an equal distribution of calibration points having equal angular distance and constant distance to the camera is a preferred point for a value r₃ in;

14. A vision system of a vehicle, said vision system comprising:

a plurality of cameras configured to be disposed at a vehicle so as to have respective fields of view exterior of the vehicle, wherein at least one camera of said plurality of cameras comprises a fish eye lens;

wherein each of said cameras comprises a pixelated imaging array having a plurality of photosensing elements;

an image processor operable to process image data captured by said at least one camera;

wherein said vision system provides camera calibration of said at least one camera using a monoview noncoplanar three dimensional semi-spherical calibration pattern; and

wherein an equal distribution of calibration points having equal angular distance and constant distance to the camera is a preferred point for a value r₃ in;

15. The vision system of claim 14, wherein said camera calibration comprises an estimation of extrinsic parameters and an estimation of intrinsic parameters responsive to determination of the monoview noncoplanar three dimensional calibration pattern distributed on a semi-sphere.

16. The vision system of claim 14, wherein said plurality of cameras comprises (i) a forward viewing camera disposed at the vehicle so as to have a forward field of view forward of the vehicle, (ii) a rearward viewing camera disposed at the vehicle so as to have a rearward field of view rearward of the vehicle, (iii) a driver side viewing camera disposed at the vehicle so as to have a sideward field of view sideward of a driver side of the vehicle, and (iv) a passenger side viewing camera disposed at the vehicle so as to have a sideward field of view sideward of a passenger side of the vehicle, and wherein said image processor is operable to process image data captured by said cameras, and wherein said vision system provides camera calibration of said cameras using said monoview noncoplanar three dimensional semi-spherical calibration pattern.