Targetless vehicle camera calibration system

• US 10,453,217 B2
• Filed: 03/23/2017
• Issued: 10/22/2019
• Est. Priority Date: 03/24/2016
• Status: Active Grant

• Current Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Original Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Inventors: Singh, Jagmal, Pliefke, Sebastian, Hess, Holger
• Primary Examiner(s): Hilaire, Clifford
• Application Number:
  US15/467,245
• Publication Number:
  US 20170278270A1
• Time in Patent Office: 943 Days
• Field of Search: None
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: B60R 2300/402 Image calibration
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
• CPC Class Codes: H04N 17/002 for television cameras

Abstract

A vehicular camera calibration system includes a camera disposed at a vehicle and having a field of view exterior of the vehicle and being operable to capture image data. Responsive to processing of image data captured by the camera, the camera calibration system determines features at the ground adjacent to the vehicle and tracks the determined features over two or more frames of captured image data. Responsive to processing of frames of captured image data during movement of the vehicle along a straight path on a planar surface, the camera calibration system is operable to determine a misalignment of the camera. The camera calibration system calibrates the misaligned camera by using an assumption that the determined features lie in a plane parallel to the planar surface along which the vehicle is traveling. The camera calibration system calibrates the camera by reducing ambiguity in vectors normal to the planar surface.

Claims

1. A camera calibration system for a vehicle, said camera calibration system comprising:

a camera disposed at a vehicle and having a field of view exterior of the vehicle;

wherein said camera is operable to capture image data;

an image processor that processes frames of image data captured by said camera as the vehicle moves along a vehicle assembly line;

wherein, responsive to processing by said image processor of frames of image data captured by said camera as the vehicle moves along the vehicle assembly line, said camera calibration system (i) determines features at the ground at the vehicle assembly line and adjacent to the vehicle as the vehicle moves along the vehicle assembly line and (ii) tracks the determined features over two or more frames of image data captured by said camera as the vehicle moves along the vehicle assembly line;

wherein, responsive to processing by said image processor of frames of image data captured by said camera as the vehicle moves along a straight path at a planar ground surface at the vehicle assembly line, said camera calibration system determines misalignment of said camera;

wherein said camera calibration system, responsive to determination of misalignment of said camera, calibrates said camera by using an assumption that the determined features lie in a plane parallel to the planar ground surface along the straight path at the vehicle assembly line;

wherein said camera calibration system determines misalignment of said camera and calibrates said camera without use of a target pattern; and

wherein said camera calibration system calibrates said camera by reducing ambiguity in vectors normal to the planar ground surface.

2. The camera calibration system of claim 1, wherein said camera calibration system calibrates said camera by using an assumption that the determined features lie at the planar ground surface.

3. The camera calibration system of claim 2, wherein said camera calibration system, responsive to processing by said image processor of frames of image data captured by said camera as the vehicle moves along the straight path at the ground plane at the vehicle assembly line, extracts features that lie on the planar ground surface without use of a target pattern at the planar ground surface at the vehicle assembly line.

4. The camera calibration system of claim 3, wherein said camera calibration system determines vectors normal to the planar ground surface and determines misalignment of said camera at the vehicle responsive to the determined vectors over two or more frames of image data captured by said camera as the vehicle moves along the vehicle assembly line.

5. The camera calibration system of claim 1, wherein said camera calibration system calibrates said camera as the vehicle is driven along an end-of-line test of the vehicle assembly line.

6. The camera calibration system of claim 5, wherein said camera calibration system calibrates said camera without use of a target pattern at the end-of-line test of the vehicle assembly line.

7. The camera calibration system of claim 1, comprising a plurality of cameras disposed at the vehicle and having respective fields of view exterior of the vehicle, wherein said image processor processes image data captured by said cameras as the vehicle moves along the vehicle assembly line, and wherein said camera calibration system calibrates said cameras without use of target patterns in the field of view of said cameras.

8. The camera calibration system of claim 1, wherein said camera calibration system comprises a plurality of cameras disposed at the vehicle, each having a respective field of view exterior of the vehicle.

9. The camera calibration system of claim 8, wherein image data captured by at least some of said plurality of cameras is used for a surround view system of the vehicle.

10. The camera calibration system of claim 8, wherein one camera of said plurality of cameras comprises a rear camera disposed at a rear portion of the vehicle, and another camera of said plurality of cameras comprises a side camera disposed at a driver-side exterior rearview mirror assembly of the vehicle and another camera of said plurality of cameras comprises a passenger side camera disposed at a passenger-side exterior rearview mirror assembly of the vehicle.

11. The camera calibration system of claim 1, wherein said camera calibration system determines features below a threshold level in the camera'"'"'s field of view and ignores features that are above the threshold level in the camera'"'"'s field of view.

12. A camera calibration system for a vehicle, said camera calibration system comprising:

a plurality of cameras disposed at a vehicle, each having respective field of view exterior of the vehicle;

wherein said cameras are operable to capture image data;

an image processor that processes frames of image data captured by said cameras as the vehicle moves along a vehicle assembly line;

wherein, responsive to processing by said image processor of frames of image data captured by said cameras as the vehicle moves along the vehicle assembly line, said camera calibration system (i) determines features at the ground at the vehicle assembly line and adjacent to the vehicle as the vehicle moves along the vehicle assembly line and (ii) tracks the determined features over two or more frames of image data captured by said cameras as the vehicle moves along the vehicle assembly line;

wherein, responsive to processing by said image processor of frames of image data captured by said cameras as the vehicle moves along a straight path at a ground plane at the vehicle assembly line, said camera calibration system determines misalignment of at least one of said cameras;

wherein said camera calibration system, responsive to determination of misalignment of at least one of said cameras, calibrates the determined misaligned camera by using an assumption that the determined features lie at the ground plane at the vehicle assembly line;

wherein said camera calibration system calibrates the determined misaligned camera by reducing ambiguity in vectors normal to the ground plane; and

wherein said camera calibration system calibrates the determined misaligned camera without use of a target pattern.

13. The camera calibration system of claim 12, wherein said camera calibration system, responsive to processing by said image processor of frames of captured image data as the vehicle moves along the straight path at the ground plane at the vehicle assembly line, extracts features that lie on the ground plane.

14. The camera calibration system of claim 13, wherein said camera calibration system determines vectors normal to the ground plane and determines misalignment of at least one of said cameras at the vehicle responsive to the determined vectors over two or more frames of image data captured by said at least one of said cameras as the vehicle moves along the vehicle assembly line.

15. The camera calibration system of claim 12, wherein said camera calibration system calibrates said cameras as the vehicle is driven along an end-of-line test of the vehicle assembly line.

16. The camera calibration system of claim 12, wherein one camera of said plurality of cameras comprises a rear camera disposed at a rear portion of the vehicle, and another camera of said plurality of cameras comprises a side camera disposed at a driver-side exterior rearview mirror assembly of the vehicle and another camera of said plurality of cameras comprises a passenger side camera disposed at a passenger-side exterior rearview mirror assembly of the vehicle.

17. The camera calibration system of claim 12, wherein said camera calibration system determines features below a threshold level in the cameras'"'"' fields of view and ignores features that are above the threshold level in the cameras'"'"' fields of view.

18. A camera calibration system for a vehicle, said camera calibration system comprising:

a plurality of cameras disposed at a vehicle, each having respective field of view exterior of the vehicle;

wherein said cameras are operable to capture image data;

wherein one camera of said plurality of cameras comprises a rear camera disposed at a rear portion of the vehicle, and another camera of said plurality of cameras comprises a side camera disposed at a driver-side exterior rearview mirror assembly of the vehicle and another camera of said plurality of cameras comprises a passenger side camera disposed at a passenger-side exterior rearview mirror assembly of the vehicle;

wherein image data captured by at least some of said plurality of cameras is used for a surround view system of the vehicle;

an image processor that processes frames of image data captured by said cameras as the vehicle moves along a vehicle assembly line;

wherein, responsive to processing by said image processor of frames of image data captured by said cameras as the vehicle moves along the vehicle assembly line, said camera calibration system (i) determines features at the ground at the vehicle assembly line and adjacent to the vehicle as the vehicle moves along the vehicle assembly line and (ii) tracks the determined features over two or more frames of image data captured by said cameras as the vehicle moves along the vehicle assembly line;

wherein said camera calibration system determines and tracks features below a threshold level in the cameras'"'"' fields of view and ignores features that are above the threshold level in the cameras'"'"' fields of view;

wherein, responsive to processing by said image processor of frames of image data captured by said cameras as the vehicle moves along a straight path at a planar ground surface at the vehicle assembly line, said camera calibration system determines misalignment of at least one of said cameras;

wherein said camera calibration system, responsive to determination of a misaligned camera, calibrates the determined misaligned camera by using an assumption that the determined features lie in a plane parallel to the planar ground surface along the straight path at the vehicle assembly line;

wherein said camera calibration system determines misalignment of said camera and calibrates said camera without use of a target pattern;

wherein said camera calibration system determines vectors normal to the plane and determines misalignment of the misaligned camera responsive to the determined vectors over two or more frames of captured image data; and

wherein said camera calibration system calibrates the determined misaligned camera by reducing ambiguity in vectors normal to the planar ground surface.

19. The camera calibration system of claim 18, wherein said camera calibration system calibrates the determined misaligned camera as the vehicle is driven along an end-of-line test of the vehicle assembly line.

20. The camera calibration system of claim 19, wherein said camera calibration system calibrates the determined misaligned camera without use of a target pattern at the end-of-line test of the vehicle assembly line.