System for Calibrating a Vision System

• US 20120320190A1
• Filed: 05/30/2012
• Published: 12/20/2012
• Est. Priority Date: 06/01/2011
• Status: Active Grant

• Current Assignee: Harman Becker Automotive Systems GmbH (Samsung Electronics Co. Ltd.)
• Original Assignee: Harman Becker Automotive Systems GmbH (Samsung Electronics Co. Ltd.)
• Inventors: Natroshvili, Koba, Scholl, Kay-Ulrich, Quast, Johannes
• Granted Patent: US 9,311,706 B2
• Time in Patent Office: Days
• US Class Current: 348/135
• CPC Class Codes: G06T 2207/30208 Marker matrix
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 5/80 Geometric correction
• CPC Class Codes: G06T 7/85 Stereo camera calibration

Abstract

A system for calibrating a vision system of an object. For example, a system for calibrating a surround view system of a vehicle. The system may be configured to capture, via a first and second image sensor, a first and second image of a plurality of objects. Upon capturing such information, the system may be configured to determine a position and orientation of the first and the second image sensor relative to the plurality of objects based on the first and the second captured image. Further, the system may be configured to determine a relative position and orientation between the first and the second image sensor based on a result of the determining a position and orientation of the first and the second image sensor. Also, the system may be configured to determine intrinsic and extrinsic parameters of the first and the second image sensor.

Claims

1. Logic for calibrating a vision system of an object, configured to:

capture, via a first image sensor of the vision system, a first image of a plurality of objects;

capture, via a second image sensor of the vision system, a second image of the plurality of objects, where the plurality of objects is located in an overlapping region of a field of view of the first image sensor and a field of view of the second image sensor;

determine a position and orientation of the first and the second image sensor relative to the plurality of objects based on the first and the second captured image, respectively;

determine a relative position and orientation between the first and the second image sensor based on a result of the determining a position and orientation of the first and the second image sensor;

determine extrinsic parameters of the first and the second image sensor in a reference frame based on results of the determining a relative position and orientation between the first and the second image sensor; and

determine intrinsic parameters of first and the second image sensor based on the first and the second captured image, respectively.

2. The logic of claim 1, wherein the logic is further configured to:

repeat the capturing and determinations for a plurality of pairs of image sensors having overlapping fields of view.

3. The logic of claim 2, wherein a first pair of the plurality of pairs of image sensors includes a first pair image sensor and an intermediate image sensor, wherein a second pair of the plurality of pairs of image sensors includes the intermediate image sensor and a second pair image sensor, and wherein the logic is further configured to:

determine a relative position and orientation between the first pair image sensor and the second pair image sensor based on a result of the determining a relative position and orientation between the first and the second image sensor for the first pair and the second pair, respectively.

4. The logic of claim 3, wherein the logic is further configured to:

compare the relative position and orientation between the first pair image sensor and the second pair image sensor to the relative position and orientation between an other first pair image sensor and an other second pair image sensor of an other plurality of pairs of image sensors.

5. The logic of claim 3, wherein a field of view of the first pair image sensor and a field of view of second pair image sensor are disjoint.

6. The logic of claim 3,

wherein the first pair image sensor and the second pair image sensor are provided at opposite sides of the object.

7. The logic of claim 1, wherein the object is a vehicle and the vision system is a vehicle vision system.

8. The logic of claim 1, wherein the determining a position and orientation of an image sensor comprises:

determining a vector {right arrow over (M)}, wherein ∥

L{right arrow over (M)}∥

becomes a minimum under a constraint that the vector {right arrow over (M)} comprises a fixed modulus, wherein L is a matrix having a number of columns that is equal to a number of rows of the vector {right arrow over (M)}, wherein ∥

∥

denotes a vector modulus function, and wherein the vector {right arrow over (M)} comprises matrix elements that are elements of a camera matrix of the first image sensor.

9. The logic of claim 8, wherein the vector {right arrow over (M)} is defined as

10. The logic of claim 8, wherein the vector {right arrow over (M)} is independent of intrinsic parameters of the image sensor.

11. The logic of claim 8, wherein the matrix L is independent of intrinsic parameters of the image sensor.

12. The logic of claim 1, wherein determining the extrinsic parameters in the reference frame comprises determining projections of the plurality of objects.

13. The logic of claim 7, wherein the vehicle vision system is a surround view system and the logic is further configured to:

render a surround view based on the captured images.

14. A method for calibrating a vision system of an object, comprising:

capturing, via a first image sensor of the vision system, a first image of a plurality of objects;

capturing, via a second image sensor of the vision system, a second image of the plurality of objects, where the plurality of objects is located in an overlapping region of a field of view of the first image sensor and a field of view of the second image sensor;

determining a position and orientation of the first and the second image sensor relative to the plurality of objects based on the first and the second captured image, respectively;

determining a relative position and orientation between the first and the second image sensor based on a result of the determining a position and orientation of the first and the second image sensor; and

determining a relative position and orientation between a first pair image sensor and a second pair image sensor based on a result of the determining a relative position and orientation between the first and the second image sensor for the first pair and the second pair, respectively, wherein the first pair of the plurality of pairs of image sensors includes the first pair image sensor and an intermediate image sensor, and wherein the second pair of the plurality of pairs of image sensors includes the intermediate image sensor and the second pair image sensor.

15. The method of claim 14, wherein the plurality of objects includes at least three objects located on a first plane and at least three objects located on a second plane.

16. The method of claim 15, wherein the plurality of objects includes at least three objects located on a third plane, and wherein the first plane, the second plane, and the third plane are orthogonal.

17. A vehicle vision system of a vehicle, comprising:

a plurality of image sensors each comprising an electro-optical component to capture images comprising a non-linear distortion, wherein the image sensors are located at different locations on a vehicle hosting the vehicle vision system;

an image processing unit configured to process image data captured by the plurality of image sensors; and

calibration logic, wherein the calibration logic is coupled to the image processing unit, and wherein the calibration logic is configured to;

identify, via a first image sensor of the vehicle vision system, a first image of a plurality of objects;

identify, via a second image sensor of the vehicle vision system, a second image of the plurality of objects, where the plurality of objects is located in an overlapping region of a field of view of the first image sensor and a field of view of the second image sensor;

determine a position and orientation of the first and the second image sensor relative to the plurality of objects based on the first and the second captured image, respectively;

determine a relative position and orientation between the first and the second image sensor based on a result of the determining a position and orientation of the first and the second image sensor;

determine extrinsic parameters of the first and the second image sensor in a reference frame based on results of the determining a relative position and orientation between the first and the second image sensor; and

determine intrinsic parameters of first and the second image sensor based on the first and the second captured image, respectively.

18. The system of claim 17, wherein the system is a surround view system.

19. The system of claim 17, wherein the plurality of image sensors comprise fisheye lenses.

20. The system of claim 17, wherein at least two of the plurality of image sensors are located at two respective side door mirrors of the vehicle hosting the system.