Method for calibrating a vehicular camera system

• US 8,421,865 B2
• Filed: 05/01/2012
• Issued: 04/16/2013
• Est. Priority Date: 10/24/2008
• Status: Expired due to Fees

• Current Assignee: Magna Electronics Europe Gmbh & Co. Kg
• Original Assignee: Magna Electronics Europe Gmbh & Co. Kg
• Inventors: Euler, Christian, Adameck, Markus
• Primary Examiner(s): Rao, Andy
• Application Number:
  US13/460,896
• Publication Number:
  US 20120224064A1
• Time in Patent Office: 350 Days
• Field of Search: 348 36- 39, 348113-119, 348/148
• US Class Current: 348/148
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: B60R 1/27 providing all-round vision, e.g. using omnidirectional cameras
• CPC Class Codes: B60R 2300/102 using 360 degree surveillance camera system
• CPC Class Codes: B60R 2300/105 using multiple cameras
• CPC Class Codes: B60R 2300/303 using joined images, e.g. multiple camera images
• CPC Class Codes: B60R 2300/402 Image calibration
• CPC Class Codes: B60R 2300/607 from a bird's eye viewpoint
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums

Abstract

A method of calibrating a vehicular multi-camera system includes equipping a vehicle with a plurality of cameras wherein each camera of the plurality of cameras captures image data, equipping the vehicle with an image processor, inputting image data from each of the plurality of cameras to the image processor, the image processor processing input image data in order to calibrate the vehicular multi-camera system, and wherein calibration of the vehicular multi-camera system is achieved independently of a model of the real world.

Claims

1. A method of calibrating a vehicular multi-camera system, said method comprising:

equipping a vehicle with a plurality of cameras wherein each camera of said plurality of cameras captures image data;

wherein said plurality of cameras comprises a first camera mounted at a forward portion of the vehicle and having a field of view that includes a ground surface forward of the vehicle, a second camera mounted at a rearward portion of the vehicle and having a field of view that includes a ground surface rearward of the vehicle, a third camera mounted at a sideward portion of the vehicle and having a field of view that includes a ground surface sideward of the vehicle at one side of the vehicle, and a fourth camera mounted at another sideward portion of the vehicle and having a field of view that includes a ground surface sideward of the vehicle at the other side of the vehicle;

equipping said vehicle with an image processor;

inputting image data from each of said plurality of cameras to said image processor;

said image processor processing input image data in order to calibrate said vehicular multi-camera system;

wherein calibration of said vehicular multi-camera system is achieved independently of a model of the real world;

producing a chronological sequence of image data by recurrently applying a selection criterion to select from a sequence of subimage data records recorded by said first, second, third and fourth cameras;

determining parameters of transformations of said subimage data records via an optimization method that uses a quality criterion for an overall image; and

assembling a sequence of transformed subimage data records to form a sequence of overall image data.

2. The method of claim 1, wherein equipping a vehicle with a plurality of cameras comprises equipping a vehicle with a plurality of CMOS cameras.

3. The method of claim 1, wherein calibration of said vehicular multi-camera system is achieved independently of the shape of the outer vehicle skin.

4. The method of claim 1, comprising an initial calibration before producing a chronological sequence of image data.

5. The method of claim 1, wherein said quality criterion comprises a scalar value.

6. The method of claim 5, wherein said scalar value is dependent on said subimage data.

7. The method of claim 6, wherein said scalar value is dependent on stored information relating to at least one of (i) a position of at least one of said first, second, third and fourth cameras and (ii) a direction of said field of view of at least one of said first, second, third and fourth cameras.

8. The method of claim 6, wherein said scalar value is dependent on stored information relating to (i) a position of at least one of said first, second, third and fourth cameras and (ii) a direction of said field of view of at least one of said first, second, third and fourth cameras.

9. The method of claim 6, wherein said scalar value is dependent on stored information relating to at least one of (i) respective positions of said first, second, third and fourth cameras and (ii) respective directions of said fields of view of said first, second, third and fourth cameras.

10. The method of claim 1, wherein said quality criterion reflects a quality of said overall image data.

11. The method of claim 1, wherein said calibration is recurrently performed over time in response to said selection criterion selecting a particular subimage data record.

12. The method of claim 11, wherein said selection criterion selects a particular subimage data record when a particular state of the vehicle at the time at which said particular subimage data record was recorded is within a prescribed limit.

13. A method of calibrating a vehicular multi-camera system, said method comprising:

equipping a vehicle with a plurality of CMOS cameras wherein each camera of said plurality of CMOS cameras captures image data;

wherein said plurality of CMOS cameras comprises a first camera mounted at a forward portion of the vehicle and having a field of view that includes a ground surface forward of the vehicle, a second camera mounted at a rearward portion of the vehicle and having a field of view that includes a ground surface rearward of the vehicle, a third camera mounted at a sideward portion of the vehicle and having a field of view that includes a ground surface sideward of the vehicle at one side of the vehicle, and a fourth camera mounted at another sideward portion of the vehicle and having a field of view that includes a ground surface sideward of the vehicle at the other side of the vehicle;

equipping said vehicle with an image processor;

inputting image data from each of said plurality of cameras to said image processor;

said image processor processing input image data in order to calibrate said vehicular multi-camera system;

wherein calibration of said vehicular multi-camera system is achieved independently of a model of the real world, and wherein calibration of said vehicular multi-camera system is achieved independently of the shape of the outer vehicle skin;

producing a chronological sequence of image data by recurrently applying a selection criterion to select from a sequence of subimage data records recorded by said first, second, third and fourth cameras;

determining parameters of transformations of said subimage data records via an optimization method that uses a quality criterion for an overall image; and

assembling a sequence of transformed subimage data records to form a sequence of overall image data.

14. The method of claim 13, comprising an initial calibration before producing a chronological sequence of image data.

15. The method of claim 13, wherein said quality criterion comprises a scalar value, and wherein said scalar value is dependent on stored information relating to at least one of (i) a position of at least one of said first, second, third and fourth cameras and (ii) a direction of said field of view of at least one of said first, second, third and fourth cameras.

16. The method of claim 13, wherein said calibration is recurrently performed over time in response to said selection criterion selecting a particular subimage data record, and wherein said selection criterion selects a particular subimage data record when a particular state of the vehicle at the time at which said particular subimage data record was recorded is within a prescribed limit.

17. A method of calibrating a vehicular multi-camera system, said method comprising:

equipping a vehicle with a plurality of CMOS cameras wherein each camera of said plurality of CMOS cameras captures image data;

wherein said plurality of CMOS cameras comprises a first camera mounted at a forward portion of the vehicle and having a field of view that includes a ground surface forward of the vehicle, a second camera mounted at a rearward portion of the vehicle and having a field of view that includes a ground surface rearward of the vehicle, a third camera mounted at a sideward portion of the vehicle and having a field of view that includes a ground surface sideward of the vehicle at one side of the vehicle, and a fourth camera mounted at another sideward portion of the vehicle and having a field of view that includes a ground surface sideward of the vehicle at the other side of the vehicle;

equipping said vehicle with an image processor;

inputting image data from each of said plurality of cameras to said image processor;

said image processor processing input image data in order to calibrate said vehicular multi-camera system;

wherein calibration of said vehicular multi-camera system is achieved independently of a model of the real world;

producing a chronological sequence of image data by recurrently applying a selection criterion to select from a sequence of subimage data records recorded by said first, second, third and fourth cameras;

determining parameters of transformations of said subimage data records via an optimization method that uses a quality criterion for an overall image;

assembling a sequence of transformed subimage data records to form a sequence of overall image data; and

initially calibrating said vehicular multi-camera system before producing a chronological sequence of image data.

18. The method of claim 17, comprising an initial calibration before producing a chronological sequence of image data.

19. The method of claim 17, wherein said quality criterion comprises a scalar value, and wherein said scalar value is dependent on stored information relating to at least one of (i) a position of at least one of said first, second, third and fourth cameras and (ii) a direction of said field of view of at least one of said first, second, third and fourth cameras.

20. The method of claim 17, wherein said calibration is recurrently performed over time in response to said selection criterion selecting a particular subimage data record, and wherein said selection criterion selects a particular subimage data record when a particular state of the vehicle at the time at which said particular subimage data record was recorded is within a prescribed limit.