IN-VEHICLE CAMERA CALIBRATOR, IMAGE GENERATOR, IN-VEHICLE CAMERA CALIBRATION METHOD, AND IMAGE GENERATION METHOD

• US 20170309042A1
• Filed: 09/24/2015
• Published: 10/26/2017
• Est. Priority Date: 10/09/2014
• Status: Active Grant

• Current Assignee: DENSO Corporation
• Original Assignee: DENSO Corporation
• Inventors: Tanaka, Hitoshi, Morishita, Youji, Matsumoto, Muneaki
• Granted Patent: US 10,176,594 B2
• Time in Patent Office: Days
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: B60R 2300/402 Image calibration
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
• CPC Class Codes: H04N 17/002 for television cameras
• CPC Class Codes: H04N 23/60 Control of cameras or camera modules

Abstract

An in-vehicle camera calibrator is provided. The in-vehicle camera calibrator includes a targeted camera selector for selecting a targeted camera requiring the calibration, an intermediate camera selector for selecting an intermediate camera which is the in-vehicle camera that intervenes between the targeted camera and a criterial camera, a first calibrator for, when the intermediate camera is not selected, calibrating the targeted camera based on a captured image of an overlapping region between the criterial camera and the targeted camera, and a second calibrator for, when the intermediate camera is selected, calibrating the intermediate camera based on a captured image of an overlapping region between the criterial camera and the targeted camera, and then calibrating the targeted camera based on a captured image between the calibrated intermediate camera and the targeted camera.

Claims

1. An in-vehicle camera calibrator that

is used for a vehicle mounted with a plurality of in-vehicle cameras attached so that capture regions of the in-vehicle cameras adjacent to each other partially overlap and

calibrates at least one of an installation position and an installation angle of any of the in-vehicle cameras that has misalignment of the at least one of the installation position and the installation angle with reference to the vehicle,

the in-vehicle camera calibrator comprising:

a variation detector section that detects a variation from an initial state of at least one of an installation angle and an installation position of each of the in-vehicle cameras with reference to the vehicle;

a targeted camera selector section that selects the in-vehicle camera indicating the variation greater than a predetermined threshold value as a targeted camera that requires the calibration;

a criterial camera selector section that selects the in-vehicle camera indicating the smallest variation out of the in-vehicle cameras as a criterial camera when the targeted camera is selected from the in-vehicle cameras;

an intermediate camera selector section that selects an intermediate camera when the intermediate camera exists, wherein the intermediate camera is the in-vehicle camera that intervenes between the targeted camera and the criterial camera;

a first calibrator section that, when the intermediate camera is not selected, calibrates the targeted camera based on a captured image of an overlapping region where the capture regions of the criterial camera and the targeted camera overlap; and

a second calibrator section that, when the intermediate camera is selected, calibrates the intermediate camera based on a captured image of an overlapping region where the capture regions of the criterial camera and the targeted camera overlap, and then calibrates the targeted camera based on a captured image of an overlapping region where the capture regions of the calibrated intermediate camera and the targeted camera overlap.

2. The in-vehicle camera calibrator according to claim 1 further comprising:

a third calibrator section that calibrates the uncalibrated in-vehicle camera after the targeted camera is calibrated.

3. The in-vehicle camera calibrator according to claim 1 further comprising:

an alarm outputter section that outputs a predetermined alarm when the variation of the in-vehicle camera selected as the criterial camera is greater than a predetermined tolerance.

4. An image generator that

is used for a vehicle mounted with a plurality of in-vehicle cameras attached so that capture regions of the in-vehicle cameras adjacent to each other partially overlap and

generates a monitoring image for monitoring a situation around the vehicle based on captured images acquired by the in-vehicle cameras, the image generator comprising:

a captured image acquirer section that acquires the captured images from the in-vehicle cameras;

a variation detector section that detects a variation from an initial state of at least one of an installation angle and an installation position of each of the in-vehicle cameras with reference to the vehicle;

a targeted camera selector section that selects the in-vehicle camera indicating the variation greater than a predetermined threshold value as a targeted camera that requires calibrating the installation angle or the installation position;

a criterial camera selector section that selects the in-vehicle camera indicating the smallest variation out of the in-vehicle cameras as a criterial camera when the targeted camera is selected from the in-vehicle cameras;

an intermediate camera selector section that selects an intermediate camera when the intermediate camera exists, wherein the intermediate camera is the in-vehicle camera that intervenes between the targeted camera and the criterial camera;

a first calibrator section that, when the intermediate camera is not selected, calibrates the targeted camera based on the captured image of an overlapping region where the capture regions of the criterial camera and the targeted camera overlap;

a second calibrator section that, when the intermediate camera is selected, calibrates the intermediate camera based on the captured image of an overlapping region where the capture regions of the criterial camera and the intermediate camera overlap, and then calibrates the targeted camera based on the captured image of an overlapping region where the capture regions of the calibrated intermediate camera and the targeted camera overlap; and

an image generator section that generates the monitoring image by applying visual line conversion to the captured images acquired from the in-vehicle cameras based on a result of calibrating the in-vehicle camera.

5. An in-vehicle camera calibration method that

is used for a vehicle mounted with a plurality of in-vehicle cameras attached so that capture regions of the in-vehicle cameras adjacent to each other partially overlap and

calibrates one of an installation position and an installation angle of any of the in-vehicle cameras that has misalignment of one of an installation position and an installation angle with reference to the vehicle,

the method comprising:

detecting a variation from an initial state of one of an installation angle and an installation position of each of the in-vehicle cameras with reference to the vehicle;

selecting the in-vehicle camera indicating the variation greater than a predetermined threshold value as a targeted camera that requires the calibration;

selecting the in-vehicle camera indicating the smallest variation out of the in-vehicle cameras as a criterial camera when the targeted camera is selected from the in-vehicle cameras;

selecting an intermediate camera when the intermediate camera exists, wherein the intermediate camera is the in-vehicle camera that intervenes between the targeted camera and the criterial camera;

when the intermediate camera is not selected, calibrating the targeted camera based on a captured image of an overlapping region where the capture regions of the criterial camera and the targeted camera overlap; and

when the intermediate camera is selected, calibrating the intermediate camera based on a captured image of an overlapping region where the capture regions of the criterial camera and the intermediate camera overlap, and then calibrating the targeted camera based on a captured image of an overlapping region where the capture regions of the calibrated intermediate camera and the targeted camera overlap.

6. An image generation method that

is used for a vehicle mounted with a plurality of in-vehicle cameras attached so that capture regions the in-vehicle cameras adjacent to each other partially overlap and

generates a monitoring image for monitoring a situation around the vehicle based on captured images acquired by the in-vehicle cameras,

the method comprising:

acquiring the captured images from the in-vehicle cameras;

detecting a variation from an initial state of at least one of an installation angle and an installation position of each of the in-vehicle cameras with reference to the vehicle;

selecting the in-vehicle camera indicating the variation greater than a predetermined threshold value as a targeted camera that requires calibrating the installation angle or the installation position;

selecting the in-vehicle camera indicating the smallest variation out of the in-vehicle cameras as a criterial camera when the targeted camera is selected from the in-vehicle cameras;

selecting an intermediate camera when the intermediate camera exists, wherein the intermediate camera is the in-vehicle camera that intervenes between the targeted camera and the criterial camera;

when the intermediate camera is not selected, calibrating the targeted camera based on the captured image of an overlapping region where the capture regions of the criterial camera and the targeted camera overlap;

when the intermediate camera is selected, calibrating the intermediate camera based on the captured image of an overlapping region where the capture regions of the criterial camera and the intermediate camera overlap, and then calibrating the targeted camera based on the captured image of an overlapping region where the capture regions of the calibrated intermediate camera and the targeted camera overlap; and

generating the monitoring image by applying visual line conversion to the captured image acquired from the in-vehicle cameras based on the result of calibrating the in-vehicle camera.