Calculation device, camera device, vehicle, and calibration method

• US 10,373,338 B2
• Filed: 05/26/2016
• Issued: 08/06/2019
• Est. Priority Date: 05/27/2015
• Status: Active Grant

• Current Assignee: Kyocera Corporation
• Original Assignee: Kyocera Corporation
• Inventors: Inoue, Shushin, Koishi, Tomofumi
• Primary Examiner(s): Tsai, Tsung Yin
• Application Number:
  US15/576,781
• Publication Number:
  US 20180165833A1
• Time in Patent Office: 1,167 Days
• CPC Class Codes: G01C 21/28 with correlation of data from several navigational instruments
• CPC Class Codes: G01C 25/00 Manufacturing, calibrating, cleaning, or repairing instruments or devices referred to in the other groups of this subclass testing, calibrating or compensating compasses G01C17/38
• CPC Class Codes: G01C 3/14 with binocular observation at a single point, e.g. stereoscopic type G01C3/20 takes precedence
• CPC Class Codes: G06T 2207/30256 Lane; Road marking
• CPC Class Codes: G06T 3/00 Geometric image transformations in the plane of the image
• CPC Class Codes: G06T 7/85 Stereo camera calibration
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
• CPC Class Codes: G08G 1/165 for passive traffic, e.g. including static obstacles, trees
• CPC Class Codes: G08G 1/166 for active traffic, e.g. moving vehicles, pedestrians, bikes
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/246 Calibration of cameras
• CPC Class Codes: H04N 17/002 for television cameras
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals
• CPC Class Codes: H04N 23/60 Control of cameras or camera modules
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums

Abstract

A calculation device comprises a controller configured to extract, from a captured image of a subject, a plurality of lines corresponding to straight lines parallel to each other in the subject, and to calculate first calibration data so that mapping lines obtained when the extracted plurality of lines are mapped to a three-dimensional coordinate space are parallel to each other.

Claims

1. A calculation device comprising:

a controller configured to extract, from a captured image of a subject, a plurality of lines corresponding to straight lines parallel to each other located on a plane in the subject, to map the extracted plurality of lines to a three-dimensional coordinate space, and to calculate first calibration data so that mapping lines obtained are parallel to each other,

wherein the controller is configured to calibrate an angle θ

_C between an optical axis of a camera that captures the image and the plane, based on the first calibration data, and

wherein the mapping of the plurality of lines to the three-dimensional coordinate space is conducted according to the following conversion expressions, when coordinate on the captured image expressed as a pixel value is (u, v), and coordinate on XZ-plane in the three-dimensional coordinate space corresponding to the plane is (X, Z),

2. The calculation device according to claim 1,

wherein the controller is configured to:

calculate the first calibration data for each of a plurality of captured images of the same subject, the plurality of images having parallax therebetween; and

calculate second calibration data by comparing corresponding mapping lines in the plurality of images.

3. The calculation device according to claim 2,

wherein the controller is configured to perform calibration for a parallax deviation between a plurality of cameras that captures the plurality of images, based on the second calibration data.

4. A camera device comprising:

a calculation device including a controller configured to extract, from a captured image of a subject, a plurality of lines corresponding to straight lines parallel to each other located on a plane in the subject, to map the extracted plurality of lines to a three-dimensional coordinate space, and to calculate first calibration data so that mapping lines obtained are parallel to each other; and

one or more cameras configured to capture the image of the subject,

wherein the controller is configured to calibrate an angle θ

_C between an optical axis of at least one camera of the one or more cameras that captures the image and the plane, based on the first calibration data, and

wherein the mapping of the plurality of lines to the three-dimensional coordinate space is conducted according to the following conversion expressions, when coordinate on the captured image expressed as a pixel value is (u, v), and coordinate on XZ-plane in the three-dimensional coordinate space corresponding to the plane is (X, Z),

5. A vehicle comprising:

a calculation device including a controller configured to extract, from a captured image of a subject, a plurality of lines corresponding to straight lines parallel to each other located on a plane in the subject, to map the extracted plurality of lines to a three-dimensional coordinate space, and to calculate first calibration data so that mapping lines obtained are parallel to each other; and

one or more cameras configured to capture the image of the subject,

wherein the controller is configured to calibrate an angle θ

_C between an optical axis of at least one camera of the one or more cameras that captures the image and the plane, based on the first calibration data, and

wherein the mapping of the plurality of lines to the three-dimensional coordinate space is conducted according to the following conversion expressions, when coordinate on the captured image expressed as a pixel value is (u, v), and coordinate on XZ-plane in the three-dimensional coordinate space corresponding to the plane is (X, Z),

6. The vehicle according to claim 5,

wherein the one or more cameras face outside the vehicle.

7. The vehicle according to claim 5,

wherein the controller is configured to perform calibration based on the first calibration data, using a mounting angle of depression of each of the cameras as a reference value.

8. The vehicle according to claim 5,

wherein the controller is configured to extract the plurality of straight lines from a ground portion in the image.

9. A calibration method comprising:

extracting, from a captured image of a subject, a plurality of lines corresponding to straight lines parallel to each other located on a plane in the subject;

mapping the extracted plurality of lines to a three-dimensional coordinate space; and

calculating first calibration data so that mapping lines obtained are parallel to each other,

wherein an angle θ

_C between an optical axis of a camera that captures the image and the plane is calibrated, based on the first calibration data, and

wherein the mapping of the plurality of lines to the three-dimensional coordinate space is conducted according to the following conversion expressions, when coordinate on the captured image expressed as a pixel value is (u, v), and coordinate on XZ-plane in the three-dimensional coordinate space corresponding to the plane is (X, Z),

10. The calibration method according to claim 9, comprising:

calculating the first calibration data for each of a plurality of images having parallax therebetween; and

calculating second calibration data by comparing corresponding mapping lines in the plurality of images.