Camera calibrating apparatus

• US 7,352,388 B2
• Filed: 04/21/2003
• Issued: 04/01/2008
• Est. Priority Date: 04/22/2002
• Status: Active Grant

• Current Assignee: Panasonic Automotive Systems Company of America (Panasonic Holdings Corporation)
• Original Assignee: Matsushita Electric Industrial Company Limited (Panasonic Holdings Corporation)
• Inventors: Mato, Ryuichi, Miwa, Michio, Sato, Masaki, Masuda, Satoru
• Primary Examiner(s): Vu; Ngoo-Yen
• Assistant Examiner(s): Meyers; James A
• Application Number:
  US10/512,039
• Publication Number:
  US 20050179801A1
• Time in Patent Office: 1,807 Days
• Field of Search: 348/148, 348/187
• US Class Current: 348/187
• CPC Class Codes: G03B 15/00 Special procedures for taking photographs; Apparatus therefor
• CPC Class Codes: G06T 5/80 Geometric correction
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration

Abstract

Herein disclosed is a camera calibrating apparatus for calibrating an optical parameter of a camera mounted on an automotive vehicle, comprising: a first housing position information storing section (115) for storing first housing position information, a second housing position information storing section (116) for storing second housing position information, a first optical position information producing section (117) for producing first optical position information, a first optical position information storing section (118) for storing the first optical position information, a second optical position information producing section (120) for producing second optical position information, a second optical position information storing section (130) for storing the second optical position information, and a calibrating section (160) for calibrating the second optical position information stored in the second optical position information storing section (130) on the basis of image information obtained by a camera (110) in a second coordination system (102).

Claims

1. A camera calibrating apparatus to be operative in combination with an imaging device which includes a housing unit and an optical section supported by said housing unit to obtain image information through said optical section, and adapted to calibrate optical position information indicative of a position of said optical section, comprising:

first housing position information storing means for storing first housing position information indicative of a position of said housing unit in the first coordinate system in which a revising marker is located;

second housing position information storing means for storing second housing position information indicative of a position of said housing unit in the second coordinate system in which a calibrating marker is located;

first optical position information calculating means for calculating first optical position information indicative of a position of said optical section in said first coordinate system on the basis of said image information of said revising marker obtained by said imaging device;

first optical position information storing means for storing said first optical position information produced by said first optical position information producing means;

second optical position information calculating means for calculating second optical position information indicative of a position of said optical section to said second coordinate system from said second housing position information stored by said second housing position information storing means on the basis of said first housing position information stored by said first housing position information storing means and said first optical position information stored by said first optical position information storing means;

estimated position information calculating means for calculating estimated position information indicative of a position of said calibrating marker to an image coordinate system of said imaging device on the basis of said second optical position information produced by said second optical position information producing means;

second optical position information storing means for storing said second optical position information produced by said second optical position information producing means;

estimated position information storing means for storing said estimated position information produced by said estimated position information producing means; and

calibrating means for calibrating said second optical position information stored by said second optical position information storing means on the basis of said image information of said calibrating marker obtained by said imaging device and said estimated position information stored by said estimated position information storing means.

2. A camera calibrating apparatus as set forth in claim 1, in which said calibrating means includes:

image location information extracting means for extracting an image location information indicative of an image location of said calibrating marker to said image coordinate system of said imaging device on the basis of said image information of said calibrating marker obtained by said imaging device;

calibration value calculating means for calculating a calibration value of said second optical position information stored by said second optical position information storing means on the basis of said image location information extracted by said image location information extracting means and said estimated position information stored by said estimated position information storing means; and

optical position information calibrating means for calibrating said second optical position information stored by said second optical position information storing means on the basis of said calibration value calculated by said calibration value calculating means.

3. A camera calibrating apparatus as set forth in claim 2, in which

said calibrating means is adapted to calibrate a deviation of a rotational component of said second optical position information.

4. A camera calibrating apparatus as set forth in claim 2, in which

said image location information extracting means includes:

image displaying means for displaying an image of said calibrating marker obtained by said imaging device, and

image location specifying means for specifying said image location of said calibrating marker on said image of said calibrating marker displayed by said image displaying means and to ensure that said image location information is extracted.

5. A camera calibrating apparatus as set forth in claim 2, in which

said image location information extracting means includes:

estimated area information storing means for storing estimated area information indicative of said calibrating marker to said image coordinate system of said imaging device; and

image location searching means for searching said image location of said calibrating marker from said image information of said calibrating marker obtained by said imaging device on the basis of said estimated area information stored by said estimated area information storing means and said estimated position information stored by said estimated position information storing means to ensure that said image location information is extracted.

11. A camera calibrating apparatus as set forth in any one of claims 1 to 6 and 7 to 10, in which said imaging device is mounted on an automotive vehicle.

12. An imaging system comprises a camera calibrating apparatus as set forth in any one of claims 1 to 6 and 7 to 10.

13. An imaging control system comprises a camera calibrating apparatus as set forth in any one of claims 1 to 6 and 7 to 10.

6. A camera calibrating apparatus to be operative in combination with an imaging device which includes a housing unit and an optical section supported by said housing unit to obtain image information through said optical section, and adapted to calibrate optical position information indicative of a position of said optical section, comprising:

first housing position information storing means for storing first housing position information indicative of a position of said housing unit in the first coordinate system in which a revising marker is located;

second housing position information storing means for storing second housing position information indicative of a position of said housing unit in the second coordinate system;

first optical position information calculating means for calculating first optical position information indicative of a position of said optical section in said first coordinate system on the basis of said image information obtained by said imaging device and indicative of said revising marker;

first optical position information storing means for storing said first optical position information produced by said first optical position information producing means;

second optical position information calculating means for calculating second optical position information indicative of a position of said optical section to said second coordinate system from said second housing position information stored by said second housing position information storing means on the basis of said first housing position information stored by said first housing position information storing means and Said first optical position information stored by said first optical position information storing means;

second optical position information storing means for storing said second optical position information produced by said second optical position information producing means; and

calibrating means for calibrating said second optical position information stored by said second optical position information storing means on the basis of a motion vector of said image information obtained by said imaging device in said second coordinate system,

wherein said calibrating means includes;

plane-projected image producing means for producing a plane-projected image from said image information obtained by said imaging device in said second coordinate system;

plane-projected image dividing means for dividing said plane-projected image produced by said plane-projected image producing means into a plurality of image segments;

motion vector extracting means for extracting said motion vector from said image segments divided by said plane-projected image dividing means;

calibration value calculating means for calculating a calibration value of said second optical position information stored by said second optical position information storing means on the basis of said motion vector extracted by said motion vector extracting means; and

optical position information calibrating means for calibrating said second optical position information stored by said second optical position information storing means on the basis of said calibration value calculated by said calibration value calculating means.

7. A camera calibrating apparatus as set forth in claim 6, in which

a dividing marker is located in said second coordinate system in a predetermined relationship with said position of said housing unit represented by said second position information stored by said second position information storing means, and in which

plane-projected image dividing means is adapted to divide said plane-projected image produced by said plane-projected image producing means into a plurality of image segments on the basis of said image information of said dividing marker obtained by said imaging device.

8. A camera calibrating apparatus to be operative in combination with an imaging device which includes a housing unit and an optical section supported by said housing unit to obtain image information through said optical section, and adapted to calibrate position information of said optical section, comprising:

first housing position information storing means for storing first housing position information indicative of a position of said housing unit in the first coordinate system in which a revising marker is located;

second housing position information storing means for storing second housing position information indicative of a position of said housing unit in the second coordinate system in which an automotive vehicle is located;

first optical position information producing calculating means for calculating first optical position information indicative of a position of said optical section in said first coordinate system on the basis of said image information of said revising marker obtained by said imaging device;

first optical position information storing means for storing said first optical position information produced by said first optical position information producing means;

second optical position information calculating means for calculating second optical position information indicative of a position of said optical section to said second coordinate system from said second housing position information stored by said second housing position information storing means on the basis of said first housing position information stored by said first housing position information storing means and said first optical position information stored by said first optical position information storing means;

estimated position information calculating means for calculating estimated position information indicative of a position of said automotive vehicle to an image coordinate system of said imaging device on the basis of said second optical position information produced by said second optical position information producing means;

second optical position information storing means for storing said second optical position information produced by said second optical position information producing means;

estimated position information storing means for storing said estimated position information produced by said estimated position information estimating means; and

calibrating means for calibrating said second optical position information stored by said second optical position information storing means on the basis of said image information of said automotive vehicle obtained by said imaging device and said estimated position information stored by said estimated position information storing means.

9. A camera calibrating apparatus as set forth in claim 8, in which

said calibrating means includes:

image location information extracting means for extracting an image location information indicative of an image location of said automotive vehicle in said image coordinate system of said imaging device on the basis of said image information of said automotive vehicle obtained by said imaging device;

calibration value calculating means for calculating a calibration value of said second optical position information stored by said second optical position information storing means on the basis of said image location information extracted by said image location information extracting means and said estimated position information stored by said estimated position information storing means; and

optical position information calibrating means for calibrating said second optical position information stored by said second optical position information storing means on the basis of said calibration value calculated by said calibration value calculating means.

10. A camera calibrating apparatus as set forth in claim 9, in which

said calibration value calculating means includes:

superimposing means for superimposing a profile line of said automotive vehicle represented by said image location information on a profile line of said automotive vehicle represented by said estimated position information;

extracting means for extracting a plurality of points from said overlapped profile lines of said automotive vehicle superimposed by said superimposing means, and

calculating means for calculating a calibration value of said second optical position information by comparing said points of said image location information with said points of said estimated position information.

14. A camera calibrating method of calibrating optical position information indicative of a position of an optical section supported by a housing unit of a camera for obtaining image information through said optical section, comprising:

a first housing position information storing step of storing first housing position information indicative of a position of said housing unit in the first coordinate system in which a revising marker is located;

a second housing position information storing step of storing second housing position information indicative of a position of said housing unit in the second coordinate system in which a calibrating marker is located;

a first optical position information calculating step of calculating first optical position information indicative of a position of said optical section in said first coordinate system on the basis of said image information of said revising marker obtained by said imaging device;

a first optical position information storing step of storing said first optical position information produced in said first optical position information producing step;

a second optical position information calculating step of calculating second optical position information indicative of a position of said optical section to said second coordinate system from said second housing position information stored in said second housing position information storing step on the basis of said first housing position information stored in said first housing position information storing step and said first optical position information stored in said first optical position information storing step;

an estimated position information calculating step of calculating estimated position information indicative of a position of said calibrating marker to an image coordinate system of said imaging device on the basis of said second optical position information produced in said second optical position information producing step;

a second optical position information storing step of storing said second optical position information produced in said second optical position information producing step;

an estimated position information storing step of storing said estimated position information produced in said estimated position information estimating step; and

a calibrating step of calibrating said second optical position information stored in said second optical position information storing step on the basis of said image information of said calibrating marker obtained by said imaging device and said estimated position information stored in said position information storing step.

17. A camera calibrating method as set forth in any one of claims 14 to 16, in which said imaging device is mounted on an automotive vehicle.

15. A camera calibrating method of calibrating optical position information indicative of a position of an optical section supported by a housing unit of a camera for obtaining image information through said optical section, comprising:

a first housing position information storing step of storing first housing position information indicative of a position of said housing unit in the first coordinate system in which a revising marker is located;

a second housing position information storing step of storing second housing position information indicative of a position of said housing unit in the second coordinate system;

a first optical position information calculating step of calculating first optical position information indicative of a position of said optical section in said first coordinate system on the basis of said image information obtained by said imaging device and indicative of said revising marker;

a first optical position information storing step of storing said first optical position information produced in said first optical position information producing step;

a second optical position information calculating step of calculating second optical position information indicative of a position of said optical section to said second coordinate system from said second housing position information stored in said second housing position information storing step on the basis of said first housing position information stored in said first housing position information storing step and said first optical position information stored in said first optical position information storing step;

a second optical position information storing step of storing said second optical position information produced in said second optical position information producing step; and

a calibrating step of calibrating said second optical position information stored in said second optical position information storing step on the basis of a motion vector of said image information obtained by said imaging device in said second coordinate system,

wherein said calibrating step includes;

a plane-projected image producing step of producing a plane-projected image from said image information obtained by said imaging device in said second coordinate system;

a plane-projected image dividing step of dividing said plane-projected image produced by said plane-projected image producing step into a plurality of image segments;

a motion vector extracting step of extracting said motion vector from said image segments divided by said plane-projected image dividing step;

a calibration value calculating step of calculating a calibration value of said second optical position information stored by said second optical position information storing step on the basis of said motion vector extracted by said motion vector extracting step; and

a optical position information calibrating step of calibrating said second optical position information stored by said second optical position information storing step on the basis of said calibration value calculated by said calibration value calculating step.

16. A camera calibrating method of calibrating optical position information indicative of a position of an optical section supported by a housing unit of a camera for obtaining image information through said optical section, comprising:

a first housing position information storing step of storing first housing position information indicative of a position of said housing unit in the first coordinate system in which a revising marker is located;

a second housing position information storing step of storing second housing position information indicative of a position of said housing unit in the second coordinate system in which an automotive vehicle is located;

a first optical position information calculating step of calculating first optical position information indicative of a position of said optical section in said first coordinate system on the basis of said image information of said revising marker obtained by said imaging device;

a first optical position information storing step of storing said first optical position information produced in said first optical position information producing step;

a second optical position information calculating step of calculating second optical position information indicative of a position of said optical section to said second coordinate system from said second housing position information stored in said second housing position information storing step on the basis of said first housing position information stored in said first housing position information storing step and said first optical position information stored in said first optical position information storing step;

an estimated position information calculating step of calculating estimated position information indicative of a position of said automotive vehicle to an image coordinate system of said imaging device on the basis of said second optical position information produced in said second optical position information producing step;

a second optical position information storing step of storing said second optical position information produced in said second optical position information producing step;

an estimated position information storing step of storing said estimated position information produced in said estimated position information estimating step; and

a calibrating step of calibrating said second optical position information stored in said second optical position information storing step on the basis of said image information of said automotive vehicle obtained by said imaging device and said estimated position information stored in said estimated position information storing step.