Calibration device and calibration method for a stereo camera without placing physical markers

• US 9,424,645 B2
• Filed: 05/23/2012
• Issued: 08/23/2016
• Est. Priority Date: 06/06/2011
• Status: Expired due to Fees

• Current Assignee: Sovereign Peak Ventures, LLC (Dominion Harbor Enterprises, LLC)
• Original Assignee: Panasonic Intellectual Property Management Co., Ltd. (Panasonic Holdings Corporation)
• Inventors: Oho, Masahiro, Nishimura, Hirofumi
• Primary Examiner(s): Czekaj, Dave
• Assistant Examiner(s): SLATER, ALISON T
• Application Number:
  US14/123,946
• Publication Number:
  US 20140104393A1
• Time in Patent Office: 1,553 Days
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/30236 Traffic on road, railway or crossing
• CPC Class Codes: G06T 7/85 Stereo camera calibration
• CPC Class Codes: H04N 13/246 Calibration of cameras

Abstract

Provided is a calibration device that can calculate the mounting state of a stereo camera without the placing of a physical marker. In this device, an area setting processor (113) preliminarily sets, for an integrated image in which a road surface image region and a setting-prohibited region are integrated, an arrangement pattern having a plurality of area setting frames, and adjusts the arrangement pattern so that the plurality of area setting frames recede from each other. An extraction processor (114) extracts from distance images a plurality of partial images corresponding to the plurality of area setting frames. A mounting-state calculation unit (104) calculates a mounting-state parameter on the basis of a plurality of partial images, such parameter relating to the mounting state of a stereo camera used to pick up stereo images.

Claims

1. A calibration apparatus comprising:

distance image generation circuitry which, in operation, calculates a distance from a stereo camera for each of a plurality of pixels included in a stereo image including a road surface and that generates a distance image in which each of the plurality of pixels is associated with the calculated distance;

first region detection circuitry which, in operation, detects a road surface image region based on the stereo image including the road surface;

second region detection circuitry which, in operation, detects a setting prohibited region including a group of non-effective pixels with which calculating the distance in the distance image is not possible;

area setting processing circuitry which, in operation, adjusts an arrangement pattern in a direction in which positions of a plurality of area setting frames become distant from each other, based on the road surface image region and the setting prohibited region;

extraction processing that circuitry which, in operation, extracts, from the distance image, a plurality of partial images corresponding to the plurality of area setting frames included in the adjusted arrangement pattern; and

calculation circuitry which, in operation, calculates an installation state parameter used in an installation state of the stereo camera of the stereo image, based on the plurality of extracted partial images.

2. The calibration apparatus according to claim 1, wherein the area setting processing circuitry adjusts, in a state where a position of a reference point of the arrangement pattern is fixed, the arrangement pattern by moving the area setting frames to be mutually distant from the reference point up until immediately before each of the area setting frames and the setting prohibited image region overlap each other while each of the area setting frames and the road surface image region overlap each other.

3. The calibration apparatus according to claim 1, further comprising an adjustment circuitry that, when a difference between a number P1 of effective pixels of a first partial image and a number P2 of effective pixels of a second partial image, where P1>

P2, in the plurality of extracted partial images is greater than a threshold, reduces the number of effective pixels of the first partial image until the difference becomes equal to or less than the threshold.

4. The calibration apparatus according to claim 3,

wherein the adjustment circuitry reduces the number of effective pixels of the first partial image by deleting an effective pixel of the first partial image, the effective pixel being associated with a distance away from the stereo camera in the distance image.

5. The calibration apparatus according to claim 3, wherein the adjustment circuitry reduces the number of effective pixels of the first partial image by deleting an effective pixel of the first partial image, the effective pixel being distant from a center of the distance image.

6. The calibration apparatus according to claim 2, wherein the area setting processing circuitry switches the position at which the reference point of the arrangement pattern is fixed, based on characteristics of a lens used for capturing of the stereo image including the road surface.

7. A calibration method comprising:

calculating, by a calibration apparatus, a distance from a stereo camera for each of a plurality of pixels included in a stereo image including a road surface;

generating, by the calibration apparatus, a distance image in which each of the plurality of pixels is associated with the calculated distance;

detecting, by the calibration apparatus, a road surface image region based on the stereo image including the road surface;

detecting, by the calibration apparatus, a setting prohibited region including a group of non-effective pixels with which calculating the distance in the distance image is not possible;

adjusting, by the calibration apparatus, an arrangement pattern in a direction in which a plurality of area setting frames become distant from each other, based on the road surface image region and the setting prohibited region;

extracting, by the calibration apparatus, from the distance image, a plurality of partial images corresponding to the plurality of area setting frames included in the adjusted arrangement pattern; and

calculating, by the calibration apparatus, an installation state parameter used in an installation state of the stereo camera, based on the plurality of extracted partial images.

8. The calibration method according to claim 7, further comprising:

adjusting, in a state where a position of a reference point of the arrangement pattern is fixed, the arrangement pattern by moving the area setting frames to be mutually distant from the reference point up until immediately before each of the area setting frames and the setting prohibited image region overlap each other while each of the area setting frames and the road surface image region overlap each other.

9. The calibration method according to claim 7, further comprising:

when a difference between a number P1 of effective pixels of a first partial image and a number P2 of effective pixels of a second partial image, where P1>

P2, in the plurality of extracted partial images is greater than a threshold, reducing the number of effective pixels of the first partial image until the difference becomes equal to or less than the threshold.

10. The calibration method according to claim 9,

wherein the reducing the number of effective pixels of the first partial image includes deleting an effective pixel of the first partial image, the effective pixel being associated with a distance away from the stereo camera in the distance image.

11. The calibration method according to claim 9,

wherein the reducing the number of effective pixels of the first partial image includes deleting an effective pixel of the first partial image, the effective pixel being associated with a distance away from a center of the distance image.

12. The calibration method according to claim 7, further comprising:

switching the position at which the reference point of the arrangement pattern is fixed, based on characteristics of a lens used for capturing of the stereo image including the road surface.