Three-dimensional object detection apparatus, stereo camera apparatus, vehicle, and three-dimensional object detection method

• US 10,567,736 B2
• Filed: 06/23/2016
• Issued: 02/18/2020
• Est. Priority Date: 06/23/2015
• Status: Active Grant

• Current Assignee: Kyocera Corporation
• Original Assignee: Kyocera Corporation
• Inventors: Koishi, Tomofumi
• Primary Examiner(s): Mehta, Bhavesh M
• Assistant Examiner(s): Lemieux, Ian L
• Application Number:
  US15/739,481
• Publication Number:
  US 20180324406A1
• Time in Patent Office: 1,335 Days
• CPC Class Codes: G01C 11/06 by comparison of two or more pictures of the same area
• CPC Class Codes: G01C 3/14 with binocular observation at a single point, e.g. stereoscopic type G01C3/20 takes precedence
• CPC Class Codes: G01S 11/12 using electromagnetic waves other than radio waves
• CPC Class Codes: G01V 8/10 Detecting, e.g. by using light barriers by reflection from the object G01S17/00
• CPC Class Codes: G06T 1/00 General purpose image data processing
• CPC Class Codes: G06T 7/00 Image analysis
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 20/584 of vehicle lights or traffic lights
• CPC Class Codes: H04N 13/225 using parallax barriers
• CPC Class Codes: H04N 13/275 from 3D object models, e.g. computer-generated stereoscopic image signals

Abstract

A three-dimensional object detection apparatus includes an input interface and a controller. The input interface accepts input of first and second images having mutual parallax in a first direction. The controller generates a parallax image that uses a pixel value to represent a difference in the first direction between positions of a subject in the first image and in the second image and divides the parallax image into a plurality of partial regions extending along a second direction. For each partial region, the controller segments a distance in the direction towards the subject into a plurality of distance intervals; and calculates, on the basis of a pixel count of pixels belonging to the distance interval, an evaluation value of a three-dimensional object belonging to the distance interval. The controller uses a different threshold for each distance interval to determine whether or not a three-dimensional object is present.

Claims

1. A three-dimensional object detection apparatus comprising:

an input interface circuit; and

a controller, wherein

the input interface is configured to accept input of a first image and a second image having mutual parallax in a first direction,

the controller is configured to

generate a parallax image that uses a pixel value to represent a difference in the first direction between a position of a subject in the first image and a position of the subject in the second image, and

divide at least a portion of the parallax image into a plurality of partial regions extending along a second direction intersecting the first direction,

the controller is configured, for each partial region, to

segment a distance in a direction towards the subject into a plurality of distance intervals,

calculate, from the pixel value, the distance interval to which each pixel of the partial region belongs,

calculate, on the basis of a pixel count of pixels belonging to the distance interval, an evaluation value corresponding to a length in the second direction of a three-dimensional object belonging to the distance interval, and

determine whether a three-dimensional object is present in each distance interval on the basis of the evaluation value,

the controller is configured to use a different threshold for each distance interval to determine that a three-dimensional object is present when the evaluation value is greater than the threshold,

the evaluation value is calculated by dividing the pixel count of each distance interval by a pixel count of the partial region in the first direction, and

the threshold satisfies a conditional expression,

2. The three-dimensional object detection apparatus of claim 1, wherein the threshold becomes a smaller value with increasing distance in the direction towards the subject.

3. The three-dimensional object detection apparatus of claim 1, wherein the conditional expression is applied at a distance equal to or greater than a distance at which a three-dimensional object with a minimum height targeted for determination is entirely included in the first image and the second image.

4. A stereo camera apparatus comprising:

a stereo camera; and

a controller, wherein

the stereo camera is configured to output a first image and a second image having mutual parallax in a first direction,

the controller is configured to

generate a parallax image that uses a pixel value to represent a difference in the first direction between a position of a subject in the first image and a position of the subject in the second image, and

divide at least a portion of the parallax image into a plurality of partial regions extending along a second direction intersecting the first direction,

the controller is configured, for each partial region, to

segment a distance in a direction towards the subject into a plurality of distance intervals,

calculate, from the pixel value, the distance interval to which each pixel of the partial region belongs,

calculate, on the basis of a pixel count of pixels belonging to the distance interval, an evaluation value corresponding to a length in the second direction of a three-dimensional object belonging to the distance interval, and

determine whether a three-dimensional object is present in each distance interval on the basis of the evaluation value,

the controller is configured to use a different threshold for each distance interval to determine that a three-dimensional object is present when the evaluation value is greater than the threshold,

the evaluation value is calculated by dividing the pixel count of each distance interval by a pixel count of the partial region in the first direction, and

the threshold satisfies a conditional expression,

5. A vehicle comprising:

a stereo camera; and

a controller, wherein

the stereo camera is configured to output a first image and a second image having mutual parallax in a first direction,

the controller is configured to

generate a parallax image that uses a pixel value to represent a difference in the first direction between a position of a subject in the first image and a position of the subject in the second image, and

divide at least a portion of the parallax image into a plurality of partial regions extending along a second direction intersecting the first direction,

the controller is configured, for each partial region, to

segment a distance in a direction towards the subject into a plurality of distance intervals,

calculate, from the pixel value, the distance interval to which each pixel of the partial region belongs,

calculate, on the basis of a pixel count of pixels belonging to the distance interval, an evaluation value corresponding to a length in the second direction of a three-dimensional object belonging to the distance interval, and

determine whether a three-dimensional object is present in each distance interval on the basis of the evaluation value,

the controller is configured to use a different threshold for each distance interval to determine that a three-dimensional object is present when the evaluation value is greater than the threshold,

the stereo camera is configured to image an external environment of the vehicle,

the evaluation value is calculated by dividing the pixel count of each distance interval by a pixel count of the partial region in the first direction, and

the threshold satisfies a conditional expression,

6. A three-dimensional object detection method comprising:

accepting, from a stereo camera, input of a first image and a second image having mutual parallax in a first direction;

generating a parallax image that uses a pixel value to represent a difference in the first direction between a position of a subject in the first image and a position of the subject in the second image; and

dividing at least a portion of the parallax image into a plurality of partial regions extending along a second direction intersecting the first direction;

for each partial region,

segmenting a distance in a direction towards the subject into a plurality of distance intervals;

calculating, from the pixel value, the distance interval to which each pixel of the partial region belongs; and

calculating, on the basis of a pixel count of pixels belonging to the distance interval, an evaluation value corresponding to a length in the second direction of a three-dimensional object belonging to the distance interval; and

using a different threshold for each distance interval, and determining that a three-dimensional object is present when the evaluation value is greater than the threshold, wherein

the evaluation value is calculated by dividing the pixel count of each distance interval by a pixel count of the partial region in the first direction, and

the threshold satisfies a conditional expression,