Three-dimensional object detection device and three-dimensional object detection method

• US 10,397,544 B2
• Filed: 07/29/2011
• Issued: 08/27/2019
• Est. Priority Date: 08/19/2010
• Status: Active Grant

• Current Assignee: Nissan Motor Co., Ltd.
• Original Assignee: Nissan Motor Co., Ltd.
• Inventors: Tsuchiya, Chikao, Furushou, Hiroyuki, Tanaka, Shinya, Hayakawa, Yasuhisa
• Primary Examiner(s): Harold, Jefferey F
• Assistant Examiner(s): Karwan, Sihar A
• Application Number:
  US13/817,011
• Publication Number:
  US 20130141542A1
• Time in Patent Office: Days
• Field of Search: 348 46
• CPC Class Codes: G06T 2207/30261 Obstacle
• CPC Class Codes: G06T 3/00 Geometric image transformations in the plane of the image
• CPC Class Codes: G06T 7/12 Edge-based segmentation
• CPC Class Codes: G06V 10/44 Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 20/64 Three-dimensional objects
• CPC Class Codes: H04N 13/204 using stereoscopic image cameras stereoscopic photography G03B35/00

Abstract

A three-dimensional object detection device 1 includes a camera 10 and a calculator 20. The calculator 20 performs viewpoint conversion processing on an image captured by the camera 10 to create a bird'"'"'s eye view image, calculates, for each of a plurality of positions along a vertical imaginary line extending in a vertical direction in an actual space, a luminance difference between two pixels near the position, and detects a three-dimensional object on the basis of continuities of the calculated luminance differences of the respective positions.

Claims

1. A three-dimensional object detection device comprising:

a camera that captures a single image of a predetermined area;

a computer programmed to

perform viewpoint conversion processing on the image captured by the camera to create a conversion image viewed from above,

set a line segment extending in a vertical direction in an actual space as a vertical imaginary line in the conversion image, set two pixels at the same height in the actual space for each of a plurality of positions along the vertical imaginary line, and calculate a luminance difference in the conversion image between the two pixels set for each of the plurality of positions along the vertical imaginary line,

detect an edge line on the basis of continuities of the luminance differences among the plurality of positions, and detect a three-dimensional object on the basis of the edge line.

2. The three-dimensional object detection device according to claim 1, wherein

the vertical imaginary line consists of a first vertical imaginary line which extends in the vertical direction in the actual space and a second vertical imaginary line which is away from the first vertical imaginary line by a predetermined distance in the actual space and which extends in the vertical direction in the actual space, and

the computer calculates the luminance differences between a pixel on the first vertical imaginary line and a pixel on the second vertical imaginary line.

3. The three-dimensional object detection device according to claim 1, wherein

the computer calculates the luminance difference between the two pixels which are equidistant from the vertical imaginary line in the actual space.

4. The three-dimensional object detection device according to claim 1, wherein

the computer calculates the luminance difference between the two pixels which are equidistant from the vertical imaginary line in the conversion image, and

the computer sets a smaller threshold for a higher position among the plurality of positions along the vertical imaginary line in the actual space, and a threshold for determining the three-dimensional object on the basis of the luminance difference is calculated by the computer.

5. The three-dimensional object detection device according to claim 1, wherein

the computer detects the three-dimensional object on the basis of a quantity of the edge lines.

6. The three-dimensional object detection device according to claim 5, wherein

the computer adds an attribute to each of the positions on the vertical imaginary line on the basis of the luminance difference of the position on the vertical imaginary line, and

the computer determines whether the vertical imaginary line is the edge line on the basis of continuities of the attributes.

7. The three-dimensional object detection device according to claim 5, wherein when the change in luminance of pixels along any of the edge lines detected by the computer is larger than a predetermined value, the computer avoids using the edge line for detection of the three-dimensional object.

8. The three-dimensional object detection device according to claim 1, wherein:

the computer calculates an edge intensity of the vertical imaginary line on the basis of the luminance difference, and

the computer detects the three-dimensional object based on a sum of edge intensities.

9. A three-dimensional object detection method comprising:

capturing a single image of a predetermined area;

performing viewpoint conversion processing on the captured image to create a conversion image viewed from above;

setting a line segment extending in a vertical direction in an actual space as a vertical imaginary line in the conversion image, setting two pixels at the same height in the actual space for each of a plurality of positions along the vertical imaginary line, and calculating a luminance difference between the two pixels set for each of the plurality of positions along the vertical imaginary line in the conversion image;

detecting an edge line on the basis of continuities of the luminance differences among the plurality of positions; and

detecting a three-dimensional object on the basis of the edge line.

10. A three-dimensional object detection method, comprising:

capturing a single image of a predetermined area;

performing viewpoint conversion processing on the captured image to create a conversion image viewed from above;

setting a line segment extending in a vertical direction in an actual space as a vertical imaginary line in the conversion image, setting two pixels at the same height in the actual space for each of a plurality of positions along the vertical imaginary line, and calculating a luminance difference between the two pixels set for each of the plurality of positions along the vertical imaginary line in the conversion image; and

detecting an edge line on the basis of continuities of the luminance differences among the plurality of positions, and detecting a three-dimensional object on the basis of the edge line.

11. A three-dimensional object detection method, comprising:

capturing a single image of a predetermined area;

performing viewpoint conversion processing on the captured image to create a conversion image viewed from above;

setting a line segment extending in a vertical direction in an actual space as a vertical imaginary line in the conversion image, setting two pixels at the same height in the actual space for each of a plurality of positions along the vertical imaginary line, and calculating a luminance difference between the two pixels set for each of the plurality of positions along the vertical imaginary line in the conversion image;

detecting

an edge line on the basis of continuities of the luminance differences among the plurality of positions, and detecting a three-dimensional object on the basis of the edge line.