System and method for detecting obstacle

• US 7,242,817 B2
• Filed: 09/13/2005
• Issued: 07/10/2007
• Est. Priority Date: 05/23/2001
• Status: Expired due to Fees

• Current Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Original Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Inventors: Hattori, Hiroshi, Onoguchi, Kazunori, Takeda, Nobuyuki
• Primary Examiner(s): Kassa; Yosef
• Application Number:
  US11/223,977
• Publication Number:
  US 20060013439A1
• Time in Patent Office: 665 Days
• Field of Search: 382/103, 382/104, 382/219, 382/278, 382/284, 348/207.99, 348/208.12, 348/211.11
• US Class Current: 382/278
• CPC Class Codes: G01C 11/06 by comparison of two or more pictures of the same area
• CPC Class Codes: G01S 11/12 using electromagnetic waves other than radio waves
• CPC Class Codes: G06T 7/30 Determination of transform parameters for the alignment of images, i.e. image registration
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
• CPC Class Codes: G08G 1/166 for active traffic, e.g. moving vehicles, pedestrians, bikes
• CPC Class Codes: G08G 1/167 Driving aids for lane monitoring, lane changing, e.g. blind spot detection

Abstract

A system for realizing support of safety driving or automatic driving is provided, in which a vehicle-mounted stereo camera is used to detect obstacles on a road surface such as vehicles going ahead and pedestrians with a high accuracy even in a situation in which a reflected image of an obstacle or a surrounding structure is appeared under a wet road condition in rainy weather. The system includes an image capture section 12 capturing images from two cameras, an image transforming section 14 transforming a first camera'"'"'s image by image transform T that transforms any given pixel in a road surface region in the first camera'"'"'s image to a corresponding pixel in a second camera'"'"'s image, a lane detecting section 16 detecting a traffic lane on a road on the basis of characteristics of the image obtained from the second camera, a similarity calculating section 18 sequentially establishing processing regions, each having a base formed by one of image lines in a region within a lane, for carrying out inter-image arithmetic operations between the first camera'"'"'s image and the second camera'"'"'s image and between the second camera'"'"'s image and the transformed image on the assumption that an obstacle stands from a contact line on a road surface as an image line, and obtaining respective degrees of similarity between the first camera'"'"'s image and the second camera'"'"'s image and between the second camera'"'"'s image and the transformed image, and an obstacle judging section 20 detecting the object from a difference between the two kinds of degrees of similarity on the road region sequentially obtained by the similarity calculating section 18.

Claims

1. A vehicle detecting an obstacle by using two cameras, the obstacle being on a reference plane in a field of view common to the two cameras, the vehicle comprising:

image inputting means for inputting a first image and a second image from the two cameras, respectively;

image transforming means for transforming the first image to a transformed image on a basis of image transform which is introduced from a geometrical relationship between the reference plane and the two cameras for transforming any given pixel in a reference plane region in the first image to a corresponding pixel in the second image;

similarity calculating means for obtaining;

a degree of similarity D indicating similarity between an image in a processing region established in the second image and an image in a corresponding processing region established in the first image and being a function of position in the vertical direction of the image,

on an assumption that any given image line in a specified region is the second image is a contact line of the obstacle with the reference plane, the processing region being established for an inter-image arithmetic operation between the first image and the second image, and a plurality of the processing regions being established with the image line being shifted in the vertical direction, and

a degree of similarity P indicating similarity between an image in the processing region established in the second image and an image in a corresponding processing region established in the transformed image and being a function of position in the vertical direction of the image,

the processing region being established for an inter-image arithmetic operation between the second image and the transformed image, and a plurality of the processing regions being established with the image line being shifted in the vertical direction; and

obstacle judging means for detecting the obstacle on a basis of the degree of similarity D and the degree of similarity P on a reference plane region obtained by the similarity calculating means.

2. The vehicle as claimed in claim 1 wherein the obstacle judging means obtains a difference K between the degree of similarity D and the degree of similarity P both obtained as being on the reference plane region by the similarity calculating means, and judges that the obstacle is at a position where the difference K is large.

3. The vehicle as claimed in claim 1 or claim 2 wherein the similarity calculating means determines a width and a height of the processing region on the basis of a position in the vertical direction on the second image.

4. The vehicle as claimed in claim 2 wherein the obstacle judging means:

judges that there is no obstacle when there is no difference K of degree of similarity equal to or greater than a threshold value set beforehand about the vertical directions of all of the images, and

detects that there is the obstacle when there is the difference K of degree of similarity equal to or more than the threshold value in the vertical direction of the image.

5. The vehicle as claimed in claim 2 wherein:

the similarity calculating means prepares n (1<

i≦

n) kinds of different functions which determine the width and the height of each of the processing regions for obtaining the degree of similarity D_i and the degree of similarity P_i in each of the processing regions determined by each of the functions; and

the obstacle judging means detects a position of the obstacle from each of the degree of similarity D_i and the degree of similarity Pi in each of the processing regions produced by each of the functions, and judges each of all or a part of the detected positions of the obstacle, or a position of the detected obstacle being the closest to the two cameras as the position of the obstacle.

6. The vehicle as claimed in any one of claims 2 and 4, wherein the obstacle judging means:

carries out threshold value processing of the difference K with a threshold value equal to or greater than zero, in which the difference K is given as K(y) as function of y the direction of which is taken as the vertical direction of the image and a result of the threshold value processing of K(y) is taken as K_th(y);

judges that there is none of the obstacle in a case when K_th(y) satisfies

$\underset{y}{\sum <br><br>}<br><br>K_{\mathrm{th}}<br><br>\left(Y\right)=0;<br><br>$

and

judges that y_o that makes

$<br><br>\mathrm{\alpha <br><br>}<br><br>\underset{y}{\sum <br><br>}<br><br>K_{\mathrm{th}}<br><br>\left(y\right)-\underset{y}{\overset{y_o}{\sum <br><br>}}<br><br>K_{\mathrm{th}}<br><br>\left(y\right)<br><br>$

the minimum is the position of the obstacle in the cases other than the above, where α

is any one of constants from 0 to 1.

7. The vehicle as claimed in any one of claims 1, 2 and 4 wherein three or more cameras are provided, of which any given two cameras detect the obstacle, and

either a result of detection of a plurality of the obstacles or a position of an obstacle detected as being the closest to the cameras is selected and outputted.

8. The vehicle as claimed in claim 3, wherein the obstacle judging means:

carries out threshold value processing of the difference K with a threshold value equal to or more than zero, in which the difference K is given as K(y) as a function of y the direction of which is taken as the vertical direction of the image and a result of the threshold value processing of K(y) is taken as K_th(y);

judges that there is none of the obstacle in a case when K_th(y) satisfies

$\underset{y}{\sum <br><br>}<br><br>K_{\mathrm{th}}<br><br>\left(Y\right)=0;<br><br>$

and

judges that y_o that makes

$<br><br>\mathrm{\alpha <br><br>}<br><br>\underset{y}{\sum <br><br>}<br><br>K_{\mathrm{th}}<br><br>\left(y\right)-\underset{y}{\overset{y_o}{\sum <br><br>}}<br><br>K_{\mathrm{th}}<br><br>\left(y\right)<br><br>$

the minimum is the position of the obstacle in the cases other than the above, where α

is any one of constants from 0 to 1.

9. The vehicle as claimed in claim 3, wherein three or more cameras are provided, of which any given two cameras detect the obstacle, and

either a result of detection of a plurality of the obstacles or a position of an obstacle detected as being the closest to the cameras is selected and outputted.