System and method for detecting obstacle

• US 6,990,253 B2
• Filed: 05/22/2002
• Issued: 01/24/2006
• Est. Priority Date: 05/23/2001
• Status: Active Grant

• Current Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Original Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Inventors: Hattori, Hiroshi, Onoguchi, Kazunori, Takeda, Nobuyuki
• Primary Examiner(s): Shah, Sanjiv
• Assistant Examiner(s): Kassa, Yosef
• Application Number:
  US10/151,804
• Publication Number:
  US 20020191837A1
• Time in Patent Office: 1,343 Days
• Field of Search: 382/103, 382/104, 382/154, 382/282, 382/384, 382/294, 382/209, 382/219, 382/278, 340/935, 340/938, 340/937, 701/23, 701/50, 348/139, 348/143
• US Class Current: 382/276
• CPC Class Codes: G01C 11/06 by comparison of two or more pictures of the same area
• CPC Class Codes: G01S 11/12 using electromagnetic waves other than radio waves
• CPC Class Codes: G06T 7/30 Determination of transform parameters for the alignment of images, i.e. image registration
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
• CPC Class Codes: G08G 1/166 for active traffic, e.g. moving vehicles, pedestrians, bikes
• CPC Class Codes: G08G 1/167 Driving aids for lane monitoring, lane changing, e.g. blind spot detection

Abstract

A system for realizing support of safety driving or automatic driving, in which a vehicle-mounted stereo camera is used to detect obstacles on a road surface even in the presence of a reflected image of an obstacle under a wet road condition. The system includes two cameras, an image transforming section transforming a first camera'"'"'s image by image transform T that transforms any given pixel in a road surface region in the first camera'"'"'s image to a corresponding pixel in a second camera'"'"'s image, a lane detecting section detecting a traffic lane on a road on the basis of characteristics of the image obtained from the second camera, a similarity calculating section sequentially establishing processing regions, each having a base formed by one of image lines in a region within a lane, for carrying out inter-image arithmetic operations on the assumption that an obstacle stands from a contact line on a road surface as an image line, and obtaining respective degrees of similarity between the first cameras s image and the second camera'"'"'s image and between the second camera'"'"'s image and the transformed image, and an obstacle judging section detecting the object from a difference between the two kinds of degrees of similarity on the road region sequentially obtained.

Claims

1. An obstacle detecting system detecting an obstacle by using two cameras, the obstacle being on a reference plane in a field of view common to the two cameras, the system comprising:

image inputting means for inputting a first image and a second image from the two cameras, respectively;

image transforming means for transforming the first image to a transformed image on a basis of image transform which is introduced from a geometrical relationship between the reference plane and the two cameras for transforming any given pixel in a reference plane region in the first image to a corresponding pixel in the second image;

similarity calculating means for obtaining;

a degree of similarity D indicating similarity between an image in a processing region established in the second image and an image in a corresponding processing region established in the first image and being a function of position in the vertical direction of the image,

said similarity calculating means treating any given image line in a specified region in the second image as being a contact line of the obstacle with the reference plane, the processing region being established for an inter-image arithmetic operation between the first image and the second image, and a plurality of the processing regions being established with the image line being shifted in the vertical direction, and

a degree of similarity P indicating similarity between an image in the processing region established in the second image and an image in a corresponding processing region established in the transformed image and being a function of position in the vertical direction of the image,

the processing region being established for an inter-image arithmetic operation between the second image and the transformed image, and a plurality of the processing regions being established with the image line being shifted in the vertical direction; and

obstacle judging means for detecting the obstacle on a basis of the degree of similarity D and the degree of similarity P on a reference plane region obtained by the similarity calculating means.

2. The obstacle detecting system as claimed in claim 1 wherein the obstacle judging means obtains a difference K between the degree of similarity D and the degree of similarity P both obtained as being on the reference plane region by the similarity calculating means, and judges that the obstacle is at a position where the difference K is large.

3. The obstacle detecting system as claimed in claim 1 or claim 2 wherein the similarity calculating means determines a width and a height of the processing region on the basis of a position in the vertical direction on the second image.

4. The obstacle detecting system as claimed in claim 2 wherein the obstacle judging means:

judges that there is no obstacle when there is no difference K of degree of similarity equal to or more than a threshold value set beforehand about the vertical directions of all of the images, and

detects that there is the obstacle when there is the difference K of degree of similarity equal to or more than the threshold value in the vertical direction of the image.

5. The obstacle detecting system as claimed in claim 2 wherein:

the similarity calculating means prepares n (1<

i≦

n) kinds of different functions which determine the width and the height of each of the processing regions for obtaining the degree of similarity D_i and the degree of similarity P_i in each of the processing regions determined by each of the functions; and

the obstacle judging means detects a position of the obstacle from each of the degree of similarity D_i and the degree of similarity Pi in each of the processing regions produced by each of the functions, and judges each of all or a part of the detected positions of the obstacle, or a position of the detected obstacle being the closest to the two cameras as the position of the obstacle.

6. The obstacle detecting system as claimed in any one of claims 2 and 4 wherein the obstacle judging means:

carries out threshold value processing of the difference K with a threshold value equal to or more than zero, in which the difference K is given as K(y) as a function of y the direction of which is taken as the vertical direction of the image and a result of the threshold value processing of K(y) is taken as K_th(y);

judges that there is none of the obstacle in a case when K_th(y) satisfies $\underset{y}{\sum <br><br>}<br><br>K_{\mathrm{th}}<br><br>\left(y\right)=0;<br><br>$

and

judges that y_o that makes $<br><br>\mathrm{\alpha <br><br>}<br><br>\underset{y}{\sum <br><br>}<br><br>K_{\mathrm{th}}<br><br>\left(y\right)-\underset{y}{\overset{y_0}{\sum <br><br>}}<br><br>K_{\mathrm{th}}<br><br>\left(y\right)<br><br>$

the minimum is the position of the obstacle in the cases other than the above, where α

is any one of constants from 0 to 1.

7. The obstacle detecting system as claimed in any one of claims 1, 2, and 4, wherein three or more cameras are provided, of which any given two cameras detect the obstacle, and

either a result of detection of a plurality of the obstacles or a position of an obstacle detected as being the closest to the cameras is selected and outputted.

10. The obstacle detecting system as claimed in claim 3, wherein the obstacle judging means:

carries out threshold value processing of the difference K with a threshold value equal to or more than zero, in which the difference K is given as K(y) as a function of y the direction of which is taken as the vertical direction of the image and a result of the threshold value processing of K(y) is taken as K_th(y);

judges that there is none of the obstacle in a case when K_th(y) satisfies $\underset{y}{\sum <br><br>}<br><br><br><br>K_{\mathrm{th}}<br><br>\left(Y\right)=0;<br><br>$

and

judges that y_o that makes $<br><br>\mathrm{\alpha <br><br>}<br><br>\underset{y}{\sum <br><br>}<br><br><br><br>K_{\mathrm{th}}<br><br>\left(y\right)-\underset{y}{\stackrel{y_o}{\sum <br><br>}}<br><br><br><br>K_{\mathrm{th}}<br><br>\left(y\right)<br><br>$

the minimum is the position of the obstacle in the cases other than the above, where α

is any one of constants from 0 to 1.

11. The obstacle detecting system as claimed in claim 3, wherein three or more cameras are provided, of which any given two cameras detect the obstacle, and

either a result of detection of a plurality of the obstacles or a position of an obstacle detected as being the closest to the cameras is selected and outputted.

8. A method of detecting an obstacle by using two cameras, the obstacle being on a reference plane in a field of view common to the two cameras, the method comprising:

an image inputting step of inputting a first image and a second image from the two cameras, respectively;

an image transforming step of transforming the first image to a transformed image on the basis of image transform which is introduced from a geometrical relationship between the reference plane and the two cameras for transforming any given pixel in a reference plane region in the first image to a corresponding pixel in the second image;

an similarity calculating step of obtaining;

a degree of similarity D indicating similarity between an image in a processing region established in the second image and an image in a corresponding processing region established in the first image and being a function of position in the vertical direction of the image,

said similarity calculating means treating any given image line in a specified region in the second image as being a contact line of the obstacle with the reference plane, the processing region being established for an inter-image arithmetic operation between the first image and the second image, and a plurality of the processing regions being established with the image line being shifted in the vertical direction, and

a degree of similarity P indicating similarity between an image in the processing region established in the second image and an image in a corresponding processing region established in the transformed image and being a function of position in the vertical direction of the image,

the processing region being established for an inter-image arithmetic operation between the second image and the transformed image, and a plurality of the processing regions being established with the image line being shifted in the vertical direction; and

an obstacle judging step of detecting the obstacle on the basis of the degree of similarity D and the degree of similarity P on a reference plane region obtained by the similarity calculating step.

9. A computer program product storing computer program instructions which when executed by a computer causes the computer to perform an obstacle detection method in conjunction with two cameras for detecting the obstacle on a reference plane in a field of view common to the two cameras,

the method comprising:

an image inputting step of inputting a first image and a second image from the two cameras, respectively;

an image transforming step of transforming the first image to a transformed image on the basis of image transform which is introduced from a geometrical relationship between the reference plane and the two cameras for transforming any given pixel in a reference plane region in the first image to a corresponding pixel in the second image;

a similarity calculating step of obtaining;

a degree of similarity D indicating similarity between an image in a processing region established in the second image and an image in a corresponding processing region established in the first image and being a function of position in the vertical direction of the image,

said similarity calculating means treating any given image line in a specified region in the second image as being a contact line of the obstacle with the reference plane, the processing region being established for an inter-image arithmetic operation between the first image and the second image, and a plurality of the processing regions being established with the image line being shifted in the vertical direction, and

a degree of similarity P indicating similarity between an image in the processing region established in the second image and an image in a corresponding processing region established in the transformed image and being a function of position in the vertical direction of the image,

the processing region being established for an inter-image arithmetic operation between the second image and the transformed image, and a plurality of the processing regions being established with the image line being shifted in the vertical direction; and

an obstacle judging step of detecting the obstacle on the basis of the degree of similarity D and the degree of similarity P on a reference plane region obtained by the similarity calculating function.