Obstacle detection apparatus and a method therefor

• US 20060013438A1
• Filed: 07/12/2005
• Published: 01/19/2006
• Est. Priority Date: 07/13/2004
• Status: Abandoned Application

• Current Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Original Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Inventors: Kubota, Susumu
• Application Number:
  US11/178,274
• Publication Number:
  US 20060013438A1
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: B60W 40/04 Traffic conditions
• CPC Class Codes: G06T 2207/30248 Vehicle exterior or interior
• CPC Class Codes: G06T 7/30 Determination of transform parameters for the alignment of images, i.e. image registration
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads

Abstract

An apparatus of detecting an object on a road surface includes a stereo set of video cameras mounted on a vehicle to produce right and left images, a storage to store the right and left images, a parameter computation unit to compute a parameter representing road planarity constraint based on the images of the storage, a corresponding point computation unit to compute correspondence between a first point on one of the right and left images and a second point on the other, which corresponds to the first point, based on the parameter, an image transformation unit to produce a transformed image from the one image using the correspondence, and a detector to detect an object having a dimension larger than a given value in a vertical direction with respect to the road surface, using the correspondence and the transformed image.

Claims

1. An apparatus of detecting an object on a road surface comprising:

a stereo set of video cameras mounted on a vehicle to produce right and left images on a road surface;

an image storage unit configured to store the right and left images;

a parameter computation unit configured to compute a parameter representing road planarity constraint based on the images stored in the image storage unit;

a corresponding point computation unit configured to compute correspondence between a first point on one image of the right and left images on the road surface and a second point on other image of the right and left images, which corresponds to the first point, based on the parameter;

an image transformation unit configured to produce a transformed image from the one image using the correspondence; and

a detector to detect an object having a dimension larger than a given value in a substantially vertical direction with respect to the road surface, using the correspondence and the transformed image.

2. The apparatus according to claim 1, wherein the detector includes:

a setting unit configured to set a target region including a target point to the one image,

a setting unit configured to set a first reference region corresponding to the target region to the other image,

a setting unit configured to set a second reference region to the transformed image, and

a detecting unit configured to detect the object by comparing the target region with the first reference region and the second reference region.

3. The apparatus according to claim 1, wherein the image transformation unit comprises an affine transformer to transform the other image by using affine transformation.

4. The apparatus according to claim 1, wherein the corresponding point computation unit computes a position of the second point by applying the road planarity constraint to the first point of the one image.

5. The apparatus according to claim 1, wherein the detector includes

a setting unit configured to set a vertical stripe region having a constant length to the one image,

a computation unit configured to compute a matching score between the vertical stripe region and corresponding regions of the other image and the transformed image while changing a hypothesized grounding point in the vertical direction, the hypothesized grounding point being a switching point of reference to the other image and the transformed image,

a position detection unit configured to detect the hypothesized grounding point indicating maximum matching score as a grounding position of the obstacle.

6. A method of detecting an object on a road surface comprising:

acquiring right and left images by a set of stereo video cameras mounted on a vehicle;

storing the right and left images in a storage unit;

obtaining a parameter representing road planarity constraint based on the right and left images stored in the storage unit;

computing the correspondence between a first point set to one image of the right and left images on the road surface and a second point on other image of the right and left images, which corresponds to the first point, based on the parameter;

generating a transformed image from the one image using the correspondence; and

detecting as an obstacle an object having a height larger than a given value with respect to the road surface, using the correspondence and the transformed image.

7. The method according to claim 6, wherein the detection includes setting a target region including a target point to the one image, setting a first reference region corresponding to the target region to the other image, setting a second reference region to the transformed image, and detecting the object as an obstacle by comparing the target region with the first reference region and the second reference region.

8. The method according to claim 6, wherein generating the transformed image comprises producing the transformed image by subjecting the other image to affine transformation.

9. The method according to claim 6, wherein computing the correspondence computes a position of the second point of the other image that corresponds to the first point of the one image by applying the road planarity constraint to the first point of the one image.

10. The method according to claim 6, wherein detecting the object includes setting a vertical strip region having a constant length to the one image, computing goodness of the match between the strip region and the corresponding regions of the other image and the transformed image while moving a hypothetical grounding point in vertical direction, and detecting the grounding point indicating maximum match as a grounding position of the obstacle.

11. An apparatuses to detect an object on a road surface, the apparatus comprising:

a plurality of cameras each capturing an image;

an image memory to store a plurality of images captured by the cameras respectively;

a dividing unit configured to divide a target image captured by one camera of the cameras into a plurality of stripe regions;

a correspondence computing unit configured to compute correspondence between the target image and a reference image captured by other camera of the plurality of cameras for each of the stripe regions based on the stripe regions, when receiving a boundary line function representing a boundary between a road area and an obstacle area in the target image;

an image matching score computing unit configured to compute matching score between a whole of the target image or a part thereof and a corresponding region of the reference image for each of the stripe regions based on the correspondence obtained by correspondence computing unit; and

a boundary line function optimization unit configured to generate an optimum boundary line function maximizing the correspondence with respect to the whole of the target image or the part thereof.

12. The apparatus according to claim 11, wherein, as for a grounding point on a road surface, the correspondence computing unit includes:

a road region corresponding point computing unit configured to obtain correspondence between a coordinate of a projection point corresponding to the grounding point in the target image and a coordinate of a projection point corresponding to the grounding point in the reference image, and

an obstacle region corresponding point computing unit configured to obtain correspondence between a coordinate of a projection point corresponding to a point in the target image including the grounding point and a line perpendicular to the road surface and a coordinate of a projection point in the reference image, and wherein

as for a point in the road region, correspondence between the target image and the reference image is obtained by the road region corresponding point computing unit, and

as for a point in the obstacle region, correspondence between the target image and the reference image is obtained based on the grounding point of the corresponding obstacle by the obstacle region corresponding point computing unit.

13. The apparatus according to claim 11, wherein the image matching score computing unit comprises:

a road surface image transformation unit configured to generate a road surface transformed image of the reference image wherein a pattern of the road region of the reference image matches with a pattern of the road region of the target image, and

an obstacle image transformation unit configured to generate a transformed image of the reference image wherein a reference image pattern of the obstacle standing on the road surface matches with a target image pattern, and wherein

when receiving a target region image representing whole of the target image or part thereof and a boundary line between the obstacle included in the target region image and the road surface, the matching score computing unit computes matching score between the road surface region below the grounding line in the target region image and the road surface transformed image obtained by transforming the reference image with the road surface image transformation unit, and computes matching score between the obstacle region above the grounding line in the target image and the obstacle surface transformed image obtained by transforming the reference image with the obstacle image transformation unit.

14. The apparatus according to claim 11, wherein the plurality of cameras comprise two cameras whose optical axes are substantially parallel with each other, and in which a plane including the optical axes is substantially parallel with the road surface.

15. A method of detecting an object on a road surface, the method comprising:

capturing multiple images with multiple cameras, respectively;

storing in a memory the multiple images;

dividing a target image captured by one camera of the cameras into multiple strip regions;

computing correspondence between the target image and a reference image captured by the other camera of the cameras for each of the strip regions, using a boundary line function representing a boundary between a road area and an obstacle area in the target image;

computing matching value between whole of the target image or a part thereof and a corresponding region of the reference image for each of the strip regions based on the correspondence obtained by correspondence computing unit; and

generating an optimum boundary line function maximizing the matching value with respect to the whole of the target image or the part thereof.

16. The method according to claim 15, wherein, as for a grounding point on a road surface, the correspondence computing includes

computing correspondence between a coordinate of a projection point corresponding to the grounding point in the target image and a coordinate of a projection point corresponding to the grounding point in the reference image, and

computing correspondence between a coordinate of a projection point corresponding to a point in the target image including the grounding point and a line perpendicular to the road surface and a coordinate of a projection point of the grounding point in the reference image, and

as for a point in the road region, correspondence between the target image and the reference image being obtained by the road region corresponding point computing, and

as for a point in the obstacle region, correspondence between the target image and the reference image being obtained based on the grounding point of the corresponding obstacle by the obstacle region corresponding point computing.

17. The method according to claim 15, wherein computing the matching value comprises:

generating a road surface transformed image of the reference image wherein a pattern of the road region of the reference image matches with a pattern of the road region of the target image, and

generating a transformed image of the reference image wherein a reference image pattern of the obstacle standing on the road surface matches with a target image pattern, and receiving the target image region which is the whole or a part of the target image and the boundary line between the obstacle included in the target image region and the road surface as input, and wherein the matching value computation includes computing matching value between the part of the target image region below the boundary line and the road surface transformed image obtained by transforming the reference image by the road surface image transformation unit and computing matching value between the part of the target image region above the boundary line and the obstacle surface transformed image obtained by transforming the reference image with the obstacle image transforming unit.

18. The method according to claim 15, wherein the cameras comprise two cameras whose optical axes are substantially parallel with each other, and in which a plane including the optical axes is substantially parallel with the road surface.