Obstacle detection system and method therefor

• US 20050196034A1
• Filed: 04/28/2005
• Published: 09/08/2005
• Est. Priority Date: 09/09/1999
• Status: Active Grant

• Current Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Original Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Inventors: Hattori, Hiroshi, Onoguchi, Kazunori, Takeda, Nobuyuki
• Granted Patent: US 7,362,881 B2
• Time in Patent Office: Days
• US Class Current: 382/154
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/30261 Obstacle
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: G06V 10/147 Details of sensors, e.g. sensor lenses fingerprint or palmprint sensors G06V40/13; vascular sensors G06V40/145; eye sensors G06V40/19
• CPC Class Codes: G06V 10/255 Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 2201/08 Detecting or categorising vehicles

Abstract

An obstacle detection system using stereo cameras mounted on a vehicle, to detect an obstacle existing on a ground plane at a high speed and in a high precision even with the stereo cameras being uncalibrated and with a vibration during a traveling and a change in the inclination of the ground plane. The obstacle detection system comprises: a plurality of uncalibrated TV cameras for inputting stereo images; an image storage unit 2 for storing a plurality of images inputted from the TV cameras; a feature extraction unit 3 for extracting a plurality of mutually parallel lines existing on the ground plane; a parameter computation unit 4 for determining a relation to hold between the projected positions of an arbitrary point of the ground plane upon the individual images, from the plurality of lines extracted by the feature extraction unit 3; and a detection unit 5 for detecting an object having a height from the ground plane, by using the relation determined by the parameter computation unit 4.

Claims

1-5. -5. (canceled)

6. An obstacle detection system comprising:

a plurality of image pickup units having light receiving units having light receiving units disposed on a driver'"'"'s own vehicle at a substantial spacing from each other for taking the regions, to which the light receiving units are directed, simultaneously as images;

an image storage unit for storing the images taken by said image pickup units;

a feature extraction unit for extracting such ones of the regions taken by said image pickup units as correspond to parallel members disposed generally in parallel with each other on a plane, as can be traveled by said own vehicle, from the first image taken by said first image pickup unit and the second image taken by said second image pickup unit, as stored in said image storage unit, to determine intersection points at which said extracted regions intersect in said first and second images;

a difference detection unit for determining the corresponding region in the second image, as corresponding to an arbitrary region in said first image, assuming that said arbitrary region is arising from said plane, from the epipolar constraint to hold between said extracted region and said first and second images, to compare the intensities of said arbitrary region and said corresponding region, thereby to extract the region having a substantially different intensity as an obstacle region to obtain an obstacle region image from the extracted result; and

a height computation unit for extracting a polygonal regions, as composed of an intensity higher than a standard value, of said obstacle region image thereby to detect as a true obstacle region said polygonal region of a threshold or higher value of the ratio which is determined from the vertical size of said polygonal region in said obstacle region image and the size from the lower end of said polygonal region to scan-line in said obstacle region image including said intersection point.

7. An obstacle diction system comprising;

a plurality of image pickup units having light receiving units disposed at a substantial spacing from each other for taking the regions, to which the light receiving units are directed, simultaneously as images;

an image storage unit for storing the images taken by said image pickup units;

a difference detection unit for determining the corresponding region in the second image, as corresponding to an arbitrary region in said first image, assuming that said arbitrary region is arising from a plane in a three-dimensional space, to compare the intensities of said arbitrary region and said corresponding region, thereby to extract the region having a substantially different intensity as an obstacle region to obtain an obstacle region image from the extracted result; and

a height computation unit for extracting a polygonal region, as composed of an intensity higher than a standard value, of said obstacle region image thereby to detect as a true obstacle region said polygonal region of a threshold or higher value of the reatio which is determined from the vertical size of said polygonal region in said obstacle region image and the size from the lower end of said polygonal region to the scan-line set in said obstacle region image.

8. An obstacle detection system comprising:

a first image pickup unit and a second image pickup unit for obtaining a first image information of a first image and a second image information of a second image, respectively, by taking the surrounding region of a driver'"'"'s own vehicle substantially simultaneously as images formed of a set of pixels from light receiving units arranged at a spacing on the own vehicle;

an image information storage unit for storing said first image information and said second image information;

an intensity difference image forming unit for forming an intensity difference image by determining the corresponding pixels in said second image of the second image information, as assuming that an arbitrary pixel of said first image of said first image information stored in said image information storage unit exists on the ground plane being traveled by said own vehicle, to determine the intensity difference between said arbitrary pixel and said corresponding pixel;

a discrimination image forming unit for obtaining a discrimination image by discriminating each pixel in said intensity difference image into a pixel having an intensity difference no less than a standard value and a pixel having an intensity difference less than the standard value; and

a decision unit for detecting and deciding a region having a generally wedge-shaped set of pixels in said discrimination image as an obstacle region.

9. An obstacle detection system according to claim 8,

wherein said detection unit decides that the lowermost pixel in said wedge-shaped region of said discrimination image is either at a point of contact between said obstacle region in said first or second image, as corresponding to said discrimination image, and the ground being traveled by said own vehicle, or at a portion of said obstacle region and the closest to said own vehicle.

10. An obstacle detection system according to claim 8 or 9,

wherein said detection unit decides that such one of said generally wedge-shaped region existing generally in the can-line direction of said first and second images, as corresponding to said discrimination image, that its side is at a higher location of said first and second images than the apexes opposed thereto, is the obstacle region.

11. An obstacle detection system according to claim 8,

wherein said detection unit decides one pair of wedge-shaped regions generally of the same shape, as located at spacing on the generally identical scan-line in said discrimination image, and decides the region between said paired wedge-shaped regions as the obstacle.

12. An obstacle detection system comprising:

an image input unit for inputting and storing at least two images of different pickup points;

a feature extraction unit for extracting a projected point of motion of an object, as stands still or moves on a plane in a three-dimensional space with respect to the pickup point of a standard image, upon said standard image corresponding to an infinite point, by employing one of the images stored by said image input unit as said standard image and other as a reference image;

a detection unit for calculating a corresponding point on said reference image when it is assumed that an arbitrary point of said standard image is on said plane, to detect a point non-existing on said plane from the intensity difference between said arbitrary point and said corresponding point; and

a contact time computing unit for computing the time period for the point non-existing on said plane to come to the taken point of said standard image, one the basis of the point non-existing on said plane in said standard image, on the basis of the point non-existing on said plane in said standard image detected by said detection unit and said projected point extracted from said feature extraction unit.

13. An obstacle detection system according to claim 12,

wherein said contact time computation unite computes the time period for the point on the boundary with said plane to come to the taken point of said standard image, on the basis of such ones of the points detected by said detection unit but not existing on said plane, as are located on the boundary line with said plane in said standard image and are extracted as said projected point by said feature extraction unit.

14. An obstacle detection system according to claim 12 or 13,

wherein said feature extraction unit extracts a plurality of lines aligned to the direction of the motion of an object which stands still or moves on said plane relative to the pickup point of said standard image to employ the point of intersection of the extracted lines, as said projected point.

15. An obstacle detection system according to any claims 12 to 14,

wherein said images are unknown on the relative positions and orientations of their pickup points and on their focal lengths and principal points.

16. An obstacle diction system according to any of claims 12 to 15,

wherein said detection unit computes the corresponding point on said reference image when the arbitrary point on said standard image is assumed to be one said plane, to detect said non-existing point from a similarity of the surrounding intensities of said arbitrary point and said corresponding point.

17-21. -21. (canceled)

22. An obstacle detection method comprising:

a plurality of image pickup steps of taking the regions, to which light receiving units disposed on a driver'"'"'s own vehicle at a substantial spacing from each other are directed, simultaneously as images;

an image storage step of storing the images taken by said image pickup steps;

a feature extraction step of extracting such ones of the regions taken by said image pickup steps as correspond to parallel members disposed generally in parallel with each other on a plane, as can be traveled by said own vehicle, from the first image taken by said first image pickup step and the second image taken by said second image pickup step, as stored in said image storage step, to determine a point of intersection at which said extracted regions intersect in said first and second images;

a difference detection step of determining the corresponding region in the second image, as corresponding to an arbitrary region in said first image, assuming that said arbitrary region is caused by said plane, from the epipolar constraint to hold between said extracted region and said first and second images, to compare the intensities of said arbitrary region and said corresponding region, thereby to extract the region having a substantially different intensity as an obstacle region to obtain an obstacle region image from the extracted result; and

a height computation step of extracting a polygonal region, as composed of an intensity higher than a standard value, of said obstacle region image thereby to detect as a true obstacle region said polygonal region of a threshold or higher value of the ratio which is determined from the vertical size of said polygonal region in said obstacle region image and the size from the lower end of said polygonal region to the scan-line in said obstacle region image including said intersection point.

23. An obstacle detection method comprising:

a plurality of image pickup steps of taking the regions, to which the light receiving units disposed at a substantial spacing from each other are directed, simultaneously as images;

an image storage step of storing the images taken by said image pickup steps;

a difference detection step of determining the corresponding region in the second image, as corresponding to an arbitrary region in said first image, assuming that said arbitrary region is caused by a plane in a three-dimensional space, to compare the intensities of said arbitrary region and said corresponding region, thereby to extract the region having a substantially different intensity as an obstacle region to obtain an obstacle region image from the extracted result; and

a height computation step of extracting a polygonal region, as composed of an intensity higher than a standard value, of said obstacle region image thereby to detect as a true obstacle region said polygonal region of a threshold or higher value of the ration which is determined from the vertical size of said polygonal region in said obstacle region image and the size from the lower end of said polygonal region to the scan-line set in said obstacle region image.

24. An obstacle detection method comprising:

a first image pickup step and a second image pickup step of obtaining a first image information of a first image and a second image information of a second image, respectively, by taking the surrounding region of a driver'"'"'s own vehicle substantially simultaneously as images formed of a set of pixels from light receiving units arranged at a spacing on the own vehicle;

an image information storage step of storing said first image information and said second image information;

an intensity difference image forming step of forming an intensity difference image by determining the corresponding pixels in said second image of the second image information, as assuming that an arbitrary pixel of said first image of said first image information stored in said information storage step exists on the ground plane being traveled by said own vehicle, to determine the intensity difference between said arbitrary pixel and said corresponding pixel;

a discrimination image forming step of obtaining a discrimination image by discriminating each pixel in said intensity difference image into a pixel having an intensity difference no less than a standard value and a pixel having an intensity difference less than the standard value; and

a decision step of detecting and deciding a region having a generally wedge-shaped set of pixels in said discrimination image as an obstacle region.

25. An obstacle detection method according to claim 24,

wherein said detection step decides that the lowermost pixel in said wedge-shaped region of said discrimination image is either at a point of contact between said obstacle region in said first or second image, as corresponding to said discrimination image, and the ground being traveled by said own vehicle, or at a portion of said obstacle region and the closest to said own vehicle.

26. An obstacle detection method according to claim 24 or 25,

wherein said detection step decides that such one of said generally wedge-shaped region existing generally in the scan-line direction of said first and second images, as corresponding to said discrimination image, that its side is at a higher location of said first and second images than the apexes opposed thereto, is the obstacle region.

27. An obstacle detection method according to claim 24,

wherein said detection step decides one pair of wedge-shaped regions generally of the same shape, as located at spacing on the generally identical scan-line in said discrimination image, and decides the region between said paired wedge-shaped regions as the obstacle.

28. An obstacle detection method comprising:

an image input step of inputting and storing at least two images of different pickup points;

a feature extraction step of extracting a projected point of motion of an object, as stands still or moves on a plane in a three-dimensional space with respect to the pickup point of a standard image, upon said standard image corresponding to an infinite point, by employing one of the images stored by said image input step as said standard image and the other as a reference image;

a detection step of calculating a corresponding point on said reference image when it is assumed that an arbitrary point on said standard image is on said plane, to detect a point non-existing on said plane from intensity difference between said arbitrary point and said corresponding point; and

a contact time computation step of computing the time period for the point non-existing on said plane to come to the taken point of said standard image, one the basis of the point non-existing on said plane in the said standard image detected by said detection step and said projected point extracted from said feature extraction step.

29. An obstacle detection method according to claim 28,

wherein said contact time computation step computes the time period for the point on the boundary with said plane to come to the taken point of said standard image, on the basis of such ones of the points detected by said detection step but not existing on said plane, as are located on the boundary line with said plane in said standard image and are extracted as said projected point by said feature extraction step.

30. An obstacle detection method according to claim 28 or 29,

wherein said feature extraction step extracts a plurality of lines aligned to the direction of the motion of an object which stands still or moves on said plane relative to the pickup point of said standard image, to employ the point of intersection of the extracted lines, as said projected point.

31. An obstacle detection method according to any claims 28 to 30,

wherein said images are unknown on the relative locations and positions of their pickup points and on their focal lengths and principal points.

32. An obstacle detection method according to any of claims 28 to 31,

wherein said detection step computes the corresponding point on said reference image when the arbitrary point on said standard image is assumed to be on said plane, to detect said non-existing point from the similarity of the surrounding intensities of said arbitrary point and said corresponding point.