Obstacle detection system and method therefor

• US 6,963,661 B1
• Filed: 09/11/2000
• Issued: 11/08/2005
• Est. Priority Date: 09/09/1999
• Status: Expired due to Term

• Current Assignee: Toshiba Digital Solutions Corporation (Toshiba Corporation), Toshiba Electronic Devices & Storage Corporation (Toshiba Corporation)
• Original Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Inventors: Hattori, Hiroshi, Onoguchi, Kazunori, Takeda, Nobuyuki
• Primary Examiner(s): AHMED, SAMIR ANWAR
• Application Number:
  US09/659,815
• Time in Patent Office: 1,884 Days
• Field of Search: 382/103, 382/154, 382/199, 382/294, 382/285, 345/419, 345/420, 345/421, 345/422, 345/423, 345/424, 345/425, 345/426, 345/427, 348/142, 348/169, 356/12, 356/26
• US Class Current: 382/154
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/30261 Obstacle
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: G06V 10/147 Details of sensors, e.g. sensor lenses fingerprint or palmprint sensors G06V40/13; vascular sensors G06V40/145; eye sensors G06V40/19
• CPC Class Codes: G06V 10/255 Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 2201/08 Detecting or categorising vehicles

Abstract

An obstacle detection system using stereo cameras mounted on a vehicle, to detect an obstacle existing on a ground plane at a high speed and in a high precision even with the stereo cameras being uncalibrated and with a vibration during a traveling and a change in the inclination of the ground plane. The obstacle detection system comprises: a plurality of uncalibrated TV cameras for inputting stereo images; an image storage unit 2 for storing a plurality of images inputted from the TV cameras; a feature extraction unit 3 for extracting a plurality of mutually parallel lines existing on the ground plane; a parameter computation unit 4 for determining a relation to hold between the projected positions of an arbitrary point of the ground plane upon the individual images, from the plurality of lines extracted by the feature extraction unit 3; and a detection unit 5 for detecting an object having a height from the ground plane, by using the relation determined by the parameter computation unit 4.

Claims

1. An obstacle detection system comprising:

a plurality of TV cameras for inputting multiple images;

an image storage unit for storing a plurality of images inputted from said TV cameras;

a feature extraction unit for extracting from the images, lines existing on a plane in a three-dimensional space;

a parameter computation unit for determining a relation to hold between the projected positions of an arbitrary point on said plane, upon the individual of said stored images, from the lines extracted by said feature extraction unit;

a detection unit for detecting a region absent from said plane, by using the relation computed by said parameter computation unit; and

said parameter computation unit configured to predetermine said relation in a still state, and to determine an undated relation upon an elapsed time based on the relation in a still state and a movement in said plane during the elapsed time of at least one line extracted by said feature extracting unit.

2. An obstacle detection system according to claim 1,

wherein said TV cameras are unknown on their relative positions and orientations and on their focal lengths and principal points.

3. An obstacle detection system according to claim 1 or 2,

wherein the relation to hold between the projected points of an arbitrary point on the plane in the three-dimensional space upon the individual images is expressed by a two-dimensional affine transformation thereby to determine the affine transformation parameters.

4. An obstacle detection system according to claim 1 or 2,

wherein said feature extraction unit extracts a plurality of lines, as existing on the plane in the three-dimensional space and parallel to each other in the three-dimensional space, from the images, and determines the vanishing points of said lines.

5. An obstacle detection system according to claim 1 or 2,

wherein said feature extraction unit extracts a plurality of lines, as existing on the plane in the three-dimensional and parallel to each other in the three-dimensional space, from the images, and determines the inclinations of said lines on the images and the vanishing points of said lines.

11. An obstacle detection system according to claim 3,

wherein said feature extraction unit extracts a plurality of lines, as existing on the plane in the three-dimensional space and parallel to each other in the three-dimensional space, from the images, and determines the vanishing points of said lines.

12. An obstacle detection system according to claim 3,

wherein said feature extraction unit extracts a plurality of lines, as existing on the plane in the three-dimensional and parallel to each other in the three-dimensional space, from the images, and determines the inclinations of said lines on the images and the vanishing points of said lines.

6. An obstacle detection method comprising:

an image storage step of storing a plurality of images inputted from a plurality of TV cameras;

a feature extraction step of extracting from the images, a line existing in a plane of three-dimensional space;

a parameter computation step of determining a relation to hold between the projected positions of an arbitrary point in said plane upon individual of said stored images, from the line extracted at said feature extraction step;

a detection step of detecting a region absent from said plane, by using the relation computed at said parameter computation step; and

said parameter computation step predetermines said relation in a still state, and determines an updated relation upon an elapsed time based on the relation in a still state and a movement of at least one line extracted in said feature extracting step, existing in said plane during the elapsed time.

7. An obstacle detection method according to claim 6,

wherein said TV cameras are unknown on their relative locations and positions and on their focal lengths and principal points.

8. An obstacle detection method according to claim 6 or 7,

wherein said parameter computation step expresses the relation to hold between the projected points of an arbitrary point of the plane in the three-dimensional space upon the individual images by a two-dimensional affine transformation thereby to determine the affine transformation parameters.

9. An obstacle detection method according to claim 6 or 7,

wherein said feature extraction step extracts a plurality of lines, as existing on the plane in the three-dimensional space and parallel to each other in the three dimensional space, from the images, and determines the vanishing points of said lines.

10. An obstacle detection method according to any of claim 6 or 7,

wherein said feature extraction step extracts a plurality of lines, as existing on the plane in the three-dimensional space and parallel to each other in the three dimensional space, from the images and determines the inclinations of said lines on the images and the vanishing points of said lines.

13. An obstacle detection method according to claim 8,

wherein said feature extraction step extracts a plurality of lines, as existing on the plane in the three-dimensional space and parallel to each other in the three dimensional space, from the images, and determines the vanishing points of said lines.

14. An obstacle detection method according to claim 8, wherein said feature extraction step extracts a plurality of lines, as existing on the plane in the three-dimensional space and parallel to each other in the three dimensional space, from the images and determines the inclinations of said lines on the images and the vanishing points of said lines.