Obstacle detection apparatus and method

• US 20040056950A1
• Filed: 09/03/2003
• Published: 03/25/2004
• Est. Priority Date: 09/25/2002
• Status: Active Grant

• Current Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Original Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Inventors: Takeda, Nobuyuki
• Granted Patent: US 6,985,075 B2
• Time in Patent Office: Days
• US Class Current: 348/92
• CPC Class Codes: G08G 1/166 for active traffic, e.g. moving vehicles, pedestrians, bikes
• CPC Class Codes: G08G 1/167 Driving aids for lane monitoring, lane changing, e.g. blind spot detection
• CPC Class Codes: H04N 7/188 Capturing isolated or intermittent images triggered by the occurrence of a predetermined event, e.g. an object reaching a predetermined position signal generation from motion picture films H04N5/253

Abstract

A stereo camera inputs a first image and a second image each including a predetermined plane. An image conversion unit projects each pixel of the predetermined plane from the first image onto a corresponding pixel of the second image as a third image. A feature conversion unit respectively emphasizes edges included in the first image, the second image and third image. A similarity calculation unit sets a plurality of areas on the predetermined plane from the second feature conversion image, calculates a first similarity between each area of the second feature conversion image and a corresponding area of the first feature conversion image, and calculates a second similarity between each area of the second feature conversion image and a corresponding area of the third feature conversion image. An obstacle decision unit detects an obstacle based on the first similarity and the second similarity.

Claims

1. An obstacle detection apparatus, comprising:

a first camera inputting a first image including a predetermined plane and a second camera inputting a second image including the predetermined plane;

an image memory configured to sequentially store the first image and the second image;

an image conversion unit configured to project a coordinate of each pixel of the predetermined plane from the first image onto a corresponding coordinate of a corresponding pixel of the second image, the projected first image being a third image;

a feature conversion unit configured to emphasize a plurality of edges included in the first image, the second image and the third image, wherein the edge-emphasized first image, the edge-emphasized second image and the edge-emphasized third image being respectively a first feature conversion image, a second feature conversion image and a third feature conversion image;

a similarity calculation unit configured to set a plurality of areas on the predetermined plane from the second feature conversion image, to calculate a first similarity between each of the areas of the second feature conversion image and a corresponding one of the areas of the first feature conversion image, and to calculate a second similarity between each of the areas of the second feature conversion image and a corresponding one of the areas of the third feature conversion image; and

an obstacle decision unit configures to detect an obstacle based on the first similarity and the second similarity.

3. The object detection apparatus according to claim 1 or 2,

wherein the first camera is one of a left side camera and a right side camera each being mounted on a vehicle, and

wherein the second camera is the other of the left side camera and the right side camera.

4. The object detection apparatus according to claim 3,

wherein the first image and the second image include a front scene of the vehicle along a moving direction of the vehicle, and

wherein the predetermined plane is a road surface on which the vehicle is moving.

5. The object detection apparatus according to claim 4,

further comprising a lane detection unit configured to respectively detect two lines from the first image and the second image, and to respectively detect the road surface from the first image and the second image by using the detection result of the two lines.

6. The object detection apparatus according to claim 1,

wherein said image conversion unit converts the coordinate of each pixel of the predetermined plane from the first image based on a view position of the second camera.

7. The object detection apparatus according to claim 6,

wherein said feature conversion unit respectively multiplies each pixel value of the first image, the second image and the third image by a coefficient, and corrects each multiplied pixel value to a predetermined value if the multiplied pixel value is above the predetermined value.

10. The object detection apparatus according to claim 1 or 2,

wherein said similarity calculation unit sets each of the areas of which bases are different pixel lines along a horizontal direction on the predetermined plane of the second feature conversion image.

11. The object detection apparatus according to claim 10,

wherein said similarity calculation unit determines the corresponding area of the first feature conversion image by inversely projecting each of the areas of the second feature conversion image onto the first feature conversion image, the inverse-projection representing an inverse-conversion of the projection by said image conversion unit.

12. The object detection apparatus according to claim 11,

wherein said similarity calculation unit determines the corresponding area of the third feature conversion image by setting the same position of each of the areas of the second feature conversion image on the third feature conversion image.

13. The object detection apparatus according to claim 12,

wherein said obstacle decision unit calculates a difference between the first similarity and the second similarity, and selects one of the areas based on the difference as an obstacle area.

14. The object detection apparatus according to claim 13,

wherein said obstacle decision unit selects one of the areas having the largest difference above a threshold as the obstacle area.

15. The object detection apparatus according to claim 1 or 2,

wherein said similarity calculation unit respectively reduces the first feature conversion image, the second feature conversion image and the third feature conversion image by a predetermined reduction ratio, sets a plurality of areas on the predetermined plane from the reduced second feature conversion image, calculates a first similarity between each of the areas of the reduced second feature conversion image and a corresponding one of the areas of the reduced first feature conversion image, and calculates a second similarity between each of the areas of the reduced second feature conversion image and a corresponding one of the areas of the reduced third feature conversion image

16. The object detection apparatus according to claim 1 or 2,

wherein said similarity calculation unit shifts the corresponding area on the first feature conversion image around a position of each of the areas of the second feature conversion image, respectively calculates a similarity between each shifted area of the first feature conversion image and each of the areas of the second feature conversion image, selects one similarity having the maximum as the first similarity, shifts the corresponding area on the third feature conversion image around a position of each of the areas of the second feature conversion image, respectively calculates a similarity between each shifted area of the third feature conversion image and each of the areas of the second feature conversion image, selects one similarity having the maximum as the second similarity degree.

2. An obstacle detection apparatus, comprising:

a first camera inputting a first image including a predetermined plane and a second camera inputting a second image including the predetermined plane;

an image memory configured to sequentially store the first image and the second image;

a feature conversion unit configured to emphasize a plurality of edges included in the first image and the second image, wherein the edge-emphasized first image and the edge-emphasized second image being respectively a first feature conversion image and a second feature conversion image;

an image conversion unit configured to project a coordinate of each pixel of the predetermined plane from the first feature conversion image onto a corresponding coordinate of a corresponding pixel of the second feature conversion image, the projected first feature conversion image being a third feature conversion image;

a similarity calculation unit configured to set a plurality of areas on the predetermined plane from the second feature conversion image, to calculate a first similarity between each of the areas of the second feature conversion image and a corresponding one of the areas of the first feature conversion image, and to calculate a second similarity between each of the areas of the second feature conversion image and a corresponding one of the areas of the third feature conversion image; and

an obstacle decision unit configures to detect an obstacle based on the first similarity and the second similarity.

8. The object detection apparatus according to claim 2,

wherein said feature conversion unit respectively multiplies each pixel value of the first image and the second image by a coefficient, and corrects each multiplied pixel value to a predetermined value if the multiplied pixel value is above the predetermined value.

9. The object detection apparatus according to claim 8,

wherein said image conversion unit converts the coordinate of each pixel of the predetermined plane of the first feature conversion image based on a view position of the second camera.

17. A method for detecting an obstacle, comprising:

inputting a first image including a predetermined plane through a first camera;

inputting a second image including the predetermined plane through a second camera;

projecting a coordinate of each pixel of the predetermined plane from the first image onto a corresponding coordinate of a corresponding pixel of the second image, the projected first image being a third image;

emphasizing a plurality of edges included in the first image, the second image and the third image, wherein the edge-emphasized first image, the edge-emphasized second image and the edge-emphasized third image being respectively a first feature conversion image, a second feature conversion image and a third feature conversion image;

setting a plurality of areas on the predetermined plane of the second feature conversion image;

calculating a first similarity between each of the areas of the second feature conversion image and a corresponding one of the areas of the first feature conversion image;

calculating a second similarity between each of the areas of the second feature conversion image and a corresponding one of the areas of the third feature conversion image; and

detecting an obstacle based on the first similarity and the second similarity.

18. A method for detecting an obstacle, comprising:

inputting a first image including a predetermined plane through a first camera;

inputting a second image including the predetermined plane through a second camera;

emphasizing a plurality of edges included in the first image and the second image, wherein the edge-emphasized first image and the edge-emphasized second image being respectively a first feature conversion image and a second feature conversion image;

projecting a coordinate of each pixel of the predetermined plane from the first feature conversion image onto a corresponding coordinate of a corresponding pixel of the second feature conversion image, the projected first feature conversion image being a third feature conversion image;

setting a plurality of areas on the predetermined plane of the second feature conversion image;

calculating a first similarity between each of the areas of the second feature conversion image and a corresponding one of the areas of the first feature conversion image;

calculating a second similarity between each of the areas of the second feature conversion image and a corresponding one of the areas of the third feature conversion image; and

detecting the obstacle based on the first similarity and the second similarity.

19. A computer program product, comprising:

a computer readable program code embodied in said product for causing a computer to detect an obstacle, said computer readable program code comprising;

a first program code to input a first image including a predetermined plane through a first camera and to input a second image including the predetermined plane through a second camera;

a second program code to project a coordinate of each pixel of the predetermined plane from the first image onto a corresponding coordinate of a corresponding pixel of the second image, the projected first image being a third image;

a third program code to emphasize a plurality of edges included in the first image, the second image and the third image, wherein the edge-emphasized first image, the edge-emphasized second image and the edge-emphasized third image being respectively a first feature conversion image, a second feature conversion image and a third feature conversion image;

a fourth program code to set a plurality of areas on the predetermined plane of the second feature conversion image;

a fifth program code to calculate a first similarity between each of the areas of the second feature conversion image and a corresponding one of the areas of the first feature conversion image;

a sixth program code to calculate a second similarity between each of the areas of the second feature conversion image and a corresponding one of the areas of the third feature conversion image; and

a seventh program code to detect the obstacle based on the first similarity and the second similarity.

20. A computer program product, comprising:

a computer readable program code embodied in said product for causing a computer to detect an obstacle, said computer readable program code comprising;

a first program code to input a first image including a predetermined plane through a first camera and to input a second image including the predetermined plane through a second camera;

a second program code to emphasize a plurality of edges included in the first image and the second image, wherein the edge-emphasized first image and the edge-emphasized second image being respectively a first feature conversion image and a second feature conversion image;

a third program code to project a coordinate of each pixel of the predetermined plane from the first feature conversion image onto a corresponding coordinate of a corresponding pixel of the second feature conversion image, the projected first feature conversion image being a third feature conversion image;

a fourth program code to set a plurality of areas on the predetermined plane of the second feature conversion image;

a fifth program code to calculate a first similarity between each of the areas of the second feature conversion image and a corresponding one of the areas of the first feature conversion image;

a sixth program code to calculate a second similarity between each of the areas of the second feature conversion image and a corresponding one of the areas of the third feature conversion image; and

a seventh program code to detect the obstacle based on the first similarity and the second similarity.