Method and apparatus for obstacle avoidance with camera vision

• US 20060111841A1
• Filed: 10/27/2005
• Published: 05/25/2006
• Est. Priority Date: 11/19/2004
• Status: Abandoned Application

• Current Assignee: Jiun-Yuan Tseng
• Original Assignee: Jiun-Yuan Tseng
• Inventors: Tseng, Jiun-Yuan
• Application Number:
  US11/260,723
• Publication Number:
  US 20060111841A1
• Time in Patent Office: Days
• US Class Current: 701/301
• CPC Class Codes: B60R 2300/307 virtually distinguishing relevant parts of a scene from the background of the scene
• CPC Class Codes: B60R 2300/8093 for obstacle warning
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads

Abstract

The present invention relates to a method and an apparatus of operating an obstacle avoidance system with camera vision. The invention is used during both day and night, and provides a strategy of obstacle avoidance without complicated fuzzy inference for safe driving. The method includes the following steps: analyzing plural images of an obstacle, positioning an image sensor, providing an obstacle recognizing flow, obtaining an absolute velocity of a system carrier, obtaining a relative velocity and a relative distance of the system carrier with respect to the obstacle, and providing a strategy of obstacle avoidance.

Claims

1. A method for obstacle avoidance with camera vision, which is applied in a system carrier carrying an image sensor, comprising the steps of:

capturing and analyzing plural images of an obstacle;

positioning the image sensor;

performing an obstacle recognition flow;

obtaining an absolute velocity of the system carrier;

obtaining a relative velocity and a relative distance of the system carrier with respect to the obstacle; and

performing a strategy of obstacle avoidance.

2. The method for obstacle avoidance with camera vision of claim 1, wherein the step of positioning the image sensor is used to obtain the depression angle of the image sensor, the distance from the image sensor to the ground, the focus of the image sensor and the interval of pixels on the image plane.

3. The method for obstacle avoidance with camera vision of claim 2, wherein the step of obtaining the depression angle of the image sensor and the distance from the image sensor to the ground comprises the steps of:

scanning horizontally the images of the obstacle from bottom to top with an interval;

recognizing a character point having the character of sidelines of the road;

recognizing two first points on a first character line segment containing the character point;

scanning horizontally through the two first points to obtain two horizontal lines intersecting a second character line segment at two second points;

recognizing an intersection point of a line formed by the two first points and a line formed by the two second points;

obtaining a depression angle of the image sensor; and

obtaining a distance from the image sensor to the ground.

4. The method for obstacle avoidance with camera vision of claim 3, wherein the steps of obtaining the depression angle of the image sensor and the distance from the image sensor to the ground comprises the steps of:

calculating a focus of the image sensor; and

calculating an interval of pixels on the image plane.

5. The method for obstacle avoidance with camera vision of claim 3, wherein the depression angle of the image sensor is calculated according to the interval of pixels on the image plane, the focus of the image sensor, the intersection point and a half of the vertical length of the images.

6. The method for obstacle avoidance with camera vision of claim 3, wherein the distance from the image sensor to the ground is calculated according to the depression angle of the image sensor, the distance from one of the two horizontal lines to the image sensor and the relative distance from the other horizontal line to the image sensor.

7. The method for obstacle avoidance with camera vision of claim 3, wherein the depression angle of the image sensor is determined by the following equation:

${\mathrm{\theta <br><br>}}_1={\tan }^{-1}<br><br>\left(\frac{\mathrm{\Delta <br><br>}<br><br>\text{\hspace{1em}<br><br>}<br><br>p_l*\left(c-y_l\right)}{f}\right),$

wherein θ

₁ is the depression angle of the image sensor, Δ

p_l is the interval of pixels on the image plane, c is a half of the vertical length of the images, y₁ is the position of the intersection point and ƒ

is the focus of the image sensor.

8. The method for obstacle avoidance with camera vision of claim 3, wherein the distance from the image sensor to the ground is determined by the following equation:

$H_c=\frac{C_1}{\left(\frac{1}{\tan <br><br>\left({\mathrm{\theta <br><br>}}_1+{\mathrm{\theta <br><br>}}_2\right)}-\frac{1}{\tan <br><br>\left({\mathrm{\theta <br><br>}}_1+{\mathrm{\theta <br><br>}}_2^{\mathrm{\prime <br><br>}}\right)}\right)}$

wherein H_c is the distance from the image sensor to the ground, C₁ is the length of a line segment on the road, θ

₁ is the depression angle of the image sensor, θ

₂ and θ

₂′

satisfy

$\mathrm{La}=\frac{H_c}{\tan <br><br>\left({\mathrm{\theta <br><br>}}_1+{\mathrm{\theta <br><br>}}_2\right)}<br><br>\text{\hspace{1em}<br><br>}<br><br>\mathrm{and}<br><br>\text{\hspace{1em}<br><br>}<br><br>{\mathrm{La}}^{\mathrm{\prime <br><br>}}=\frac{H_c}{\tan <br><br>\left({\mathrm{\theta <br><br>}}_1+{\mathrm{\theta <br><br>}}_2^{\mathrm{\prime <br><br>}}\right)},$

where La is the distance from one of the two horizontal lines to the image sensor and La′

is the distance from the other horizontal line to the image sensor.

9. The method for obstacle avoidance with camera vision of claim 3, the focus of the image sensor and the distance from the image sensor to the ground are determined by the following equations:

$H_c\times <br><br>\left(\frac{\tan <br><br>\left({\mathrm{\theta <br><br>}}_1+{\mathrm{\theta <br><br>}}_2^{\mathrm{\prime <br><br>}}\right)-\tan <br><br>\left({\mathrm{\theta <br><br>}}_1+{\mathrm{\theta <br><br>}}_2\right)}{\tan <br><br>\left({\mathrm{\theta <br><br>}}_1+{\mathrm{\theta <br><br>}}_2\right)\times <br><br>\tan <br><br>\left({\mathrm{\theta <br><br>}}_1+{\mathrm{\theta <br><br>}}_2^{\mathrm{\prime <br><br>}}\right)}\right)=C_1,<br><br>H_c\times <br><br>\left(\frac{\tan <br><br>\left({\mathrm{\theta <br><br>}}_1+{\mathrm{\theta <br><br>}}_2^{\mathrm{\prime <br><br>\prime <br><br>}}\right)-\tan <br><br>\left({\mathrm{\theta <br><br>}}_1+{\mathrm{\theta <br><br>}}_2\right)}{\tan <br><br>\left({\mathrm{\theta <br><br>}}_1+{\mathrm{\theta <br><br>}}_2\right)\times <br><br>\tan <br><br>\left({\mathrm{\theta <br><br>}}_1+{\mathrm{\theta <br><br>}}_2^{\mathrm{\prime <br><br>\prime <br><br>}}\right)}\right)=C_{10}$

wherein C₁ is the length of a line segment on the road, C₁₀ is an interval of line segments on the road, H_c is the distance from the image sensor to the ground, θ

₁ is the depression angle of the image sensor;

H_c, θ

₁, θ

₂, θ

₂′ and

θ

₂″

are functions of f and Δ

p₁, f is the focus of the image sensor, Δ

p_l is the interval of pixels on the image plane, θ

₂ and θ

₂′

satisfy

$\mathrm{La}=\frac{H_c}{\tan <br><br>\left({\mathrm{\theta <br><br>}}_1+{\mathrm{\theta <br><br>}}_2\right)}<br><br>\text{\hspace{1em}<br><br>}<br><br>\mathrm{and}<br><br>\text{\hspace{1em}<br><br>}<br><br>{\mathrm{La}}^{\mathrm{\prime <br><br>}}=\frac{H_c}{\tan <br><br>\left({\mathrm{\theta <br><br>}}_1+{\mathrm{\theta <br><br>}}_2^{\mathrm{\prime <br><br>}}\right)},$

where La is the distance from one of the two horizontal lines to the image sensor and La′

is the distance from the other horizontal line to the image sensor.

10. The method for obstacle avoidance with camera vision of claim 1, wherein the step of performing an obstacle recognition flow comprises the steps of:

setting a scan mode that is selected from the group of a single line scan mode, a zigzag scan mode, a three-line scan mode, a five-line scan mode, a turn-type scan mode and a transverse scan mode;

providing a border point recognition;

setting a scan type that is a detective type or a gradual type;

providing two Boolean variables regarding a dark-color character of the obstacle, and a brightness decay character of the projected light or a reflected light from the obstacle; and

recognizing the obstacle type.

11. The method for obstacle avoidance with camera vision of claim 10, wherein the step of providing the border point recognition comprises the steps of:

calculating a Euclidean distance of pixel values between a pixel and its adjacent pixel; and

treating the pixel as the border point if the Euclidean distance is larger than a critical constant.

12. The method for obstacle avoidance with camera vision of claim 10, wherein the Boolean variable regarding the dark-color character of the obstacle is true, if

$\frac{N_{\mathrm{dark\_pixel}}}{l_{\mathrm{dw}}}\ge$

C 4

is true, where C₄ is a constant, l_dw is the length of the detective interval, and N<sub>dark—

pixel</sub> is the amount of the pixels satisfying the dark-color character.

13. The method for obstacle avoidance with camera vision of claim 12, wherein the criterion of the dark-color character is given as:

R≦

C₆×

RR for the color images and Gray≦

C₇×

Gray_r for gray-scale images, wherein R denotes the red pixel value and RR denotes the average pixel value of red, green and blue pixel of the road for color images;

Gray denotes the gray pixel value for gray-scale images and Gray_r denotes the gray pixel value of the road;

C₆ and C₇ are constants.

14. The method for obstacle avoidance with camera vision of claim 13, wherein when the relative speed of the system carrier with respect to the obstacle does not equal the absolute speed of the system carrier, the item C₆×

RR is replaced with the red color value of a pixel group and the item C₇×

Gray is replaced with the gray level color of the pixel group.

15. The method for obstacle avoidance with camera vision of claim 10, wherein the Boolean variable regarding the brightness decay character of the projected light or the reflected light from the obstacle is true, if R≧

C₈ or Gray≧

C₉ is true, where C₈ and C₉ are critical constants, R is the red pixel value in color images, Gray is the gray pixel value in gray-scale images.

16. The method for obstacle avoidance with camera vision of claim 10, further comprising the step of recognizing the obstacle and weather at rainy night, which is performed according to the character of the blue pixel value of the blue light that is emitted from an enhanced blue light installed on the system carrier and then reflected from the obstacle.

17. The method for obstacle avoidance with camera vision of claim 16, wherein the Boolean variable regarding the brightness decay character of the projected light or the reflected light from the obstacle is true, if B≧

C₁₁ or Gray≧

C₁₂ is true, where C₁₁ and C₁₂ are critical constants, B is the blue pixel value in color images, Gray is the gray pixel value in gray-scale images.

18. The method for obstacle avoidance with camera vision of claim 10, further comprising the step of switching between a day recognition and a nigh recognition, wherein the day recognition operates according to the Boolean variable regarding the dark-color character of the obstacle, the night recognition operates according to the Boolean variable regarding the brightness decay character of the projected light or the reflected light from the obstacle, and the time of switching is set in an operation unit in the system carrier.

19. The method for obstacle avoidance with camera vision of claim 10, wherein if the Boolean variable regarding the dark-color character of the obstacle is true, the obstacle is identified as an object with dark-color pixels below.

20. The method for obstacle avoidance with camera vision of claim 10, wherein if the Boolean variable regarding the brightness decay character of the projected light or the reflected light from the obstacle is true, then the obstacle is identified as a three-dimensional object.

21. The method for obstacle avoidance with camera vision of claim 10, further comprising the step of switching automatically between the high beam and the low beam, which operates when the distance between the system carrier and the obstacle in the oncoming way is below a specific distance.

22. The method for obstacle avoidance with camera vision of claim 10, further comprising the step of adjusting automatically the brightness of the headlights, which operates according to the lightness of the sky, determined by the average of the pixel values of the group of pixels of the road.

23. The method for obstacle avoidance with camera vision of claim 1, wherein the step of obtaining the absolute velocity of the system carrier comprises the steps of:

recognizing a first position of an end point of a character line segment in a first image;

recognizing a second position of the end point of the character line segment in a second image;

dividing the distance between the first position and the second position by the time interval between capturing the first and the second images, which belong to the plural images of the obstacle, with the first image captured earlier than the second image.

24. The method for obstacle avoidance with camera vision of claim 1, wherein the step of performing the strategy of obstacle avoidance comprises the steps of:

providing an equivalent velocity, which is the larger one of the absolute velocity and the relative velocity;

providing a safe distance determined by the equivalent velocity;

providing a safe coefficient, which is the ratio of the relative distance to the safe distance and is between zero and one;

providing an alarm signal, which is defined by subtracting the safe coefficient from one;

generating light, sound or vibration to alert a driver of the system carrier or surrounding persons based on the alarm signal;

capturing and displaying a frame of the obstacle in the images;

providing a sub absolute velocity, which is the product of the safe coefficient and the current absolute velocity of the system carrier; and

performing an audio/video recording.

25. The method for obstacle avoidance with camera vision of claim 24, wherein the audio/video recording is performed when the safe coefficient is below an empirical value.

26. The method for obstacle avoidance with camera vision of claim 1, wherein the absolute velocity is obtained directly from a speedometer of the system carrier.

27. The method for obstacle avoidance with camera vision of claim 1, wherein the image sensor is selected from the group of a CCD camera, a CMOS device camera, a digital camera, a single-line scanner and a camera installed in a handheld communication equipment.

28. An apparatus for obstacle avoidance with camera vision, which is applied in a system carrier, comprising:

an image sensor, which captures plural images of an obstacle and is used to recognize the obstacle; and

an operation unit, which performs the following functions;

(a) analyzing the plural images;

(b) performing an obstacle recognition to determine if the obstacle exists according to the result of analyzing the plural images; and

(c) performing a strategy of obstacle avoidance.

29. The apparatus for obstacle avoidance with camera vision of claim 28, further comprising an alarm, which emits light and sound or generates vibration if the obstacle exists.

30. The apparatus for obstacle avoidance with camera vision of claim 28, wherein the image sensor is selected from the group of a CCD camera, a CMOS device camera, a digital camera, a single-line scanner and a camera installed in a handheld communication equipment.