Apparatus and Method For Automatically Detecting Objects

• US 20080273750A1
• Filed: 11/29/2005
• Published: 11/06/2008
• Est. Priority Date: 11/30/2004
• Status: Active Grant

• Current Assignee: Nissan Motor Co., Ltd.
• Original Assignee: Nissan Motor Co., Ltd.
• Inventors: Fujimoto, Kazumi
• Granted Patent: US 7,747,039 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30256 Lane; Road marking
• CPC Class Codes: G06T 7/12 Edge-based segmentation
• CPC Class Codes: G06T 7/215 Motion-based segmentation
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands

Abstract

A device automatically detects boundary lines on the road from an image captured by a camera mounted on the vehicle. The device includes a controller that performs image processing on the image to compute the velocity information for each pixel in the image, and, on the basis of the computed velocity information for each pixel in the image, extracts the pixels that contain velocity information, detects the oblique lines formed by the extracted pixels, and detects the boundary lines on the road on the basis of the detected oblique lines.

Claims

1. An apparatus for detecting objects in one or more images captured by an image pickup device mounted on a vehicle, comprising:

(a) memory on which is stored pixels of at least one image captured by the image pickup device; and

(b) a controller operatively coupled to the memory and adapted to compute velocity information for each pixel in the image;

extract those pixels having a velocity component based on the velocity information;

detect oblique lines composed of those extracted pixels having a velocity component; and

generate a signal indicative of a road boundary in the image based on the oblique lines.

2. The apparatus of claim 1, wherein the controller is further adapted to judge that oblique lines in the image are road boundaries when the vehicle is traveling and the oblique lines are positioned on the image with bilateral symmetry and different velocity directions.

3. The apparatus of claim 1, wherein the controller is further adapted to judge that oblique lines in the image are road boundaries when the slopes of the oblique lines decrease from the center of the image toward the outside.

4. The apparatus of claim 1, wherein the controller is further adapted to detect a change point where the velocity direction of an oblique line changes, and to judge that the change point as the balance point of the pitch generated by the movement of the vehicle.

5. The apparatus of claim 1, wherein the controller is further adapted to identify a moving object that is approaching the predicted path of the vehicle by grouping the pixels having velocity information pointed from the side toward the predicted movement path of the vehicle from among the extracted pixels;

and to generate a collision danger signal indicative of the risk of collision between the vehicle and the moving object.

6. The apparatus of claim 5, wherein the controller is further adapted to transform into a real space road model the oblique lines judged as the road boundary and the moving object judged to be at the balance point and to determine the risk of collision between the vehicle and the moving object based on the relative positional relationship between the road boundary and the moving object as established in the real space road model.

7. The apparatus of claim 5, wherein the controller is further adapted to generate the collision danger signal at one of a plurality of values corresponding to collision risk levels.

8. The apparatus of claim 5, further comprising an audio alert operatively coupled to the controller and activated by the collision danger signal.

9. The apparatus of claim 5 further comprising an automatic breaking device operatively coupled to the controller and activated by the collision danger signal.

10. A vehicle, comprising:

(a) a camera mounted including an image-pickup element generating as output a plurality of images;

(b) memory on which is stored the plurality of images generated by the image pickup element; and

(c) a controller operatively coupled to the memory and adapted to compute velocity information for each pixel in at least one of the images;

extract those pixels having a velocity component based on the velocity information;

detect oblique lines composed of those extracted pixels having a velocity component; and

generate a signal indicative of a road boundary in the image based on the oblique lines.

11. The vehicle of claim 10, wherein the controller is further adapted to judge that oblique lines in the image are road boundaries when the vehicle is traveling and the oblique lines are positioned on the image with bilateral symmetry and with different velocity directions.

12. The vehicle of claim 10, wherein the controller is further adapted to judge that oblique lines in the image are road boundaries when the slopes of the oblique lines decrease from the center of the image toward the outside.

13. The vehicle of claim 10, wherein the controller is further adapted to detect a change point where the velocity direction of an oblique line changes, and to judge that the change point as the balance point of the pitch generated by the movement of the vehicle.

14. The vehicle of claim 10, wherein the controller is further adapted to identify a moving object that is approaching the predicted path of the vehicle by grouping the pixels having velocity information pointed from the side toward the predicted movement path of the vehicle from among the extracted pixels;

and to generate a collision danger signal indicative of the risk of collision between the vehicle and the moving object.

15. The vehicle of claim 14, wherein the controller is further adapted to transform into a real space road model the oblique lines judged as the road boundary and the moving object, and to determine the risk of collision between the vehicle and the moving object based on the relative positional relationship between the road boundary and the moving object as established in the real space road model.

16. The vehicle of claim 14, wherein the controller is further adapted to generate the collision danger signal at one of a plurality of values corresponding to collision risk levels.

17. The vehicle of claim 14, further comprising an audio alert operatively coupled to the controller and activated by the collision danger signal.

18. The vehicle of claim 14, further comprising an automatic breaking device operatively coupled to the controller and activated by the collision danger signal.

19. An apparatus for detecting objects in one or more images captured by an image pickup device mounted on a vehicle, comprising:

(a) velocity information computing means for processing the image to compute velocity information for each pixel in the image;

(b) pixel extracting means for extracting pixels having a velocity component on the basis of the velocity information of the pixels of the images computed by the velocity information computing means;

(c) oblique line detecting means for detecting oblique lines made of pixels having a velocity component and extracted by the pixel extracting means; and

(d) boundary line detecting means for detecting at least one boundary line on the road present in the image on the basis of the oblique line detected by the oblique line detecting means.

20. The apparatus of claim 19, wherein the boundary line detecting means judges that the oblique lines are the road boundaries when the vehicle is traveling and the oblique lines detected by the oblique line detecting means are positioned with bilateral symmetry on the images and different velocity directions.

21. The apparatus of claim 19, wherein when multiple oblique lines are detected by the oblique line detecting means, the boundary line detecting means detects the oblique lines as the boundary lines on the road when the slopes of the oblique lines decrease from the center of the image in front of the vehicle toward the outside.

22. The apparatus of claim 19, further comprising balance point judgment means for detecting a change point where the velocity direction of an oblique line detected by the oblique line detecting means changes, and judging that the change point is the balance point of the pitch generated by the movement of the vehicle.

23. The apparatus of claim 22, further comprising:

object detecting means for grouping the pixels having velocity information pointed from the side toward the predicted movement path of the vehicle from among the pixels extracted by the pixel extracting means, and detecting moving objects that approach the predicted movement path ahead of the vehicle;

transformation means for transforming the oblique lines judged as the boundary lines on the road by the boundary line detecting means and the moving object detected by the object detecting means judged by the balance point judgment means to be at the balance point, into a real space road model; and

degree of collision danger judgment means for accessing the degree of collision danger between the vehicle and the moving body on the basis of the relative positional relationship between the boundary lines on the road together with the moving object established in the real space road model by the transformation means.

24. The apparatus of claim 23, wherein:

the degree of collision danger judgment means classifies the degree of collision danger between the vehicle and the moving object into multiple risk levels and makes a judgment according to the degree of risk; and

further comprising risk avoidance means for controlling the vehicle to avoid a collision between the vehicle and the moving object according to the risk level assessed by the degree of collision danger judgment means.

25. A method for detecting objects in an image captured of the pathway of a vehicle, comprising:

(a) computing the velocity component for each pixel in the image;

(b) extracting from the image those pixels having a velocity component;

(c) detecting oblique lines made of extracted pixels having a velocity component; and

(d) detecting one or more boundary lines on the vehicle pathway in the image from the oblique lines.

26. The method of claim 25, wherein detecting the boundary line comprises judging that the oblique lines are the road boundaries when the vehicle is traveling and the oblique lines detected by the oblique line detecting means are positioned with bilateral symmetry on the images and their velocity directions are different from each other.

27. The method of claim 25, wherein multiple oblique lines are detected as the boundary lines on the vehicle pathway when the slopes of the oblique lines decrease from the center of the image in front of the vehicle toward the outside.

28. The method of claim 25, further comprising:

detecting a change point where the velocity direction of an oblique line detected by the oblique line detecting means changes, and judging that the change point is the balance point of the pitch generated by the movement of the vehicle.

29. The method of claim 25, further comprising:

grouping the pixels having velocity information pointed from the side toward the predicted movement path of the vehicle from among the extracted pixels, and detecting moving objects that approach the predicted movement path ahead of the vehicle;

transforming into a real space road model the oblique lines judged as the boundary lines on the road and the moving object;

accessing the degree of collision danger between the vehicle and the moving object on the basis of the relative positional relationship between the boundary lines on the road together with the moving object established in the real space road model.

30. The method of claim 29, further comprising:

classifying the degree of collision danger between the vehicle and the moving object into one of a plurality of risk levels and making collision danger judgment according to the classified risk level; and

generating a signal to control the vehicle to avoid a collision between the vehicle and the moving object according to the collision danger judgment.