Detecting device and method to detect an object based on a road boundary

• US 7,437,243 B2
• Filed: 03/18/2006
• Issued: 10/14/2008
• Est. Priority Date: 03/22/2005
• Status: Expired due to Fees

• Current Assignee: Nissan Motor Co., Ltd.
• Original Assignee: Nissan Motor Co., Ltd.
• Inventors: Fujimoto, Kazumi
• Primary Examiner(s): Hellner; Mark
• Application Number:
  US11/378,540
• Publication Number:
  US 20060217886A1
• Time in Patent Office: 941 Days
• Field of Search: 701/300, 382/107, 340/436
• US Class Current: 701/300
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
• CPC Class Codes: G08B 13/19647 Systems specially adapted for intrusion detection in or around a vehicle
• CPC Class Codes: G08B 13/19652 Systems using zones in a single scene defined for different treatment, e.g. outer zone gives pre-alarm, inner zone gives alarm
• CPC Class Codes: G08G 1/16 Anti-collision systems road vehicle drive control systems for predicting or avoiding probable or impending collision otherwise than by control of a particular sub-unit B60W30/08
• CPC Class Codes: G08G 1/165 for passive traffic, e.g. including static obstacles, trees
• CPC Class Codes: G08G 1/166 for active traffic, e.g. moving vehicles, pedestrians, bikes
• CPC Class Codes: G08G 1/167 Driving aids for lane monitoring, lane changing, e.g. blind spot detection

Abstract

A detecting device and corresponding method for detecting an object in an image where the image is captured by an image pickup device mounted on a vehicle. A controller is coupled to the image pickup device. The controller extracts pixels of the image captured by the image pickup device, calculates a side direction velocity component based upon the extracted pixels, detects a road boundary based upon the calculated side direction velocity component, and detects a moving direction of the object based upon a relationship between the road boundary and the object.

Claims

1. An apparatus for detecting an object in at least one image, comprising:

an image pickup device mounted on a vehicle, the image pickup device capturing an image; and

a controller coupled to the image pickup device;

wherein the controller is operable to extract pixels of the image captured by the image pickup device, to calculate a side direction velocity component based upon the extracted pixels, to detect a road boundary based upon the calculated side direction velocity component, and to detect a moving direction of the object based upon a relationship between the road boundary and the object.

2. The apparatus according to claim 1 wherein the controller is operable to detect the distance between the road boundary and the object, and to detect the moving direction of the object from a change in distance between the road boundary and the object.

3. The apparatus according to claim 1 wherein the controller is operable to divide an area outside the road boundary into object-detection regions of a prescribed size, and to separate an object present in the area from a background based on velocity dispersion in each object-detection region.

4. The apparatus according to claim 1 wherein the controller is operable to divide an area outside the road boundary into object-detection regions of a prescribed size, and to separate an object preset in the area and a background based on the difference in the velocity of edges between adjacent object-detection regions.

5. The apparatus according to claim 1 wherein the controller is operable to specify a bottom edge position and a top edge position of the object based on the lateral speed of movement of an edge in a velocity image, the object being located within the road boundary.

6. The apparatus according to claim 1 wherein the controller is operable to detect an object straddling the road boundary by specifying a top edge position and a bottom edge position of the object straddling the road boundary, the top edge and the bottom edge specified based on the lateral speed of movement of an edge in an image.

7. An apparatus for detecting objects in an image obtained by an image pick-up device mounted on a vehicle, comprising:

edge-extraction means for extracting edges of at least one object present in multiple image frames;

road boundary detection means for detecting a road boundary in the image based on respective lateral speeds of movement of at least some of the extracted edges; and

direction-of-movement determination means for determining a direction of movement of an object based on the road boundary and respective lateral speeds of movement of others of the extracted edges.

8. The apparatus according to claim 7, further comprising:

edge velocity calculation means for calculating the respective lateral speeds of movement of the extracted edges based upon at least two image frames.

9. The apparatus according to claim 7 wherein the direction-of-movement determination means further comprises means for monitoring a change in distance in at least two image frames between the object and the road boundary, the change in distance indicating the direction of movement of the object.

10. The apparatus according to claim 7, further comprising:

means for dividing an area outside the road boundary into object-detection regions of a prescribed size; and

means for separating the object from a background based on velocity dispersion in each object-detection region.

11. The apparatus according to claim 7, further comprising:

means for dividing an area outside the road boundary into object-detection regions of a prescribed size; and

wherein the direction-of movement determination means further includes means for separating the object from a background based on a difference in the lateral speed of movement of extracted edges between adjacent object-detection regions.

12. The apparatus according to claim 7, further comprising:

means for scanning a velocity image including a plurality of pixels and a lateral speed of movement associated with respective ones of the plurality of pixels;

means for defining a bottom edge position of the object by comparing a lateral speed of movement of a pixel to a prescribed velocity; and

means for defining a top edge position of the object based upon a grouping of pixels having the same lateral speed of movement of the pixel.

13. The apparatus according to claim 7 wherein the direction-of-movement determination means further includes selection means for selecting one method of determining the direction of movement when the object is outside the road boundary and for selecting another method of determining the direction of movement when the object is one of straddling the road boundary and inside the road boundary.

14. A method of detecting objects in an image obtained by an image pick-up device mounted on a vehicle, comprising:

extracting edges of at least one object present in multiple image frames;

detecting a road boundary in the image based on respective lateral speeds of movement of at least some of the extracted edges; and

determining a direction of movement of an object based on the road boundary and respective lateral speeds of movement of others of the extracted edges.

15. The method according to claim 14, further comprising:

calculating a lateral speed of movement in the image of the extracted edges based upon at least two image frames.

16. The method according to claim 14 wherein determining a direction of movement further comprises monitoring a change in distance in at least two image frames between the object and the road boundary, the change in distance indicating the direction of movement of the object.

17. The method according to claim 14, further comprising:

dividing an area outside the road boundary into object-detection regions of a prescribed size; and

separating the object from a background based on velocity dispersion in each object-detection region.

18. The method according to claim 14, further comprising:

dividing an area outside the road boundary into object-detection regions of a prescribed size; and

separating the object from a background based on a difference in the lateral speed of movement of extracted edges between adjacent object-detection regions.

19. The method according to claim 14, further comprising:

creating a velocity image including a plurality of pixels and a lateral speed of movement associated with respective ones of the plurality of pixels;

scanning the velocity image using a scanning area;

defining a bottom edge position of the object by comparing a lateral speed of movement of a pixel within the scanning area to a prescribed velocity; and

defining a top edge position of the object based upon a grouping of pixels having the same lateral speed of movement of the pixel.

20. The method according to claim 14, further comprising:

selecting one method of determining the direction of movement when the object is outside the road boundary; and

selecting another method of determining the direction of movement when the object is one of straddling the road boundary and inside the road boundary.

21. A method of detecting objects in an image obtained by an image pick-up device mounted on a vehicle, comprising:

extracting edges of at least one object present in multiple image frames;

determining an existence of a road boundary in the image based on respective lateral speeds of movement of at least some of the extracted edges;

determining a direction of movement of an object according to a first method when the object is outside the road boundary; and

determining the direction of movement of the object according to a second method when one of the road boundary does not exist and at least a portion of the object is inside the road boundary.