Object detecting system and object detecting method

• US 20070127778A1
• Filed: 11/29/2006
• Published: 06/07/2007
• Est. Priority Date: 12/07/2005
• Status: Active Grant

• Current Assignee: Nissan Motor Co., Ltd.
• Original Assignee: Nissan Motor Co., Ltd.
• Inventors: Fujimoto, Kazumi
• Granted Patent: US 7,899,211 B2
• Time in Patent Office: Days
• US Class Current: 382/104
• CPC Class Codes: G06T 2207/30261 Obstacle
• CPC Class Codes: G06T 7/12 Edge-based segmentation
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road

Abstract

An object detecting system includes an imaging device disposed on a vehicle and a controller that receives a two-dimensional image, which represents a scene ahead of the vehicle, from the imaging device. The controller includes a feature extraction unit, a motion-information calculator, a coordinate transformation unit, and an object-attribute determination unit. The feature extraction unit reads the image and extracts the edges of a feature in the image. The motion-information calculator evaluates the feature in the image and outputs feature velocity information for each image pixel that shows the edges of the feature. The coordinate transformation unit reads two-dimensional position coordinates of the image pixels and outputs transformed three-dimensional position coordinates. And the object-attribute determination unit determines, based on the transformed three-dimensional position coordinates and on the feature velocity information, whether the feature is a moving object.

Claims

1. An object detecting system for a vehicle moving along a surface, the object detecting system comprising:

an imaging device disposed on the vehicle and capturing a two-dimensional image representing a scene ahead of the vehicle, the two-dimensional image having a vertical axis generally perpendicular to the surface and a horizontal axis generally parallel to the surface; and

a controller receiving the image from the imaging device, the controller including;

a feature extraction unit reading the image and extracting from the image edges of a feature;

a motion-information calculator evaluating the feature and outputting feature velocity information for each image pixel showing the edges of the feature;

a coordinate transformation unit reading two-dimensional position coordinates of the image pixels and outputting transformed three-dimensional position coordinates; and

an object-attribute determination unit determining whether the feature is a moving object, the determining being based on the transformed three-dimensional position coordinates received from the coordinate transformation unit and on the feature velocity information received from the motion-information calculator.

2. The object detecting system according to claim 1, wherein the image comprises a first feature represented by a first set of pixels and the determining by the object-attribute determination unit comprises:

detecting that the first feature is a stationary object based on establishing that a first relationship exists between horizontal coordinates and velocity information for the first set of pixels, the first relationship being represented by a sloping line on a graph that plots horizontal coordinates along an abscissa axis and velocity along an ordinate axis;

identifying that the collection of points is disposed on an opposite side of the abscissa axis with respect to the sloping line.

3. The object detecting system according to claim 1, wherein the image comprises a second set of pixels, and the determining by the object-attribute determination unit comprises:

detecting that the second feature is a moving object based on establishing that a second relationship exists between horizontal coordinates and velocity information for the second set of pixels, the second relationship being represented by a collection of points plotted on the graph, the detecting that the second feature is a moving object including at least one of;

identifying that the collection of points has a velocity magnitude different from the sloping line.

4. The object detecting system according to claim 1, wherein the transformed three-dimensional position coordinates lie in a transformation plane that is generally parallel to the surface, and a grid splits a specified area on the transformation plane into a plurality of regions.

5. The object detecting system according to claim 4, further comprising:

a grouping unit associating in a group the image pixels that are

     1) adjacent in the vertical direction of the image and

     2) have relative feature velocity information that is within a predetermined range,

wherein the determining by the object-attribute determination unit is based on the transformed three-dimensional positional coordinates of

     1) a vertically uppermost pixel of the group of image pixels and

     2) a vertically lowermost pixel of the group of image pixels.

6. The object detecting system according to claim 5, wherein the determining by the object-attribute determination unit comprises 1) determining that the feature is a two-dimensional object when the vertically uppermost pixel and the vertically lowermost pixel lie in the same one of the plurality of regions of the specified area of the transformation plane, and 2) determining that the feature is a three-dimensional object when the vertically uppermost pixel lies outside the specified area of the transformation plane and the vertically lowermost pixel lies in any of the plurality of regions.

7. The object detecting system according to claim 5, wherein the determining by the object-attribute determination unit comprises 1) determining that a two-dimensional object candidate is defined when the transformed three-dimensional positional coordinates of a first plurality of the group of image pixels lie in any of the plurality of regions, 2) determining that a three-dimensional object candidate is defined when the transformed three-dimensional positional coordinates of a second plurality of the group of image pixels lie outside of the plurality of regions, and 3) determines that the feature is a three-dimensional object when the transformed three-dimensional position coordinates of the two-dimensional object candidate and of the three-dimensional object candidate points represent one three-dimensional object when the two-dimensional object candidate points are consecutively arranged in the vertical direction of the image.

8. The object detecting system according to claim 5, wherein the determining by the object-attribute determination unit comprises determining at least one lane border for the vehicle, the determining being based on a plurality of transformed three-dimensional coordinates of vertically lowermost pixels being disposed along a vertical line on the transformation plane;

wherein the determining is performed in each division of the image that is defined by the at least one lane border.

9. An object detecting system for a vehicle moving along a surface, the object detecting system comprising:

an imaging device capturing an image representing a scene ahead of the vehicle; and

control means for determining in less than two seconds if an object in the image is a moving object.

10. The object detecting system according to claim 9, wherein the control means determines in approximately 0.5 seconds if the object in the image is a moving object.

11. An object detecting system comprising:

image capturing means for capturing an image of a scene ahead of a vehicle;

feature extracting means for extracting a feature from the image captured by the image capturing means;

motion-information calculating means for calculating feature velocity information about pixels representing the feature;

coordinate transforming means for transforming position coordinates of the pixels into three-dimensional position coordinates of a three-dimensional coordinate system; and

object-attribute determining means for determining whether the feature is a moving object, the determining being based on the three-dimensional position coordinates of the pixels and the feature velocity information.

12. The object detecting system according to claim 11, wherein the feature comprises an edge of an object on the image captured by the image capturing means.

13. An object detecting method for detecting a moving object in an image representing a scene ahead of the vehicle, the method comprising:

preparing velocity information about a first set of pixels showing a feature of an object in the image;

transforming two-dimensional coordinates of the first set of pixels into three-dimensional coordinates; and

determining whether the object is a moving object based on the prepared feature velocity information and on the transformed three-dimensional position coordinates.

14. The object detecting method according to claim 13, wherein the determining comprises detecting a moving object when the image includes a second set of pixels having velocity information that is different from the prepared velocity information about the first set of pixels, the detecting includes evaluating horizontal coordinates of the two-dimensional coordinates relative to the velocity information corresponding to the two-dimensional coordinates.

15. The object detecting method according to claim 13, further comprising:

splitting into a plurality of regions a specified area of a transformation plane defined by the transformed three-dimensional position coordinates.

16. The object detecting method according to claim 15, further comprising:

grouping together in a separate group those of the first set of pixels that are both adjacent in a vertical direction of the image and have relative velocity information that is within a predetermined range;

wherein the determining is based on the transformed three-dimensional position coordinates of a vertically uppermost pixel of the group and of a vertically lowermost pixel of the group.

17. The object detecting method according to claim 16, wherein the determining comprises determining that the object is a two-dimensional when the vertically uppermost pixel and the vertically lowermost pixel lie in the same one of the plurality of regions, and 2) determining that the object is a three-dimensional when the vertically uppermost pixel lies outside the specified area of the transformation plane and the vertically lowermost pixel lies in any of the plurality of regions.

18. The object detecting method according to claim 16, wherein the determining comprises 1) determining that a two-dimensional object candidate is defined when the transformed three-dimensional positional coordinates of a first plurality of the group of image pixels lie in any of the plurality of regions, 2) determining that a three-dimensional object candidate is defined when the transformed three-dimensional positional coordinates of a second plurality of the group of image pixels lie outside of the plurality of regions, and 3) determines that the feature is a three-dimensional object when the transformed three-dimensional position coordinates of the two-dimensional object candidate and of the three-dimensional object candidate points represent one three-dimensional object when the two-dimensional object candidate points are consecutively arranged in the vertical direction of the image.

19. The object detecting method according to claim 16, wherein the determining comprises determining at least one lane border for the vehicle, the determining being based on a plurality of transformed three-dimensional coordinates of vertically lowermost pixels being disposed along a vertical line on the transformation plane;

wherein the determining is performed in each division of the image that is defined by the at least one lane border.