Object detecting system and object detecting method

• US 7,899,211 B2
• Filed: 11/29/2006
• Issued: 03/01/2011
• Est. Priority Date: 12/07/2005
• Status: Active Grant

• Current Assignee: Nissan Motor Co., Ltd.
• Original Assignee: Nissan Motor Co., Ltd.
• Inventors: Fujimoto, Kazumi
• Primary Examiner(s): Tabatabai; Abolfazl
• Application Number:
  US11/605,322
• Publication Number:
  US 20070127778A1
• Time in Patent Office: 1,553 Days
• Field of Search: 382/103, 382/104, 382/107, 340/436, 340/3.41, 340/3.43, 340/903, 340/961, 340/933, 701/25, 701/96, 701/300, 701/301, 701/213
• US Class Current: 382/104
• CPC Class Codes: G06T 2207/30261 Obstacle
• CPC Class Codes: G06T 7/12 Edge-based segmentation
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road

Abstract

An object detecting system includes an imaging device disposed on a vehicle and a controller that receives a two-dimensional image, which represents a scene ahead of the vehicle, from the imaging device. The controller includes a feature extraction unit, a motion-information calculator, a coordinate transformation unit, and an object-attribute determination unit. The feature extraction unit reads the image and extracts the edges of a feature in the image. The motion-information calculator evaluates the feature in the image and outputs feature velocity information for each image pixel that shows the edges of the feature. The coordinate transformation unit reads two-dimensional position coordinates of the image pixels and outputs transformed three-dimensional position coordinates. And the object-attribute determination unit determines, based on the transformed three-dimensional position coordinates and on the feature velocity information, whether the feature is a moving object.

Claims

1. An object detecting system for a vehicle moving along a surface, the object detecting system comprising:

an imaging device disposed on the vehicle and capturing a two-dimensional image representing a scene ahead of the vehicle, the two-dimensional image having a vertical axis generally perpendicular to the surface and a horizontal axis generally parallel to the surface; and

a controller receiving the image from the imaging device, the controller including;

a feature extraction unit reading the image, extracting from the image edges of a feature, producing an image of pixels showing the edges of the feature, and normalizing the edges of the feature;

a motion-information calculator evaluating the feature and outputting feature velocity information for each image pixel showing the normalized edges of the feature;

a coordinate transformation unit reading two-dimensional position coordinates of the image pixels and outputting transformed three-dimensional position coordinates; and

an object-attribute determination unit determining whether the feature is a moving object, the determining being based on the transformed three-dimensional position coordinates received from the coordinate transformation unit and on the feature velocity information received from the motion-information calculator.

2. The object detecting system according to claim 1, wherein the transformed three-dimensional position coordinates lie in a transformation plane that is generally parallel to the surface, and a grid splits a specified area on the transformation plane into a plurality of regions.

3. An object detecting system for a vehicle moving along a surface, the object detecting system comprising:

an imaging device disposed on the vehicle and capturing a two-dimensional image representing a scene ahead of the vehicle, the two-dimensional image having a vertical axis generally perpendicular to the surface and a horizontal axis generally parallel to the surface; and

a controller receiving the image from the imaging device, the controller including;

a feature extraction unit reading the image and extracting from the image edges of a feature;

a motion-information calculator evaluating the feature and outputting feature velocity information for each image pixel showing the edges of the feature;

a coordinate transformation unit reading two-dimensional position coordinates of the image pixels and outputting transformed three-dimensional position coordinates; and

an object-attribute determination unit determining whether the feature is a moving object, the determining being based on the transformed three-dimensional position coordinates received from the coordinate transformation unit and on the feature velocity information received from the motion-information calculator;

wherein the image comprises a first feature represented by a first set of pixels and the determining by the object-attribute determination unit comprises;

detecting that the first feature is a stationary object based on establishing that a first relationship exists between horizontal coordinates and velocity information for the first set of pixels, the first relationship being represented by a sloping line on a graph that plots horizontal coordinates along an abscissa axis and velocity along an ordinate axis;

identifying that the collection of points is disposed on an opposite side of the abscissa axis with respect to the sloping line.

4. An object detecting system for a vehicle moving along a surface, the object detecting system comprising:

an imaging device disposed on the vehicle and capturing a two-dimensional image representing a scene ahead of the vehicle, the two-dimensional image having a vertical axis generally perpendicular to the surface and a horizontal axis generally parallel to the surface; and

a controller receiving the image from the imaging device, the controller including;

a feature extraction unit reading the image and extracting from the image edges of a feature;

a motion-information calculator evaluating the feature and outputting feature velocity information for each image pixel showing the edges of the feature;

a coordinate transformation unit reading two-dimensional position coordinates of the image pixels and outputting transformed three-dimensional position coordinates; and

an object-attribute determination unit determining whether the feature is a moving object, the determining being based on the transformed three-dimensional position coordinates received from the coordinate transformation unit and on the feature velocity information received from the motion-information calculator;

wherein the image comprises a second set of pixels, and the determining by the object-attribute determination unit comprises;

detecting that the second feature is a moving object based on establishing that a second relationship exists between horizontal coordinates and velocity information for the second set of pixels, the second relationship being represented by a collection of points plotted on the graph, the detecting that the second feature is a moving object including at least one of;

identifying that the collection of points has a velocity magnitude different from the sloping line.

5. An object detecting system for a vehicle moving along a surface, the object detecting system comprising;

an imaging device disposed on the vehicle and capturing a two-dimensional image representing a scene ahead of the vehicle, the two-dimensional image having a vertical axis generally perpendicular to the surface and a horizontal axis generally parallel to the surface; and

a controller receiving the image from the imaging device, the controller including;

a feature extraction unit reading the image and extracting from the image edges of a feature;

a motion-information calculator evaluating the feature and outputting feature velocity information for each image pixel showing the edges of the feature;

a coordinate transformation unit reading two-dimensional position coordinates of the image pixels and outputting transformed three-dimensional position coordinates; and

an object-attribute determination unit determining whether the feature is a moving object, the determining being based on the transformed three-dimensional position coordinates received from the coordinate transformation unit and on the feature velocity information received from the motion-information calculator;

wherein the transformed three-dimensional position coordinates lie in a transformation plane that is generally parallel to the surface, and a grid splits a specified area on the transformation plane into a plurality of regions;

the object detecting system further comprising;

a grouping unit associating in a group the image pixels that are

     1) adjacent in the vertical direction of the image and

     2) have relative feature velocity information that is within a predetermined range,

wherein the determining by the object-attribute determination unit is based on the transformed three-dimensional positional coordinates of

     1) a vertically uppermost pixel of the group of image pixels and

     2) a vertically lowermost pixel of the group of image pixels.

6. The object detecting system according to claim 5, wherein the determining by the object-attribute determination unit comprises 1) determining that the feature is a two-dimensional object when the vertically uppermost pixel and the vertically lowermost pixel lie in the same one of the plurality of regions of the specified area of the transformation plane, and 2) determining that the feature is a three-dimensional object when the vertically uppermost pixel lies outside the specified area of the transformation plane and the vertically lowermost pixel lies in any of the plurality of regions.

7. The object detecting system according to claim 5, wherein the determining by the object-attribute determination unit comprises 1) determining that a two-dimensional object candidate is defined when the transformed three-dimensional positional coordinates of a first plurality of the group of image pixels lie in any of the plurality of regions, 2) determining that a three-dimensional object candidate is defined when the transformed three-dimensional positional coordinates of a second plurality of the group of image pixels lie outside of the plurality of regions, and 3) determines that the feature is a three-dimensional object when the transformed three-dimensional position coordinates of the two-dimensional object candidate and of the three-dimensional object candidate points represent one three-dimensional object when the two-dimensional object candidate points are consecutively arranged in the vertical direction of the image.

8. The object detecting system according to claim 5, wherein the determining by the object-attribute determination unit comprises determining at least one lane border for the vehicle, the determining being based on a plurality of transformed three-dimensional coordinates of vertically lowermost pixels being disposed along a vertical line on the transformation plane;

wherein the determining is performed in each division of the image that is defined by the at least one lane border.

9. An object detecting system comprising:

image capturing means for capturing an image of a scene ahead of a vehicle;

feature extracting means for extracting normalized edges of a feature from the image captured by the image capturing means, the normalized edges being represented by pixels;

motion-information calculating means for calculating feature velocity information about the pixels representing the normalized edges of the feature;

coordinate transforming means for transforming position coordinates of the pixels into three-dimensional position coordinates of a three-dimensional coordinate system; and

object-attribute determining means for determining whether the feature is a moving object, the determining being based on the three-dimensional position coordinates of the pixels and the feature velocity information.

10. An object detecting method for detecting a moving object in an image representing a scene ahead of the vehicle, the method comprising:

preparing velocity information about a first set of pixels showing a feature of an object in the image;

transforming two-dimensional coordinates of the first set of pixels into three-dimensional coordinates;

determining whether the object is a moving object based on the prepared feature velocity information and on the transformed three-dimensional position coordinates;

splitting into a plurality of regions a specified area of a transformation plane defined by the transformed three-dimensional position coordinates; and

grouping together in a separate group those of the first set of pixels that are both adjacent in a vertical direction of the image and have relative velocity information that is within a predetermined range;

wherein the determining is based on the transformed three-dimensional position coordinates of a vertically uppermost pixel of the group and of a vertically lowermost pixel of the group.

11. The object detecting method according to claim 10, wherein the determining comprises determining that the object is a two-dimensional when the vertically uppermost pixel and the vertically lowermost pixel lie in the same one of the plurality of regions, and 2) determining that the object is a three-dimensional when the vertically uppermost pixel lies outside the specified area of the transformation plane and the vertically lowermost pixel lies in any of the plurality of regions.

12. The object detecting method according to claim 10, wherein the determining comprises 1) determining that a two-dimensional object candidate is defined when the transformed three-dimensional positional coordinates of a first plurality of the group of image pixels lie in any of the plurality of regions, 2) determining that a three-dimensional object candidate is defined when the transformed three-dimensional positional coordinates of a second plurality of the group of image pixels lie outside of the plurality of regions, and 3) determines that the feature is a three-dimensional object when the transformed three-dimensional position coordinates of the two-dimensional object candidate and of the three-dimensional object candidate points represent one three-dimensional object when the two-dimensional object candidate points are consecutively arranged in the vertical direction of the image.

13. The object detecting method according to claim 10, wherein the determining comprises determining at least one lane border for the vehicle, the determining being based on a plurality of transformed three-dimensional coordinates of vertically lowermost pixels being disposed along a vertical line on the transformation plane;

wherein the determining is performed in each division of the image that is defined by the at least one lane border.