Method and device for analyzing trafficability

• US 9,690,993 B2
• Filed: 12/06/2013
• Issued: 06/27/2017
• Est. Priority Date: 12/11/2012
• Status: Active Grant

• Current Assignee: Conti Temic Microelectronic GmbH (Continental AG)
• Original Assignee: Conti Temic Microelectronic GmbH (Continental AG)
• Inventors: Grewe, Ralph, Hegemann, Stefan, Zobel, Matthias
• Primary Examiner(s): Tsai, Tsung-Yin
• Application Number:
  US14/647,958
• Publication Number:
  US 20150324649A1
• Time in Patent Office: 1,299 Days
• Field of Search: None
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/40 Analysis of texture depth or shape recovery from texture G06T7/529
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road

Abstract

A method and a device for analyzing trafficability use a computer to perform the following steps:

• • receiving image data representing an image of surroundings in front of a vehicle,
  • analyzing the image data to identify different zones in the image of the surroundings, and
  • analyzing the identified different zones in terms of trafficability thereof for the vehicle.

Claims

1. A method for a driver assistance system of a subject vehicle, comprising the steps:

with a sensor system of the subject vehicle, producing image data comprising plural image pixels and three-dimensional position data respectively associated with each one of the image pixels, wherein the image pixels represent at least one image of an external environment in front of the subject vehicle;

with a processor arrangement of the subject vehicle, analyzing the image data including the three-dimensional position data to calculate respective spatial positions of the image pixels in three-dimensional space, and from the spatial positions of the image pixels determining an estimated ground plane in the at least one image;

with the processor arrangement, classifying the image pixels into ground plane pixels and other pixels based on the three-dimensional position data, wherein the ground plane pixels belong to the estimated ground plane and the other pixels do not belong to the estimated ground plane in the at least one image;

with the processor arrangement, analyzing only the ground plane pixels and not the other pixels to segment the estimated ground plane into ground plane zones that are distinguished from one another in the at least one image based on respective characteristic features of the ground plane pixels of the ground plane zones in the image data;

with the processor arrangement, analyzing the image data to determine and assign, respectively to the ground plane zones, respective trafficability values that indicate whether the subject vehicle can safely drive in the respective areas of the external environment represented by the respective ground plane zones in the at least one image; and

with the driver assistance system, outputting an information to a driver of the subject vehicle or autonomously actuating a vehicle control of the subject vehicle based and dependent on the trafficability values.

2. The method according to claim 1, wherein each one of the trafficability values is respectively an indication that the associated respective area of the external environment is either trafficable or non-trafficable.

3. The method according to claim 1, wherein the trafficability value of a first one of the ground plane zones is determined by identifying a driving activity of an other vehicle in a first one of the areas of the external environment represented by the first ground plane zone, and dependent thereon setting the trafficability value of the first ground plane zone to indicate whether the subject vehicle can safely drive in the first area represented by the first ground plane zone.

4. The method according to claim 3, wherein the trafficability value of the first ground plane zone is set to indicate that the subject vehicle can safely drive in the first area when it was identified that the other vehicle safely drove in the first area.

5. The method according to claim 3, further comprising identifying a second one of the ground plane zones of which the characteristic features are similar to the characteristic features of the first ground plane zone, and wherein the trafficability value of the second ground plane zone is determined by setting the trafficability value of the second ground plane zone to be equal to the trafficability value determined for the first ground plane zone.

6. The method according to claim 3, wherein the trafficability value of the first ground plane zone is determined dynamically and changes over time dependent on a time-varying result of the identifying of the driving activity of the other vehicle.

7. The method according to claim 1, wherein the respective trafficability value of a respective one of the ground plane zones is determined by identifying whether an obstacle protruding upwardly from the estimated ground plane exists in the respective area of the external environment represented by the respective ground plane zone, and dependent thereon setting the respective trafficability value to indicate that the subject vehicle cannot safely drive in the respective area.

8. The method according to claim 7, wherein the three-dimensional position data is determined by a stereovision analysis of a plurality of the images, and the obstacle is detected from the three-dimensional position data.

9. The method according to claim 1, wherein the three-dimensional position data is determined by a stereovision analysis of a plurality of the images, and wherein the ground plane pixels represent the estimated ground plane as a two-dimensional plane.

10. The method according to claim 1,

wherein the three-dimensional position data is determined by a stereovision analysis of a plurality of the images, and

wherein the characteristic features of the ground plane pixels of the ground plane zones, based on which the estimated ground plane is segmented, include a respective height variance of the ground plane pixels respectively allocated to different ones of the ground plane zones.

11. The method according to claim 1, wherein the characteristic features of the ground plane pixels of the ground plane zones, based on which the estimated ground plane is segmented, include at least one of a color, a saturation, an intensity or a texture of a portion of the at least one image represented by the ground plane pixels.

12. The method according to claim 1, further comprising, with the processor arrangement, for each respective portion of any one of the ground plane zones having any of the other pixels located above the respective portion in the three-dimensional space, assigning to the respective portion a respective one of the trafficability values comprising an indication that a respective partial area of the external environment represented by the respective portion of the ground plane zone in the at least one image is non-trafficable.

13. A device for a driver assistance system of a subject vehicle, comprising:

a sensor system that is configured and arranged to produce image data comprising plural image pixels and three-dimensional position data respectively associated with each one of the image pixels, wherein the image pixels represent at least one image of an external environment in front of the subject vehicle;

a processor arrangement that is configured and arranged to perform a method including;

analyzing the image data including the three-dimensional position data to calculate respective spatial positions of the image pixels in three-dimensional space, and from the spatial positions of the image pixels determining an estimated ground plane in the at least one image;

classifying the image pixels into ground plane pixels and other pixels based on the three-dimensional position data, wherein the ground plane pixels belong to the estimated ground plane and the other pixels do not belong to the estimated ground plane in the at least one image;

analyzing only the ground plane pixels and not the other pixels to segment the estimated ground plane into ground plane zones that are distinguished from one another in the at least one image based on respective characteristic features of the ground plane pixels of the ground plane zones in the image data; and

analyzing the image data to determine and assign, respectively to the ground plane zones, respective trafficability values that indicate whether the subject vehicle can safely drive in the respective areas of the external environment represented by the respective ground plane zones in the at least one image;

and

an output configured and arranged to output an information to a driver of the subject vehicle or to output a control signal configured to autonomously actuate a vehicle control of the subject vehicle, based and dependent on the trafficability values.

14. The device according to claim 13, wherein the processor arrangement comprises a microprocessor or a microcontroller, as well as software and/or firmware configured to perform the method when executed by the microprocessor or the microcontroller.

15. The device according to claim 13, wherein the processor arrangement comprises an ASIC circuit or a PGA circuit configured and arranged to perform the method.

16. A method comprising:

with a sensor system of a vehicle, producing image pixels;

with a processor arrangement of the vehicle, performing the following steps;

calculating three-dimensional position data respectively for each one of the image pixels;

evaluating the three-dimensional position data of the image pixels to determine an estimated ground plane in the three-dimensional position data of the image pixels;

analyzing only those of the image pixels of which the respective three-dimensional position data lie on the estimated ground plane and not others of the image pixels of which the respective three-dimensional position data lie above or below the estimated ground plane, and by said analyzing, segmenting the estimated ground plane into ground plane zones that are distinguished from one another based on respective characteristic features of the analyzed image pixels;

determining and assigning respective trafficability indicators respectively to the ground plane zones, wherein each one of the trafficability indicators respectively indicates that the respective assigned ground plane zone is either trafficable or non-trafficable; and

outputting a signal dependent on and indicative of the trafficability indicators respectively assigned to the ground plane zones;

and

with a driver assistance system of the vehicle, outputting an information to a driver of the vehicle dependent on the signal, or autonomously actuating a vehicle control of the vehicle dependent on the signal.