Method for determining whether a vehicle can pass through an object by means of a 3-D camera

• US 9,495,603 B2
• Filed: 07/06/2012
• Issued: 11/15/2016
• Est. Priority Date: 09/07/2011
• Status: Active Grant

• Current Assignee: Continental Teves AG & Co. OHG (Continental AG), Conti Temic Microelectronic GmbH (Continental AG)
• Original Assignee: Continental Teves AG & Co. OHG (Continental AG), Conti Temic Microelectronic GmbH (Continental AG)
• Inventors: Hegemann, Stefan, Lueke, Stefan, Hassenpflug, Christoph
• Primary Examiner(s): Perungavoor, Sath V
• Assistant Examiner(s): Aynalem, Nathnael
• Application Number:
  US14/343,192
• Publication Number:
  US 20140218481A1
• Time in Patent Office: 1,593 Days
• Field of Search: 348/46, 348/148, 340/435
• US Class Current: 1/1
• CPC Class Codes: B60W 2420/403 Image sensing, e.g. optical camera
• CPC Class Codes: G01S 17/894 3D imaging with simultaneous measurement of time-of-flight at a 2D array of receiver pixels, e.g. time-of-flight cameras or flash lidar
• CPC Class Codes: G01S 17/931 of land vehicles
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G08G 1/165 for passive traffic, e.g. including static obstacles, trees
• CPC Class Codes: H04N 13/271 wherein the generated image signals comprise depth maps or disparity maps

Abstract

A method and a device determine whether a vehicle can pass through a passage of an object, based on image data from a 3-D camera, which records an image of surroundings of the vehicle. A trajectory of the vehicle is ascertained. From the image data, it is determined whether an object located above the trajectory is recognized and whether the object has at least one connection to the ground. The dimensions and the shape of an entry area or a passage space between the object and the roadway, through which the vehicle is to pass, are determined from the image data. By comparing the dimensions and shape of the entry area or the passage space with the dimensions and shape of the vehicle, it is determined whether and how the vehicle can pass through the entry area or passage space.

Claims

1. A method of determining whether a vehicle can pass through or pass under an object, while the vehicle is driving on a roadway, wherein the roadway is part of a ground, said method comprising steps:

with a 3-D camera, recording at least one image of surroundings ahead of the vehicle, to provide image data;

ascertaining at least one trajectory along which the vehicle is likely to move;

from at least the image data of the 3-D camera, detecting an object that includes an upper portion located above the trajectory,

from at least the image data of the 3-D camera, detecting whether the object includes at least one connection portion that connects the upper portion to the ground, from at least the image data of the 3-D camera, determining dimensions and a shape of a 2-dimensional entry area that is bounded vertically between the upper portion of the object and the roadway, and that is bounded laterally by any said connection portion that has been detected, and

determining whether it is possible for the vehicle driving along the trajectory to pass through the 2-dimensional entry area by fitting vertically and laterally in the 2-dimensional entry area.

2. The method according to claim 1, further comprising, from at least the image data of the 3-D camera, determining dimensions and a shape of a 3-dimensional passage space that extends along the roadway, and that is bounded vertically between the upper portion of the object and the roadway, and that is bounded laterally by any said connection portion that has been detected.

3. The method according to claim 2, wherein the recording of the at least one image comprises recording a sequence of images, the image data comprise data that represent the sequence of images, and the determining of the dimensions and the shape of the 3-dimensional passage space comprises evaluating the data that represent the sequence of images.

4. The method according to claim 3, wherein the determining of the dimensions and the shape of the 3-dimensional passage space further takes into account a motion of the 3-D camera.

5. The method according to claim 1, further comprising producing a depth map from the image data of the 3-D camera, and wherein the detecting of the object and the detecting of the at least one connection portion comprise evaluating the depth map.

6. The method according to claim 5, wherein the depth map is a disparity map.

7. The method according to claim 5, further comprising detecting edges of the 2-dimensional entry area from the depth map.

8. The method according to claim 5, further comprising detecting edges of the 2-dimensional entry area from the image data by performing an edge detection algorithm, and wherein the determining of the dimensions and the shape of the 2-dimensional entry area is performed using the depth map and the detected edges.

9. The method according to claim 1, further comprising detecting edges of the 2-dimensional entry area from the image data by performing an edge detection algorithm.

10. The method according to claim 9, wherein the determining of the dimensions and the shape of the 2-dimensional entry area is performed using the detected edges.

11. The method according to claim 1, wherein the recording of the at least one image comprises recording a sequence of images, the image data comprise data that represent the sequence of images, and the determining of the dimensions and the shape of the 2-dimensional entry area comprises evaluating the data that represent the sequence of images.

12. The method according to claim 11, wherein the determining of the dimensions and the shape of the 2-dimensional entry area further takes into account a motion of the 3-D camera.

13. The method according to claim 1, further comprising producing a 3-D scene reconstruction from the image data of the 3-D camera.

14. The method according to claim 1, wherein the 3-D camera comprises a stereo camera.

15. The method according to claim 1, wherein the 3-D camera comprises a photonic mixing device camera.

16. The method according to claim 1, wherein the determining of whether it is possible for the vehicle to pass through the 2-dimensional entry area comprises comparing dimensions of the vehicle, a shape of the vehicle, and a lateral position of the vehicle on the trajectory on the roadway, respectively to the dimensions of the 2-dimensional entry area, the shape of the 2-dimensional entry area, and a lateral position of the 2-dimensional entry area relative to the trajectory.

17. The method according to claim 1,

wherein the dimensions and the shape of the 2-dimensional entry area that are determined include entry area parameters regarding at least a shape of an upper boundary of the entry area bounded by the upper portion of the object, a height of the upper boundary above the roadway, and a lateral position of the shape of the upper boundary relative to the trajectory of the vehicle;

wherein the step of determining whether it is possible for the vehicle driving along the trajectory to pass through the entry area comprises sub-steps;

a) obtaining vehicle data regarding at least a cross-sectional shape of the vehicle, a height of the cross-sectional shape extending above the roadway, and a lateral position of the cross-sectional shape relative to the trajectory of the vehicle;

b) comparing the vehicle data with the entry area parameters to produce a corresponding comparison result; and

c) in response to and dependent on the comparison result, determining whether the vehicle driving along the trajectory can pass without collision through the entry area, and producing a corresponding clearance result;

and

wherein the method further comprises a step d) in response to and dependent on the clearance result, outputting to a driver of the vehicle an information indicative of the clearance result, or automatically intervening in a steering control system or a braking control system of the vehicle.

18. The method according to claim 17,

wherein the sub-step b) comprises comparing the cross-sectional shape of the vehicle with the shape of the upper boundary of the entry area, comparing the height of the cross-sectional shape of the vehicle with the height of the upper boundary, and comparing the lateral position of the cross-sectional shape of the vehicle with the lateral position of the shape of the upper boundary of the entry area;

wherein the sub-step c) determines that the vehicle driving along the trajectory cannot pass without collision through the entry area; and

wherein the step d) comprises the automatic intervening in the steering control system or the braking control system.

19. The method according to claim 17,

wherein the sub-step c) determines that the vehicle driving along the trajectory cannot pass without collision through the entry area,

wherein the method further comprises determining an alternative trajectory that the vehicle can drive along to pass without collision through the entry area, and

wherein the method further comprises outputting to the driver a further information indicative of the alternative trajectory, or automatically intervening in the steering control system to steer the vehicle to the alternative trajectory.

20. The method according to claim 2, wherein the 3 dimensional passage space extends to a far end of the upper portion of the object, and wherein the method further comprises determining passage parameters regarding a shape, a height, and a lateral position of the passage space along the trajectory to the far end of the upper portion of the object.

21. A device for determining whether a vehicle can pass through or pass under an object, while the vehicle is driving on a roadway, wherein the roadway is part of a ground, said device comprising:

a 3-D camera configured and arranged to record at least one image of surroundings ahead of the vehicle;

a circuit arrangement configured to ascertain at least one trajectory on which the vehicle is likely to move;

a circuit arrangement configured to determine from the image data of the 3-D camera;

whether an object is present that includes an upper portion located above the trajectory, and

whether said object has at least one connection portion that connects the upper portion to the ground;

and

a circuit arrangement configured to determine, from the image data, dimensions and a shape of a 2-dimensional entry area bounded vertically between the upper portion of the object and the roadway, and to determine, from the dimensions and the shape of the entry area, whether it is possible for the vehicle driving along the trajectory to pass through the entry area.