Monocular cued detection of three-dimensional structures from depth images

• US 10,776,635 B2
• Filed: 02/14/2014
• Issued: 09/15/2020
• Est. Priority Date: 09/21/2010
• Status: Active Grant

• Current Assignee: MobilEye Vision Technologies Ltd. (Intel Corporation)
• Original Assignee: MobilEye Vision Technologies Ltd. (Intel Corporation)
• Inventors: Berberian, Oded, Stein, Gideon
• Primary Examiner(s): Rahman, Mohammad J
• Application Number:
  US14/180,548
• Publication Number:
  US 20140160244A1
• Time in Patent Office: 2,405 Days
• Field of Search: None
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
• CPC Class Codes: H04N 13/271 wherein the generated image signals comprise depth maps or disparity maps

Abstract

Detection of three dimensional obstacles using a system mountable in a host vehicle including a camera connectible to a processor. Multiple image frames are captured in the field of view of the camera. In the image frames, an imaged feature is detected of an object in the environment of the vehicle. The image frames are portioned locally around the imaged feature to produce imaged portions of the image frames including the imaged feature. The image frames are processed to compute a depth map locally around the detected imaged feature in the image portions. Responsive to the depth map, it is determined if the object is an obstacle to the motion of the vehicle.

Claims

1. A method for detection of three dimensional obstacles, the method performed by a system mountable in a vehicle, wherein the system includes a processor operatively connectible to a camera, the method comprising:

capturing a plurality of two-dimensional image frames in a field of view of the camera;

detecting, in the image frames, a two-dimensional candidate of a three-dimensional object in an environment of the vehicle using monocular information;

portioning the image frames around the candidate to produce image portions including the candidate;

computing a depth map around the candidate in the image portions, wherein the depth map includes depth values related to a function of distance from the camera to the object; and

determining, based on the depth map, whether the object represented by the candidate is an obstacle to the motion of the vehicle.

2. The method of claim 1, further comprising:

representing the object with a plurality of models;

computing a plurality of model depth maps based on the respective models;

comparing the depth map of the candidate with the model depth maps; and

determining, based on the comparison, whether the object represented by the candidate is an obstacle to the motion of the vehicle.

3. The method of claim 2, wherein the models are selected from the group consisting of:

a horizontal planar model, a vertical planar model, a mixed model including horizontal and vertical portions, a spherical model, a circular model, a model of a guard rail, a model of lane marker, a model of a road curb and a model of an upright pedestrian.

4. The method of claim 1, wherein computing the depth map is performed only around the candidate in the image portions.

5. The method of claim 1, further comprising:

adjusting a resolution of the computation of the depth map to achieve an accuracy determined based on the candidate.

11. The method of claim 1, further comprising:

identifying a horizon in the image frames,

wherein the two-dimensional candidate is detected in the image frames below the horizon.

6. A system for detection of three dimensional obstacles, the system mountable in a host vehicle, the system including a processor operatively connectible to a camera, wherein the system is configured to:

capture a plurality of two-dimensional image frames in a field of view of the camera;

detect, in the image frames, a two-dimensional candidate of a three-dimensional object in an environment of the vehicle using monocular information;

portion the image frames around the candidate to produce imaged portions including the candidate;

compute a depth map around the candidate in the image portions, wherein the depth map includes depth values related to a function of distance from the camera to the object; and

determine, based on the depth map, whether the object represented by the candidate is an obstacle to the motion of the vehicle.

7. The system of claim 6, wherein the system is further configured to:

represent the object with a plurality of models;

compute a plurality of model depth maps based on the respective models;

compare the depth map of the candidate with the model depth maps; and

determine, based on the comparison, whether the object represented by the candidate is an obstacle to the motion of the vehicle.

8. The system of claim 7, wherein the models are selected from the group consisting of:

a horizontal planar model, a vertical planar model, a mixed model including horizontal and vertical portions, a spherical model, a circular model, a model of a guard rail, a model of lane marker, a model of a road curb and a model of an upright pedestrian.

9. The system of claim 6, wherein the depth map is computed only around the candidate in the image portions.

10. The system of claim 6, wherein the system is further configured to:

adjust a resolution of the computation of the depth map to achieve an accuracy determined based on the candidate.

12. The system of claim 6, wherein the system is further configured to identify a horizon in the image frames, and the two-dimensional candidate is detected in the image frames below the horizon.