Systems and methods for non-obstacle area detection

• US 9,761,000 B2
• Filed: 09/18/2015
• Issued: 09/12/2017
• Est. Priority Date: 09/18/2015
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Dane, Gokce, Bhaskaran, Vasudev
• Primary Examiner(s): Allison, Andrae S
• Application Number:
  US14/858,471
• Publication Number:
  US 20170084038A1
• Time in Patent Office: 725 Days
• Field of Search: None
• CPC Class Codes: G06F 18/254 of classification results, e.g. of results related to same input data
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/30256 Lane; Road marking
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/50 Depth or shape recovery
• CPC Class Codes: G06T 7/507 from shading G06T7/586 takes precedence
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road

Abstract

A method performed by an electronic device is described. The method includes performing vertical processing of a depth map to determine a vertical non-obstacle estimation. The method also includes performing horizontal processing of the depth map to determine a horizontal non-obstacle estimation. The method further includes combining the vertical non-obstacle estimation and the horizontal non-obstacle estimation. The method additionally includes generating a non-obstacle map based on the combination of the vertical and horizontal non-obstacle estimations.

Claims

1. A method performed by an electronic device, comprising:

performing vertical processing of a depth map to determine a vertical non-obstacle estimation, wherein performing vertical processing comprises;

dividing the depth map into segments, wherein at least one segment is a portion of a column of pixels in the depth map; and

estimating linear model parameters for the at least one segment to determine the vertical non-obstacle estimation;

performing horizontal processing of the depth map to determine a horizontal non-obstacle estimation;

combining the vertical non-obstacle estimation and the horizontal non-obstacle estimation; and

generating a non-obstacle map based on the combination of the vertical and horizontal non-obstacle estimations.

2. The method of claim 1, wherein performing vertical processing further comprises:

generating a vertical reliability map that includes a reliability value for the vertical non-obstacle estimation.

3. The method of claim 2, wherein determining the vertical reliability map comprises:

determining a segment fitting error for a given segment based on a difference between the estimated linear model parameters and pre-determined linear model parameters;

determining a reliability value for the given segment by comparing the segment fitting error to a vertical estimation threshold; and

applying the reliability value for the given segment to at least one pixel in the given segment.

4. The method of claim 3, wherein the pre-determined linear model parameters are selected from among a plurality of road condition models, wherein the plurality of road condition models have a corresponding set of linear model parameters.

5. The method of claim 1, wherein performing horizontal processing comprises:

obtaining a depth histogram for at least one row of pixels of the depth map;

determining a terrain line from the depth histogram;

determining a horizontal non-obstacle estimation based on a distance of a depth value of at least one pixel from the terrain line; and

generating a horizontal reliability map that includes a reliability value for the horizontal non-obstacle estimation.

6. The method of claim 5, wherein generating the horizontal reliability map comprises:

determining whether the depth value of a given pixel is within a range of a mode of the depth histogram, wherein the given pixel has a high reliability value when the depth value of the given pixel is within the range of the mode of the depth histogram.

7. The method of claim 1, wherein combining the vertical non-obstacle estimation and the horizontal non-obstacle estimation comprises:

performing both the vertical processing and the horizontal processing in parallel; and

merging the vertical non-obstacle estimation and the horizontal non-obstacle estimation based on a vertical reliability map and a horizontal reliability map.

8. The method of claim 7, wherein a given pixel is identified as a non-obstacle area in the non-obstacle map where both the vertical reliability map and the horizontal reliability map are characterized by a high reliability value for the given pixel.

9. The method of claim 7, wherein a given pixel is identified as an obstacle area in the non-obstacle map where at least one of the vertical reliability map or the horizontal reliability map are characterized by a low reliability value for the given pixel.

10. The method of claim 7, wherein a given pixel is identified as a non-obstacle area or obstacle area in the non-obstacle map based on a coordinate of the given pixel where the vertical reliability map and the horizontal reliability map are characterized by different reliability values for the given pixel.

11. The method of claim 1, wherein combining the vertical non-obstacle estimation and the horizontal non-obstacle estimation comprises:

performing vertical processing of the depth map;

obtaining a vertical reliability map based on a model fitting confidence;

identifying reliable regions of the vertical reliability map as non-obstacle areas; and

performing horizontal processing on unreliable regions of the vertical reliability map to determine whether the unreliable regions are non-obstacle areas.

12. The method of claim 1, wherein combining the vertical non-obstacle estimation and the horizontal non-obstacle estimation comprises:

performing horizontal processing of the depth map;

obtaining a horizontal reliability map based on a depth histogram distance;

identifying reliable regions of the horizontal reliability map as non-obstacle areas; and

performing vertical processing on unreliable regions of the horizontal reliability map to determine whether the unreliable regions are non-obstacle areas.

13. The method of claim 1, wherein the non-obstacle map is used in identifying a region of interest used by at least one of an object detection algorithm or a lane detection algorithm.

14. An electronic device, comprising:

a processor;

memory in electronic communication with the processor; and

instructions stored in the memory, the instructions being executable by the processor to;

perform vertical processing of a depth map to determine a vertical non-obstacle estimation, wherein the instructions being executable by the processor to perform vertical processing comprise instructions being executable by the processor to;

divide the depth map into segments, wherein at least one segment is a portion of a column of pixels in the depth map; and

estimate linear model parameters for the at least one segment to determine the vertical non-obstacle estimation;

perform horizontal processing of the depth map to determine a horizontal non-obstacle estimation;

combine the vertical non-obstacle estimation and the horizontal non-obstacle estimation; and

generate a non-obstacle map based on the combination of the vertical and horizontal non-obstacle estimations.

15. The electronic device of claim 14, wherein the instructions being executable by the processor to perform vertical processing further comprise instructions being executable by the processor to:

generate a vertical reliability map that includes a reliability value for the vertical non-obstacle estimation.

16. The electronic device of claim 14, wherein the instructions being executable by the processor to perform horizontal processing comprise instructions being executable by the processor to:

obtain a depth histogram for at least one row of pixels of the depth map;

determine a terrain line from the depth histogram;

determine a horizontal non-obstacle estimation based on a distance of a depth value of at least one pixel from the terrain line; and

generate a horizontal reliability map that includes a reliability value for the horizontal non-obstacle estimation.

17. The electronic device of claim 14, wherein the instructions being executable by the processor to combine the vertical non-obstacle estimation and the horizontal non-obstacle estimation comprise instructions being executable by the processor to:

perform both the vertical processing and the horizontal processing in parallel; and

merge the vertical non-obstacle estimation and the horizontal non-obstacle estimation based on a vertical reliability map and a horizontal reliability map.

18. The electronic device of claim 14, wherein the instructions being executable by the processor to combine the vertical non-obstacle estimation and the horizontal non-obstacle estimation comprise instructions being executable by the processor to:

perform vertical processing of the depth map;

obtain a vertical reliability map based on a model fitting confidence;

identify reliable regions of the vertical reliability map as non-obstacle areas; and

perform horizontal processing on unreliable regions of the vertical reliability map to determine whether the unreliable regions are non-obstacle areas.

19. The electronic device of claim 14, wherein the instructions being executable by the processor to combine the vertical non-obstacle estimation and the horizontal non-obstacle estimation comprise instructions being executable by the processor to:

perform horizontal processing of the depth map;

obtain a horizontal reliability map based on a depth histogram distance;

identify reliable regions of the horizontal reliability map as non-obstacle areas; and

perform vertical processing on unreliable regions of the horizontal reliability map to determine whether the unreliable regions are non-obstacle areas.

20. An apparatus, comprising:

means for performing vertical processing of a depth map to determine a vertical non-obstacle estimation, wherein the means for performing vertical processing comprise;

means for dividing the depth map into segments, wherein at least one segment includes a number of pixels in a column; and

means for estimating linear model parameters for at least one segment to determine the vertical non-obstacle estimation;

means for performing horizontal processing of the depth map to determine a horizontal non-obstacle estimation;

means for combining the vertical non-obstacle estimation and the horizontal non-obstacle estimation; and

means for generating a non-obstacle map based on the combination of the vertical and horizontal non-obstacle estimations.

21. The apparatus of claim 20, wherein the means for performing vertical processing further comprise:

means for generating a vertical reliability map that includes a reliability value for the vertical non-obstacle estimation.

22. The apparatus of claim 20, wherein the means for performing horizontal processing comprise:

means for obtaining a depth histogram for at least one row of pixels of the depth map;

means for determining a terrain line from the depth histogram;

means for determining a horizontal non-obstacle estimation based on a distance of a depth value of at least one pixel from the terrain line; and

means for generating a horizontal reliability map that includes a reliability value for the horizontal non-obstacle estimation.

23. The apparatus of claim 20, wherein the means for combining the vertical non-obstacle estimation and the horizontal non-obstacle estimation comprise:

means for performing both the vertical processing and the horizontal processing in parallel; and

means for merging the vertical non-obstacle estimation and the horizontal non-obstacle estimation based on a vertical reliability map and a horizontal reliability map.

24. The apparatus of claim 20, wherein the means for combining the vertical non-obstacle estimation and the horizontal non-obstacle estimation comprise:

means for performing vertical processing of the depth map;

means for obtaining a vertical reliability map based on a model fitting confidence;

means for identifying reliable regions of the vertical reliability map as non-obstacle areas; and

means for performing horizontal processing on unreliable regions of the vertical reliability map to determine whether the unreliable regions are non-obstacle areas.

25. The apparatus of claim 20, wherein the means for combining the vertical non-obstacle estimation and the horizontal non-obstacle estimation comprise:

means for performing horizontal processing of the depth map;

means for obtaining a horizontal reliability map based on a depth histogram distance;

means for identifying reliable regions of the horizontal reliability map as non-obstacle areas; and

means for performing vertical processing on unreliable regions of the horizontal reliability map to determine whether the unreliable regions are non-obstacle areas.

26. A non-transitory tangible computer-readable medium comprising computer executable code, comprising:

code for causing an electronic device to perform vertical processing of a depth map to determine a vertical non-obstacle estimation, wherein the code for causing the electronic device to perform vertical processing comprises;

code for causing the electronic device to divide the depth map into segments, wherein at least one segment includes a number of pixels in a column; and

code for causing the electronic device to estimate linear model parameters for at least one segment to determine the vertical non-obstacle estimation;

code for causing the electronic device to perform horizontal processing of the depth map to determine a horizontal non-obstacle estimation;

code for causing the electronic device to combine the vertical non-obstacle estimation and the horizontal non-obstacle estimation; and

code for causing the electronic device to generate a non-obstacle map based on the combination of the vertical and horizontal non-obstacle estimations.

27. The non-transitory tangible computer-readable medium of claim 26, wherein the code for causing the electronic device to perform vertical processing further comprises:

code for causing the electronic device to generate a vertical reliability map that includes a reliability value for the vertical non-obstacle estimation.

28. The non-transitory tangible computer-readable medium of claim 26, wherein the code for causing the electronic device to perform horizontal processing further comprises:

code for causing the electronic device to obtain a depth histogram for at least one row of pixels of the depth map;

code for causing the electronic device to determine a terrain line from the depth histogram;

code for causing the electronic device to determine a horizontal non-obstacle estimation based on a distance of a depth value of at least one pixel from the terrain line; and

code for causing the electronic device to generate a horizontal reliability map that includes a reliability value for the horizontal non-obstacle estimation.

29. The non-transitory tangible computer-readable medium of claim 26, wherein the code for causing the electronic device to combine the vertical non-obstacle estimation and the horizontal non-obstacle estimation comprises:

code for causing the electronic device to perform both the vertical processing and the horizontal processing in parallel; and

code for causing the electronic device to merge the vertical non-obstacle estimation and the horizontal non-obstacle estimation based on a vertical reliability map and a horizontal reliability map.

30. The non-transitory tangible computer-readable medium of claim 26, wherein the code for causing the electronic device to combine the vertical non-obstacle estimation and the horizontal non-obstacle estimation comprises:

code for causing the electronic device to perform vertical processing of the depth map;

code for causing the electronic device to obtain a vertical reliability map based on a model fitting confidence;

code for causing the electronic device to identify reliable regions of the vertical reliability map as non-obstacle areas; and

code for causing the electronic device to perform horizontal processing on unreliable regions of the vertical reliability map to determine whether the unreliable regions are non-obstacle areas.