Unstructured road boundary detection

• US 9,971,944 B2
• Filed: 08/23/2013
• Issued: 05/15/2018
• Est. Priority Date: 08/23/2013
• Status: Active Grant

• Current Assignee: Harman International Industries Incorporated (Samsung Electronics Co. Ltd.)
• Original Assignee: Harman International Industries Incorporated (Samsung Electronics Co. Ltd.)
• Inventors: Shao, Xisheng
• Primary Examiner(s): Carter, Aaron W
• Application Number:
  US14/909,951
• Publication Number:
  US 20160171314A1
• Time in Patent Office: 1,726 Days
• Field of Search: 382104, 382103, 382153, 382159, 382199, 180167, 180169, 315 82, 340468, 3404572, 340930, 340933, 342 70, 348113, 348116, 348119, 348148, 348439, 348 E7085, 356437, 701408, 703 2
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/30256 Lane; Road marking
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/12 Edge-based segmentation
• CPC Class Codes: G06T 7/136 involving thresholding
• CPC Class Codes: G06T 7/143 involving probabilistic approaches, e.g. Markov random field [MRF] modelling
• CPC Class Codes: G06T 7/181 involving edge growing; involving edge linking
• CPC Class Codes: G06V 10/267 by performing operations on regions, e.g. growing, shrinking or watersheds
• CPC Class Codes: G06V 10/56 relating to colour
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road

Abstract

A method for determining a road region may include: obtaining a color image; selecting a candidate road region within the color image according to a road model; identifying a seed pixel from the candidate road region; obtaining a brightness threshold and a color threshold, where the brightness threshold and the color threshold are determined according to brightness distances and color distances from pixels in the candidate road region to the seed pixel; and performing road segmentation by determining whether the pixels in the candidate road region belong to a road region based on the brightness threshold and the color threshold.

Claims

1. A method for detecting a road region, comprising:

obtaining, via a storage device, a color image;

selecting a candidate road region within the color image according to a road model;

identifying a seed pixel from the candidate road region;

determining, via a processing device, a brightness threshold and a color threshold according to brightness distances and color distances from pixels in the candidate road region to the seed pixel; and

determining that the pixels in the candidate road region belong to the road region based on the brightness threshold and the color threshold.

2. The method according to claim 1, further comprising preprocessing the color image, by:

sampling the color image using a sampling frame that has a predetermined height and a predetermined width;

calculating norms of vectors, which respectively correspond to pixels in the sampling frame, where each of the vectors represents a line pointing from an original point to a point corresponding to a pixel in a color space;

identifying a first predetermined number of vectors by filtering out a first number of vectors having maximum norms of vectors and a second number of vectors having minimum norms of vectors;

obtaining a weighted average vector, where an m^th component of the weighted average vector is obtained by calculating weighted average of m^th components of the first predetermined number of vectors;

calculating norms of differences between the first predetermined number of vectors and the weighted average vector;

identifying, from the first predetermined number of vectors, a second predetermined number of vectors having minimum norms of differences with the weighted average vector; and

obtaining median vectors based on the second predetermined number of vectors to obtain a preprocessed color image, where the candidate road region is selected within the preprocessed color image.

3. The method according to claim 2, where a weight value for an m^th component of the first predetermined number of vectors is determined based on a distance between a pixel corresponding to the m^th component of the first predetermined number of vectors and a center position of the sampling frame.

4. The method according to claim 1, wherein a brightness threshold T_achrom and a color threshold T_chrom are obtained, respectively, as follows:

5. The method according to claim 4, where determining that the pixels in the candidate road region belong to a road region comprises:

obtaining a spatial distance between a pixel of bright color and the seed pixel;

comparing the spatial distance with the color threshold; and

determining whether the pixel of bright color belongs to a road region based on the comparison;

where the spatial distance between the pixel and the seed pixel is calculated as follows;

6. The method according to claim 4, wherein determining that the pixels in the candidate road region belong to a road region comprises:

obtaining a brightness distance between a pixel of dim color and the seed pixel;

comparing the brightness distance with the brightness threshold; and

determining whether the pixel of dim color belongs to a road region according to the comparison;

where the brightness distance between the pixel j of dim color and the seed pixel i is calculated as follows;

d_i=I_i−

I_j; and

I_i and I_j represent brightness components of the seed pixel i and the pixel j in HSI color space, respectively.

7. The method according to claim 1, further comprising determining whether the road model matches an unstructured road which is included in the color image based on a road segmentation result.

8. The method according to claim 7, wherein determining whether the road model matches the unstructured road based on the road segmentation result comprises:

calculating a percentage of pixels in a region determined according to the road segmentation result coinciding with pixels in a corresponding region in a color image; and

if the percentage is greater than a predetermined value, determining that the selected road model matches the unstructured road.

9. A system for detecting a road region, comprising:

a camera for obtaining a color image; and

a processing device configured to;

select a candidate road region within the color image according to a road model;

identify a seed pixel from the candidate road region;

determine a brightness threshold and a color threshold according to brightness distances and color distances from pixels in the candidate road region to the seed pixel; and

perform road segmentation by determining that the pixels in the candidate road region belong to the road region based on the brightness threshold and the color threshold.

10. The system according to claim 9, wherein the processing device is further configured to:

sample the color image using a sampling frame that has a predetermined height and a predetermined width;

calculate norms of vectors, which respectively correspond to pixels in the sampling frame, where each of the vectors represents a line pointing from an original point to a point corresponding to a pixel in a color space;

identify a first predetermined number of vectors by filtering out a first number of vectors having maximum norms of vectors and a second number of vectors having minimum norms of vectors;

obtain a weighted average vector, where an m^th component of the weighted average vector is obtained by calculating weighted average of m^th components of the first predetermined number of vectors;

calculate norms of differences between the first predetermined number of vectors and the weighted average vector;

identify, from the first predetermined number of vectors, a second predetermined number of vectors having minimum norms of differences with the weighted average vector; and

obtain median vectors based on the second predetermined number of vectors to obtain a preprocessed color image, where the candidate road region is selected within the preprocessed color image.

11. The system according to claim 10, where a weight value for an m^th component of the first predetermined number of vectors is determined based on a distance between a pixel corresponding to the m^th component of the first predetermined number of vectors and a center position of the sampling frame.

12. The system according to claim 9, wherein a brightness threshold T_achrom and a color threshold T_chrom are obtained, respectively, as follows:

13. The system according to claim 12, where for determining whether a pixel j with bright color belongs to a road region, the processing device is further configured to:

obtain a spatial distance between a pixel of bright color and the seed pixel;

compare the spatial distance with the color threshold; and

determine whether the pixel of bright color belongs to a road region based on the comparison;

where the spatial distance between the pixel and the seed pixel is calculated as follows;

14. The system according to claim 12, where for determining whether a pixel j with dim color belongs to a road region, the processing device is further configured to:

obtain a brightness distance between a pixel of dim color and the seed pixel;

compare the brightness distance with the brightness threshold; and

determine whether the pixel of dim color belongs to a road region according to the comparison;

where the brightness distance between the pixel j of dim color and the seed pixel i is calculated as follows;

d_i=I_i−

I_j; and

I_i and I_j represent brightness components of the seed pixel i and the pixel j in HSI color space, respectively.

15. The system according to claim 9, wherein the processing device is further configured to determine whether the road model matches an unstructured road which is included in the color image based on a road segmentation result.

16. The system according to claim 15, wherein determining whether the road model matches the unstructured road based on the road segmentation result, the processing device is further configured to:

calculate a percentage of pixels in a region determined according to the road segmentation result coinciding with pixels in a corresponding region in a color image; and

if the percentage is greater than a predetermined value, determining that the selected road model matches the unstructured road.