Pixel-based texture-less clear path detection

• US 8,634,593 B2
• Filed: 05/29/2009
• Issued: 01/21/2014
• Est. Priority Date: 04/24/2008
• Status: Active Grant

• Current Assignee: GM Global Technology Operations LLC (General Motors Company)
• Original Assignee: GM Global Technology Operations LLC (General Motors Company)
• Inventors: Zhang, Wende, Naik, Sanjeev M.
• Primary Examiner(s): Hung, Yubin
• Application Number:
  US12/474,594
• Publication Number:
  US 20090295917A1
• Time in Patent Office: 1,698 Days
• Field of Search: 382/100, 382/104, 382217-221, 382/228, 700/259, 701/223, 701300-302
• US Class Current: 382/104
• CPC Class Codes: B60W 2420/403 Image sensing, e.g. optical camera
• CPC Class Codes: G06V 10/56 relating to colour
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
• CPC Class Codes: G08G 1/165 for passive traffic, e.g. including static obstacles, trees
• CPC Class Codes: G08G 1/166 for active traffic, e.g. moving vehicles, pedestrians, bikes
• CPC Class Codes: G08G 1/167 Driving aids for lane monitoring, lane changing, e.g. blind spot detection

Abstract

A method for detecting a clear path of travel for a vehicle utilizing analysis of a plurality of images generated by a camera device located upon the vehicle includes monitoring the images wherein each image comprises a plurality of pixels, utilizing a texture-less processing scheme to analyze the images including filtering the image to separate portions of the image indicating a clear surface upon which a potential road surface can be estimated from other portions of the image that do not indicate a potential road surface. The clear path is determined based upon the clear surface and the clear path is utilized to operate the vehicle.

Claims

1. Method for detecting a clear path of travel for a vehicle utilizing analysis of a plurality of images generated by a camera device located upon the vehicle, the method comprising:

monitoring the images, each image comprising a plurality of pixels;

utilizing a texture-less processing scheme to analyze the images comprising, for each image generated by the camera device;

filtering the generated image in two parallel steps for determining a first filtered image in one of the steps and a second filtered image in the other one of the steps, wherein determining the first and second filtered images includes

determining the first filtered image by filtering an area below a horizon line on the generated image from an area above the horizon line, the area below the horizon line on the generated image consisting of analyzed pixels, and

determining the second filtered image by filtering the pixels in the generated image comprising

for each pixel in the generated image, determining a difference between each of a plurality of color intensities and respective ones of a plurality of color distribution means each different from the other color distribution means, wherein each color distribution mean comprises an average color intensity distribution associated with the clear path of travel for the respective color intensity, and

selecting only pixels in the generated image for inclusion in the second filtered image that comprise each determined difference being less than respective ones of a plurality of color intensity thresholds, wherein each determined difference greater than the respective color intensity threshold is indicative of a variance in visual intensity not common to the clear path of travel and each determined difference less than the respective color intensity threshold is indicative of the variance in visual intensity common to the clear path of travel;

generating a fused image of the first filtered image and the second filtered image determined in the two parallel steps;

determining a third filtered image subsequent to generating the fused image based on differencing the generated fused image and a generated fused image of an immediately preceding iteration to only include the pixels of the fused image indicating a clear surface upon which a potential road surface can be estimated while excluding the other pixels of the fused image that do not indicate a potential road surface;

subsequent to determining the third filtered image, determining a fourth filtered image by filtering the third filtered image based upon identifying pixels in the third filtered image representing edges, wherein only the identified pixels in the third filtered image representing edges are included while the other pixels of the third filtered image that do not represent edges are excluded; and

determining a clear path based upon the fused image and the included pixels of at least one of the third and fourth filtered images subsequent to generating the fused image indicating the clear surface.

2. The method of claim 1, wherein a first color intensity is a red intensity, a second color intensity is a green intensity, and a third color intensity is blue intensity and wherein each pixel comprises a first, second, and third color intensity, and filtering the pixel when the difference between a red intensity of the pixel and a red color distribution mean is greater than a red intensity threshold, and the difference between a green intensity of the pixel and a green color distribution mean is greater than a green intensity threshold, and the difference between a blue intensity of the pixel and a blue color distribution mean is greater than a blue intensity threshold.

3. The method of claim 1, wherein differencing the generated fused image of the immediately preceding iteration comprises:

fusing a previous first filtered image and a previous second filtered image determined in the two parallel steps of the immediately preceding iteration.

4. The method of claim 3, wherein the difference is determined by subtracting color intensities associated with a pixel from corresponding color intensities to determine a color intensity difference.

5. The method of claim 3, further comprising:

filtering the individual pixels comprising excluding pixels when the difference between individual pixels on the generated fused image and corresponding individual pixels on the generated fused image of the immediately preceding iteration exceeds a predetermined threshold.

6. The method of claim 3, wherein the generated fused image of the immediately preceding iteration is adjusted based upon vehicle motion.

7. The method of claim 1, wherein filtering the third filtered image based upon identifying analyzed pixels representing edges comprises applying a Sobel filter.

8. The method of claim 1, wherein generating the fused image of the first and second filtered images comprises:

determining pixel locations included on each of the first and second filtered images;

comparing pixels at the pixel locations with corresponding pixel locations on the first and second filtered images; and

including the pixel when each of the the first and second filtered images contains the pixel at the corresponding pixel location.

9. Method for detecting a clear path of travel for a vehicle utilizing analysis of a plurality of images generated by a camera device located upon the vehicle, the method comprising:

monitoring the images;

utilizing a texture-less control scheme to analyze the images comprising, for each image generated by the camera device;

filtering the generated image in two parallel steps for determining a first filtered image in one of the steps and a second filtered image in the other one of the steps, wherein determining the first and second filtered images includes

determining the first filtered image by filtering an area below a horizon line on the generated image from an area above the horizon line, the area below the horizon line on the generated image consisting of analyzed pixels, and

determining the second filtered image by filtering pixels in the generated image based upon comparing a color intensity of the pixels to a color intensity threshold based upon a color distribution mean, comprising,

for each pixel in the generated image, determining a difference between each of a plurality of color intensities and respective ones of a plurality of color distribution means each different from the other color distribution means, wherein each color distribution mean comprises an average color intensity distribution associated with the clear path of travel for the respective color intensity, and

selecting only pixels in the generated image for inclusion in the second filtered image that comprise each determined difference being less than respective ones of a plurality of color intensity thresholds, wherein each determined difference greater than the respective color intensity threshold is indicative of a variance in visual intensity not common to the clear path of travel and each determined difference less than the respective color intensity threshold is indicative of the variance in visual intensity common to the clear path of travel

generating a fused image of the first filtered image and the second filtered image determined in the two parallel steps,

determining a third filtered image based upon differencing sequential fused images,

subsequent to determining the third filtered image, determining a fourth filtered image based upon identifying pixels in the third filtered image representing edges,

determining a clear surface based upon the fused image and the third and fourth filtered images and determining a clear path based upon the clear surface, and

utilizing the clear path to operate the vehicle.

10. Apparatus for detecting a clear path of travel for a vehicle utilizing analysis of a plurality of images generated by a camera device located upon the vehicle, the apparatus comprising:

the camera device; and

a control module monitoring the images from the camera device and monitoring the images, each image comprising a plurality of pixels, utilizing a texture-less processing scheme to analyze the images comprising

filtering the generated image in two parallel steps for determining a first filtered image in one of the steps and a second filtered image in the other one of the steps, the determining the first and second filtered images includes

determining the first filtered image by filtering an area below a horizon line on the generated image from an area above the horizon line, the area below the horizon line on the generated image consisting of analyzed pixels, and

determining the second filtered image by filtering the pixels in the generated image comprising

for each pixel in the generated image, determining a difference between each of a plurality of color intensities and respective ones of a plurality of color distribution means each different from the other color distribution means, wherein each color distribution mean comprises an average color intensity distribution associated with the clear path of travel for the respective color intensity, and

selecting only pixels in the generated image for inclusion in the second filtered image that comprise each determined difference being less than respective ones of a plurality of color intensity thresholds , wherein each determined difference greater than the respective color intensity threshold is indicative of a variance in visual intensity not common to the clear path of travel and each determined difference less than the respective color intensity threshold is indicative of the variance in visual intensity common to the clear path of travel,

generating a fused image of the first filtered image and the second filtered image determined in the two parallel steps,

determining a third filtered image subsequent to generating the fused image based on differencing the generated fused image and a generated fused image of an immediately preceding iteration to only include the pixels of the fused image indicating a clear surface upon which a potential road surface can be estimated while excluding the other pixels of the fused image that do not indicate a potential road surface,

subsequent to determining the third filtered image, determining a fourth filtered image by filtering the third filtered image based upon identifying pixels in the third filtered image representing edges, wherein only the identified pixels in the third filtered image representing edges are included while the other pixels of the third filtered image that do not represent edges are excluded, and

determining a clear path based upon the included pixels of each fused image and the included pixels of at least one of the third and fourth filtered images subsequent to generating the fused image indicating the clear surface.