Shadow Removal in an Image Captured by a Vehicle-Based Camera for Clear Path Detection

• US 20120008021A1
• Filed: 07/06/2010
• Published: 01/12/2012
• Est. Priority Date: 07/06/2010
• Status: Active Grant

• Current Assignee: GM Global Technology Operations LLC (General Motors Company)
• Original Assignee: GM Global Technology Operations Incorporated (Motors Liquidation Co. GUC Trust)
• Inventors: Zhang, Wende, Wu, Qi, Bhagavatula, Vijayakumar
• Granted Patent: US 8,294,794 B2
• Time in Patent Office: Days
• US Class Current: 348/251
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 5/90 Dynamic range modification of images or parts thereof
• CPC Class Codes: G06T 7/12 Edge-based segmentation
• CPC Class Codes: G06V 10/273 removing elements interfering with the pattern to be recognised
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle

Abstract

A method for is provided for creating a shadow-reduced image from a captured image for distinguishing a clear path of travel. Each pixel of a captured input image is plotted according to a two dimensional logarithmic graph. A specific color set relating to an associated color value of a clear path. A linear illumination-invariant axis is determined as a function of the specific color set. An illumination direction for the linear illumination-invariant axis is determined. A log-chromaticity value of each plotted pixel of the specific color set is projected on the axis. Edges in the input image and the illumination-invariant image domain are identified. The identified edges of the input image are compared to identify edges in the illumination-invariant image domain. A determination is made whether a shadow edge is present in response to comparing the edges. A shadow-reduced image is generated for scene analysis by a vehicle vision-based system.

Claims

1. A method for creating a shadow-reduced image from a captured image for distinguishing a clear path of travel, the method comprising the steps of:

(a) capturing an input image of a scene by an image capture device;

(b) plotting each pixel of the captured input image according to a two dimensional logarithmic graph, each pixel being represented by a color value of one of a plurality of color sets in the logarithmic graph;

(c) selecting a specific color set in the logarithmic graph, the color set relating to associated color values of the road;

(d) determining a linear illumination-invariant axis as a function of the specific color set;

(e) determining an illumination direction for the linear illumination-invariant axis, the linear illumination-invariant axis extending in a direction that is substantially orthogonal to the illumination direction of the specific color set;

(e) projecting a log-chromaticity value of each plotted pixel of the specific color set on the linear illumination-invariant axis, wherein each plotted pixel on the linear illumination-invariant axis represents a color value of the respective pixels of the image mapped to an illumination invariant image domain;

(f) identifying edges in the input image;

(g) identifying edges in the illumination-invariant image domain;

(h) comparing the identified edges of the input image to identify edges in the illumination invariant image domain;

(i) determining whether a shadow edge is present in response to an edge identified in the input image and an absence of a correlating edge in the illumination-invariant image domain; and

(g) generating a shadow-reduced image for scene analysis by a vehicle vision-based system.

2. The method of claim 1 further comprising the steps of:

determining a vanishing point in the image;

determining a vanishing line extending horizontally through the vanishing point in the image;

identifying a target region below the vanishing line; and

performing steps (c)-(g) only on the target region.

3. The method of claim 1 wherein other color sets are projected onto the linear illumination-invariant axis, and wherein step (d) further includes selecting the linear illumination-invariant axis that provides a substantial separation of distance between the specific color set and other color sets projected onto the illumination-invariant axis.

4. The method of claim 3 wherein the selecting the linear illumination-invariant axis substantially minimizes a distance between the projected color values within the specific color set.

5. The method of claim 4 wherein the step of selecting the linear illumination-invariant axis includes selecting a linear illumination-invariant axis that scatters the projected color values of other color sets.

6. The method of claim 4 wherein the selecting the linear illumination-invariant axis includes selecting a linear illumination-invariant axis that provides a substantial distance between the specific color set and other color sets.

7. The method of claim 3 further comprising the steps of:

determining a vanishing point in the image;

determining a vanishing line extending horizontally through the vanishing point in the image;

identifying a target region below the vanishing line; and

performing steps (c)-(g) only on the identified target region.

8. The method of claim 3 wherein step (d) further includes selecting the linear illumination-invariant axis that substantially clusters the color values of the specific color set and substantially scatters the colors values of the other color sets.

9. The method of claim 8 further comprising the steps of:

determining vanishing point in the image;

determining a vanishing line extending through the vanishing point in the image;

identifying a target region below the vanishing line; and

performing steps (c)-(g) on the identified target region.

10. The method of claim 1 wherein the logarithmic graph includes a logarithmic blue-green axis and a logarithmic red-green axis.

11. The method of claim 1 wherein identifying edges in the input image and the illumination-invariant image domain further comprises the steps of:

determining a directional gradient from the input image;

determining a directional gradient from the illumination-invariant image domain;

comparing the directional gradient of the input image and the directional gradient of the illumination-invariant image domain to at least one threshold for determining whether a shadow edge is present.

12. The method of claim 11 wherein comparing the directional gradient of the input image and the directional gradient of the illumination-invariant image domain to at least one threshold further comprises:

determining whether the directional gradient of the input image is greater than a first threshold;

determining whether the directional gradient of the illumination-invariant image domain is less a second threshold.

13. The method of claim 11 wherein comparing the directional gradient of the input image and the directional gradient of the illumination-invariant image domain to at least one threshold further comprises:

calculating a gradient difference between the directional gradient of the input image and the directional gradient of the illumination-invariant image domain;

comparing the gradient difference to the at least one threshold.

14. The method of claim 13 wherein calculating a gradient difference comprises the following steps:

determining a gradient norm of the input image;

determining a gradient norm of the illumination-invariant image domain;

calculating the gradient difference by subtracting the gradient norm of the illumination-invariant image domain from the gradient norm of the input image.

15. The method of claim 1 wherein the linear illumination-invariant axis is selected utilizing a machine learning technique.

16. The method of claim 15 wherein the machine learning technique includes a linear discriminant analysis.