Method and system for in-vehicle image processing

• US 8,107,683 B2
• Filed: 03/28/2006
• Issued: 01/31/2012
• Est. Priority Date: 04/28/2005
• Status: Active Grant

• Current Assignee: Honda Motor Co., Ltd. (Honda Motor Company)
• Original Assignee: Honda Motor Co., Ltd. (Honda Motor Company)
• Inventors: Aoki, Tomoyoshi, Kobayashi, Sachio, Mori, Naoki, Nakamori, Takuma
• Primary Examiner(s): Perungavoor, Sath V
• Assistant Examiner(s): HEIDEMANN, JASON E
• Application Number:
  US11/918,344
• Publication Number:
  US 20080317288A1
• Time in Patent Office: 2,135 Days
• Field of Search: None
• US Class Current: 382/104
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30256 Lane; Road marking
• CPC Class Codes: G06T 7/12 Edge-based segmentation
• CPC Class Codes: G06V 10/44 Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road

Abstract

An image processing system and the like capable of improving recognition accuracy of a lane mark are provided. According to the image processing system of the present invention, a first processing unit (110) recognizes a lane mark candidate on the basis of the luminance of each pixel in a road image. Moreover, there are evaluated a first index which represents continuity of an edge of the lane mark candidate, a second index which represents conformance between the width of the lane mark candidate and a lane mark standard width, and a third index which represents uniformity of the luminance of pixels contained in the lane mark candidate. According to the first, second, and third indices, a second processing unit (120) recognizes a lane mark candidate most likely to be a true lane mark as a lane mark.

Claims

1. An image processing system comprising:

a processor;

a memory operatively coupled to the processor;

a first processing unit which recognizes a lane mark candidate on the basis of the luminance of each pixel in a road image captured by a vehicle-mounted imaging device;

a first evaluation unit which evaluates a first index which represents continuity of an edge of the lane mark candidate recognized by the first processing unit;

a second evaluation unit which evaluates a second index which represents conformance between the width of the lane mark candidate recognized by the first processing unit and a lane mark standard width;

a third evaluation unit which generates a luminance histogram of pixels contained in the lane mark candidate recognized by the first processing unit and then evaluates a third index which represents a variation in the luminance space of the histogram; and

a second processing unit which recognizes a lane mark candidate most likely to

be a true lane mark as a lane mark on the basis of the first, second, and third indices

evaluated by the first, second, and third evaluation units, respectively,

wherein the histogram is configured to determine at least one lane mark candidate in shaded regions.

2. An image processing system according to claim 1, wherein the first evaluation unit sets an approximate expression of the edge of the lane mark candidate on the basis of a plurality of edge points of the lane mark candidate recognized by the first processing unit and evaluates the first index according to an increasing function of a difference between the plurality of edge points and the approximate expression.

3. An image processing system according to claim 1, wherein the second evaluation unit selects a lane mark standard width closest to the width of the lane mark candidate recognized by the first processing unit among a plurality of lane mark standard widths and evaluates the second index according to an increasing function of a deviation between the width of the lane mark candidate and the selected lane mark standard width.

4. An image processing system according to claim 1, wherein the third evaluation unit divides the histogram so as to include only one peak, calculates indices each representing the variation in the luminance space of each divided histogram, and evaluates the third index on the basis of the indices if the histogram contains a plurality of peaks.

5. An image processing system according to claim 1, wherein the third evaluation unit calculates a standard deviation as the index representing the variation in the luminance space of the histogram and evaluates the standard deviation as the third index.

6. An image processing system according to claim 1, wherein, if the first index evaluated by the first evaluation unit falls outside a first allowable range or if the second index evaluated by the second evaluation unit falls outside a second allowable range, the second processing unit recognizes the lane mark with the importance of the third index being set to be higher than the importance of the first index and that of the second index.

7. An image processing method comprising:

a first processing step of recognizing a lane mark candidate on the basis of the luminance of each pixel in a road image captured by a vehicle-mounted imaging device;

a first evaluation step of evaluating a first index which represents continuity of an edge of the lane mark candidate recognized in the first processing step;

a second evaluation step of evaluating a second index which represents conformance between the width of the lane mark candidate recognized in the first processing step and a lane mark standard width;

a third evaluation step of generating a luminance histogram of pixels contained in the lane mark candidate recognized in the first processing step and evaluating a third index which represents a variation in the luminance space of the histogram; and

a second processing step of recognizing a lane mark candidate most likely to be a true lane mark as a lane mark on the basis of the first, second, and third indices evaluated by in the first, second, and third evaluation units steps, respectively,

wherein the histogram is configured to determine at least one lane mark candidate in shaded regions.

8. An image processing program embedded in a non-transitory machine-readable medium, the image processing program causing a computer to perform:

a first processing function of recognizing a lane mark candidate on the basis of the luminance of each pixel in a road image captured by a vehicle-mounted imaging device;

a first evaluation function of evaluating a first index which represents continuity of an edge of the lane mark candidate recognized by the first processing function;

a second evaluation function of evaluating a second index which represents conformance between the width of the lane mark candidate recognized by the first processing function and a lane mark standard width;

a third evaluation function of generating a luminance histogram of pixels contained in the lane mark candidate recognized by the first processing function and evaluating a third index which represents a variation in the luminance space of the histogram; and

a second processing function of recognizing a lane mark candidate most likely to be a true lane mark as a lane mark on the basis of the first, second, and third indices evaluated by the first, second, and third evaluation functions, respectively

wherein the histogram is configured to determine at least one lane mark candidate in shaded regions.

9. The image processing program according to claim 8, wherein part of all of the program is downloaded to an in-vehicle computer.

10. The image processing program according to claim 8, wherein the program is hosted on a server.

11. A vehicle equipped with:

an imaging device;

an image processing system which performs image processing on the basis of a road image captured by the imaging device; and

a vehicle running condition control system which controls a running condition of the vehicle on the basis of a result of the image processing performed by the image processing system,

wherein the image processing system includes;

a first processing unit which recognizes a lane mark candidate on the basis of the luminance of each pixel in the road image captured by the imaging device;

a first evaluation unit which evaluates a first index which represents continuity of an edge of the lane mark candidate recognized by the first processing unit;

a second evaluation unit which evaluates a second index which represents conformance between the width of the lane mark candidate recognized by the first processing unit and a lane mark standard width;

a third evaluation unit which generates a luminance histogram of pixels contained in the lane mark candidate recognized by the first processing unit and then evaluates a third index which represents a variation in the luminance space of the histogram; and

a second processing unit which recognizes a lane mark candidate most likely to be a true lane mark as a lane mark on the basis of the first, second, and third indices evaluated by the first, second, and third evaluation units, respectively,

wherein the histogram is configured to determine at least one lane mark candidate in shaded regions.

12. A vehicle according to claim 11, wherein the first evaluation unit sets an approximate expression of the edge of the lane mark candidate on the basis of a plurality of edge points of the lane mark candidate recognized by the first processing unit and evaluates the first index according to an increasing function of a difference between the plurality of edge points and the approximate expression.

13. A vehicle according to claim 11, wherein the second evaluation unit selects a lane mark standard width closest to the width of the lane mark candidate recognized by the first processing unit among a plurality of lane mark standard widths and evaluates the second index according to an increasing function of a deviation between the width of the lane mark candidate and the selected lane mark standard width.

14. A vehicle according to claim 11, wherein the third evaluation unit divides the histogram so as to include only one peak, calculates indices each representing the variation in the luminance space of each divided histogram, and evaluates the third index on the basis of the indices if the histogram contains a plurality of peaks.

15. A vehicle according to claim 11, wherein the third evaluation unit calculates a standard deviation as an index representing the variation in the luminance space of the histogram and evaluates the standard deviation as the third index.

16. A vehicle according to claim 11, wherein, if the first index evaluated by the first evaluation unit falls outside a first allowable range or if the second index evaluated by the second evaluation unit falls outside a second allowable range, the second processing unit recognizes the lane mark with the importance of the third index being set to be higher than the importance of the first index and that of the second index.