Vehicle and road sign recognition device

• US 20100074469A1
• Filed: 05/18/2006
• Published: 03/25/2010
• Est. Priority Date: 06/03/2005
• Status: Active Grant

• Current Assignee: Honda Motor Co., Ltd. (Honda Motor Company)
• Original Assignee: Honda Motor Co., Ltd. (Honda Motor Company)
• Inventors: Mori, Naoki, Aoki, Tomoyoshi, Nakamori, Takuma, Kobayashi, Sachio
• Granted Patent: US 8,036,427 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06V 10/56 relating to colour
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road

Abstract

The present invention includes: image capturing means (2) which captures a color image of a road via imaging means (7); area extraction means (6) which extracts areas having a similar color feature value from the captured color image of the road; white balance processing means (5) which performs, for each of the extracted areas, a white balance process of correcting each pixel data in the extracted area so as to reduce the degree of divergence in the level between color components according to a level balance between the color components of the pixel data in the extracted area; and road sign detection means (6) which detects a road sign on the road from the color image subjected to the white balance process. Thus, even if the road illumination state is partially different, it is possible to accurately detect a road sign such as a lane mark from the captured color image of the road via the imaging means such as a camera.

Claims

1. A vehicle comprising:

an imaging means;

an image capturing means which captures a color image of a road via the imaging means;

an area extraction means which extracts areas having a similar color feature value from the captured color image of the road captured by the image capturing means;

a white balance processing means which performs, for each of the extracted areas extracted by the area extraction means, a white balance process of correcting each pixel data in the extracted area so as to reduce the degree of divergence in the level between color components according to a level balance between the color components of the pixel data in the extracted area; and

a road sign detection means which detects a road sign on the road from the color image subjected to the white balance process.

2. A vehicle according to claim 1, wherein:

the color components of each pixel data of the color image consist of R, G, and B values; and

the white balance processing means performs processing of multiplying the R value by a first correction factor calculated based on a ratio between a general level of the R value in the extracted area and a general level of a luminance value calculated from the color components in the extracted area, multiplying the G value by a second correction factor calculated based on a ratio between a general level of the G value in the extracted area and a general level of the luminance value calculated from the color components in the extracted area, and multiplying the B value by a third correction factor calculated based on a ratio between a general level of the B value in the extracted area and a general level of the luminance value calculated from the color components in the extracted area for each pixel data in the extracted area in the white balance process.

3. A vehicle according to claim 2, wherein:

the white balance processing means uses a sum value of R values of all pixels in the extracted area as the general level of the R value in the extracted area and uses a sum value of luminance values of all pixels in the extracted area as the general level of the luminance value calculated from the color components in the extracted area in the case where calculating the first correction factor;

the white balance processing means uses a sum value of G values of all pixels in the extracted area as the general level of the G value in the extracted area and uses a sum value of luminance values of all pixels in the extracted area as the general level of the luminance value calculated from the color components in the extracted area in the case where calculating the second correction factor; and

the white balance processing means uses a sum value of B values of all pixels in the extracted area as the general level of the B value in the extracted area and uses a sum value of luminance values of all pixels in the extracted area as the general level of the luminance value calculated from the color components in the extracted area in the case where calculating the third correction factor.

4. A vehicle according to claim 2, wherein:

the white balance processing means uses a mean value of R values of all pixels in the extracted area as the general level of the R value in the extracted area and uses a mean value of luminance values of all pixels in the extracted area as the general level of the luminance value calculated from the color components in the extracted area in the case where calculating the first correction factor;

the white balance processing means uses a mean value of G values of all pixels in the extracted area as the general level of the G value in the extracted area and uses a mean value of luminance values of all pixels in the extracted area as the general level of the luminance value calculated from the color components in the extracted area in the case where calculating the second correction factor; and

the white balance processing means uses a mean value of B values of all pixels in the extracted area as the general level of the B value in the extracted area and uses a mean value of luminance values of all pixels in the extracted area as the general level of the luminance value calculated from the color components in the extracted area in the case where calculating the third correction factor.

5. A vehicle comprising:

an imaging means;

an image capturing means which captures a color image of a road via the imaging means;

an area extraction means which extracts areas having a similar color feature value from the captured color image of the road captured by the image capturing means;

a road range specifying means which specifies a road range that the road occupies in the color image;

a white balance processing means which performs, for each of the extracted areas extracted by the area extraction means, a white balance process of correcting each pixel data in the extracted area so as to reduce the degree of divergence in the level between color components according to a level balance between the color components of the pixel data in the road range of the extracted area; and

a road sign detection means which detects a road sign on the road from the color image subjected to the white balance process.

6. A vehicle according to claim 5, wherein:

the color components of each pixel data of the color image consist of R, G, and B values; and

the white balance processing means performs processing of multiplying the R value by a first correction factor calculated based on a ratio between a general level of the R value in the road range of the extracted area and a general level of a luminance value calculated from the color components in the road range of the extracted area, multiplying the G value by a second correction factor calculated based on a ratio between a general level of the G value in the road range of the extracted area and a general level of the luminance value calculated from the color components in the road range of the extracted area, and multiplying the B value by a third correction factor calculated based on a ratio between a general level of the B value in the road range of the extracted area and a general level of the luminance value calculated from the color components in the road range of the extracted area for each pixel data in the extracted area in the white balance process.

7. A vehicle according to claim 6, wherein:

the white balance processing means uses a sum value of R values of all pixels in the road range of the extracted area as the general level of the R value in the road range of the extracted area and uses a sum value of luminance values of all pixels in the road range of the extracted area as the general level of the luminance value calculated from the color components in the road range of the extracted area in the case where calculating the first correction factor;

the white balance processing means uses a sum value of G values of all pixels in the road range of the extracted area as the general level of the G value in the road range of the extracted area and uses a sum value of luminance values of all pixels in the road range of the extracted area as the general level of the luminance value calculated from the color components in the road range of the extracted area in the case where calculating the second correction factor; and

the white balance processing means uses a sum value of B values of all pixels in the road range of the extracted area as the general level of the B value in the road range of the extracted area and uses a sum value of luminance values of all pixels in the road range of the extracted area as the general level of the luminance value calculated from the color components in the road range of the extracted area in the case where calculating the third correction factor.

8. A vehicle according to claim 6, wherein:

the white balance processing means uses a mean value of R values of all pixels in the road range of the extracted area as the general level of the R value in the road range of the extracted area and uses a mean value of luminance values of all pixels in the road range of the extracted area as the general level of the luminance value calculated from the color components in the road range of the extracted area in the case where calculating the first correction factor;

the white balance processing means uses a mean value of G values of all pixels in the road range of the extracted area as the general level of the G value in the road range of the extracted area and uses a mean value of luminance values of all pixels in the road range of the extracted area as the general level of the luminance value calculated from the color components in the road range of the extracted area in the case where calculating the second correction factor; and

the white balance processing means uses a mean value of B values of all pixels in the road range of the extracted area as the general level of the B value in the road range of the extracted area and uses a mean value of luminance values of all pixels in the road range of the extracted area as the general level of the luminance value calculated from the color components in the road range of the extracted area in the case where calculating the third correction factor.

9. A road sign recognition device comprising:

an image capturing means which captures a color image of a road via an imaging means;

an area extraction means which extracts areas having a similar color feature value from the captured color image of the road captured by the image capturing means;

a white balance processing means which performs, for each of the extracted areas extracted by the area extraction means, a white balance process of correcting each pixel data in the extracted area so as to reduce the degree of divergence in the level between color components according to a level balance between the color components of the pixel data in the extracted area; and

a road sign detection means which detects a road sign on the road from the color image subjected to the white balance process.

10. A road sign recognition device according to claim 9, wherein:

the color components of each pixel data of the color image consist of R, G, and B values; and

the white balance processing means performs processing of multiplying the R value by a first correction factor calculated based on a ratio between a general level of the R value in the extracted area and a general level of a luminance value calculated from the color components in the extracted area, multiplying the G value by a second correction factor calculated based on a ratio between a general level of the G value in the extracted area and a general level of the luminance value calculated from the color components in the extracted area, and multiplying the B value by a third correction factor calculated based on a ratio between a general level of the B value in the extracted area and a general level of the luminance value calculated from the color components in the extracted area for each pixel data in the extracted area in the white balance process.

11. A road sign recognition device according to claim 10, wherein:

the white balance processing means uses a sum value of R values of all pixels in the extracted area as the general level of the R value in the extracted area and uses a sum value of luminance values of all pixels in the extracted area as the general level of the luminance value calculated from the color components in the extracted area in the case where calculating the first correction factor;

the white balance processing means uses a sum value of G values of all pixels in the extracted area as the general level of the G value in the extracted area and uses a sum value of luminance values of all pixels in the extracted area as the general level of the luminance value calculated from the color components in the extracted area in the case where calculating the second correction factor; and

the white balance processing means uses a sum value of B values of all pixels in the extracted area as the general level of the B value in the extracted area and uses a sum value of luminance values of all pixels in the extracted area as the general level of the luminance value calculated from the color components in the extracted area in the case where calculating the third correction factor.

12. A road sign recognition device according to claim 10, wherein:

the white balance processing means uses a mean value of R values of all pixels in the extracted area as the general level of the R value in the extracted area and uses a mean value of luminance values of all pixels in the extracted area as the general level of the luminance value calculated from the color components in the extracted area in the case where calculating the first correction factor;

the white balance processing means uses a mean value of G values of all pixels in the extracted area as the general level of the G value in the extracted area and uses a mean value of luminance values of all pixels in the extracted area as the general level of the luminance value calculated from the color components in the extracted area in the case where calculating the second correction factor; and

the white balance processing means uses a mean value of B values of all pixels in the extracted area as the general level of the B value in the extracted area and uses a mean value of luminance values of all pixels in the extracted area as the general level of the luminance value calculated from the color components in the extracted area in the case where calculating the third correction factor.

13. A road sign recognition device comprising:

an image capturing means which captures a color image of a road via an imaging means;

an area extraction means which extracts areas having a similar color feature value from the captured color image of the road captured by the image capturing means;

a road range specifying means which specifies a road range that the road occupies in the color image;

a white balance processing means which performs, for each of the extracted areas extracted by the area extraction means, a white balance process of correcting each pixel data in the extracted area so as to reduce the degree of divergence in the level between color components according to a level balance between the color components of the pixel data in road range of the extracted area; and

a road sign detection means which detects a road sign on the road from the color image subjected to the white balance process.

14. A road sign recognition device according to claim 13, wherein:

the color components of each pixel data of the color image consist of R, G, and B values; and

the white balance processing means performs processing of multiplying the R value by a first correction factor calculated based on a ratio between a general level of the R value in the road range of the extracted area and a general level of a luminance value calculated from the color components in the road range of the extracted area, multiplying the G value by a second correction factor calculated based on a ratio between a general level of the G value in the road range of the extracted area and a general level of the luminance value calculated from the color components in the road range of the extracted area, and multiplying the B value by a third correction factor calculated based on a ratio between a general level of the B value in the road range of the extracted area and a general level of the luminance value calculated from the color components in the road range of the extracted area for each pixel data in the extracted area in the white balance process.

15. A road sign recognition device according to claim 14, wherein:

the white balance processing means uses a sum value of R values of all pixels in the road range of the extracted area as the general level of the R value in the road range of the extracted area and uses a sum value of luminance values of all pixels in the road range of the extracted area as the general level of the luminance value calculated from the color components in the road range of the extracted area in the case where calculating the first correction factor;

the white balance processing means uses a sum value of G values of all pixels in the road range of the extracted area as the general level of the G value in the road range of the extracted area and uses a sum value of luminance values of all pixels in the road range of the extracted area as the general level of the luminance value calculated from the color components in the road range of the extracted area in the case where calculating the second correction factor; and

the white balance processing means uses a sum value of B values of all pixels in the road range of the extracted area as the general level of the B value in the road range of the extracted area and uses a sum value of luminance values of all pixels in the road range of the extracted area as the general level of the luminance value calculated from the color components in the road range of the extracted area in the case where calculating the third correction factor.

16. A road sign recognition device according to claim 14, wherein:

the white balance processing means uses a mean value of R values of all pixels in the road range of the extracted area as the general level of the R value in the road range of the extracted area and uses a mean value of luminance values of all pixels in the road range of the extracted area as the general level of the luminance value calculated from the color components in the road range of the extracted area in the case where calculating the first correction factor;

the white balance processing means uses a mean value of G values of all pixels in the road range of the extracted area as the general level of the G value in the road range of the extracted area and uses a mean value of luminance values of all pixels in the road range of the extracted area as the general level of the luminance value calculated from the color components in the road range of the extracted area in the case where calculating the second correction factor; and

the white balance processing means uses a mean value of B values of all pixels in the road range of the extracted area as the general level of the B value in the road range of the extracted area and uses a mean value of luminance values of all pixels in the road range of the extracted area as the general level of the luminance value calculated from the color components in the road range of the extracted area in the case where calculating the third correction factor.