Systems and methods of vehicle image processing

• US 8,170,285 B2
• Filed: 07/06/2006
• Issued: 05/01/2012
• Est. Priority Date: 07/15/2005
• Status: Expired due to Fees

• Current Assignee: Honda Motor Co., Ltd. (Honda Motor Company)
• Original Assignee: Honda Motor Co., Ltd. (Honda Motor Company)
• Inventors: Nakamori, Takuma, Kobayashi, Sachio, Aoki, Tomoyoshi, Mori, Naoki
• Primary Examiner(s): Perungavoor, Sath V
• Assistant Examiner(s): Heidemann, Jason
• Application Number:
  US11/995,798
• Publication Number:
  US 20090284597A1
• Time in Patent Office: 2,126 Days
• Field of Search: None
• US Class Current: 382/104
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30256 Lane; Road marking
• CPC Class Codes: G06T 7/12 Edge-based segmentation
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road

Abstract

An image processing system and the like capable of recognizing a lane mark in a road image with high accuracy are provided even if a light illumination state on a road surface is partially different. According to the image processing system (100) mounted on a vehicle (10), a color component (Rij, Gij, Bij) of the first pixel (Pij) included in an area (Aij) set in the road image is corrected with reference to a color component (Rik, Gik, Bik) of a second pixel (Pik) in view of a fact that it is highly probable that the color component of the second pixel included in the area along with the first pixel is affected by a shadow or light on the road surface. This reduces the effect of the shadow or light on the road surface and the actual color of a road surface portion corresponding to the first pixel can be sufficiently reflected in the color components (Rij, Gij, Bij) of the first pixel (Pij) and consequently in a feature value (Qij). Therefore, a lane mark (M) and its edge (E) are recognized in the road image on the basis of the feature value (Qij) of each pixel (Pij) in the road image.

Claims

1. An image processing system comprising a processor, comprising:

a first processing section which sets an area including a plurality of pixels while gradually moving the area in a road image captured by an imaging means mounted on a vehicle;

a second processing section which corrects a color component of a first pixel included in the front end of the area in a moving direction among the plurality of pixels included in the area by the first processing section with reference to a corresponding color component of a second pixel other than the first pixel among the plurality of pixels;

a third processing section, executed via the processor, which calculates a feature value of the first pixel that depends on a noticeable color on the basis of the color component of the first pixel corrected by the second processing section; and

a fourth processing section which recognizes a lane mark having the noticeable color in the road image on the basis of feature values of a plurality of pixels obtained by sequentially repeating the correction of a color signal of the first pixel by the second processing section and the calculation of the feature value of the first pixel by the third processing section whenever the area is set by the first processing section;

wherein the area is adjusted based upon the result of the correction of the color component of the first pixel.

2. The image processing system according to claim 1, wherein the first processing section sets a one-line continuous area arranged in the moving direction in the road image or an area having a one-line continuous part arranged more forward in the moving direction than the remainder of the area.

3. The image processing system according to claim 1 wherein the first processing section can set the area while adjusting the size, shape, or size and shape of the area according to the position of the road image.

4. The image processing system according to claim 1 wherein the first processing section can set the area while adjusting the size, shape, or size and shape of the area on the basis of a result of the correction of the color component of the first pixel by the second processing section, a result of the calculation of the feature value of the first pixel by the third processing section, a result of the recognition of the lane mark by the fourth processing section, and a part or all of the traveling condition of the vehicle.

5. The image processing system according to claim 1 wherein:

the second processing section corrects the color component of the first pixel with reference to the color component of the second pixel based on a luminance value of the second pixel; and

the third processing section calculates the feature value of the first pixel that depends on the noticeable color on the basis of the corrected color component of the first pixel based on a luminance value of the first pixel.

6. An image processing method comprising:

a first processing step of setting an area including a plurality of pixels while gradually moving the area in a road image captured by an imaging means mounted on a vehicle;

a second processing step of correcting a color component of a first pixel included in the front end of the area in a moving direction among the plurality of pixels included in the area in the first processing step with reference to a corresponding color component of a second pixel other than the first pixel among the plurality of pixels;

a third processing step of calculating a feature value of the first pixel that depends on a noticeable color on the basis of the color component of the first pixel corrected in the second processing step; and

a fourth processing step of recognizing a lane mark having the noticeable color in the road image on the basis of feature values of a plurality of pixels obtained by sequentially repeating the correction of a color signal of the first pixel in the second processing step and the calculation of the feature value of the first pixel in the third processing step whenever the area is set in the first processing step;

wherein the area is adjusted based upon the result of the correction of the color component of the first pixel.

7. An image processing program comprising a non-transitory computer usable medium for causing a computer to attain:

a first processing function of setting an area including a plurality of pixels while gradually moving the area in a road image captured by an imaging means mounted on a vehicle;

a second processing function of correcting a color component of a first pixel included in the front end of the area in a moving direction among the plurality of pixels included in the area by the first processing function with reference to a corresponding color component of a second pixel other than the first pixel among the plurality of pixels;

a third processing function of calculating a feature value of the first pixel that depends on a noticeable color on the basis of the color component of the first pixel corrected by the second processing function; and

a fourth processing function of recognizing a lane mark having the noticeable color in the road image on the basis of feature values of a plurality of pixels obtained by sequentially repeating the correction of a color signal of the first pixel by the second processing function and the calculation of the feature value of the first pixel by the third processing function whenever the area is set by the first processing function;

wherein the area is adjusted based upon the result of the correction of the color component of the first pixel.

8. The image processing program according to claim 7, wherein the image processing program is distributed or broadcast to a computer mounted on the vehicle.

9. A vehicle having:

an imaging means;

an image processing system which performs image processing on the basis of a road image captured by the imaging means; and

a vehicle traveling control system which controls the travel of the vehicle on the basis of a result of the image processing performed by the image processing system,

wherein the image processing system includes;

a first processing section which sets an area including a plurality of pixels while gradually moving the area in the road image;

a second processing section which corrects a color component of a first pixel included in the front end of the area in a moving direction among the plurality of pixels included in the area by the first processing section with reference to a corresponding color component of a second pixel other than the first pixel among the plurality of pixels;

a third processing section which calculates a feature value of the first pixel that depends on a noticeable color on the basis of the color component of the first pixel corrected by the second processing section; and

a fourth processing section which recognizes a lane mark having the noticeable color in the road image on the basis of feature values of a plurality of pixels obtained by sequentially repeating the correction of the color component of the first pixel by the second processing section and the calculation of the feature value of the first pixel by the third processing section whenever the area is set by the first processing section;

wherein the area is adjusted based upon the result of the correction of the color component of the first pixel.

10. The vehicle according to claim 9, wherein the first processing section sets a one-line continuous area arranged in the moving direction in the road image or an area having a one-line continuous part arranged more forward in the moving direction than the remainder of the area.

11. The vehicle according to claim 9 wherein the first processing section can set the area while adjusting the size, shape, or size and shape of the area according to the position of the road image.

12. The vehicle according to claim 9 wherein the first processing section can set the area while adjusting the size, shape, or size and shape of the area on the basis of a result of the correction of the color component of the first pixel by the second processing section, a result of the calculation of the feature value of the first pixel by the third processing section, a result of the recognition of the lane mark by the fourth processing section, and a part or all of the traveling condition of the vehicle.

13. The vehicle according to claim 9 wherein:

the second processing section corrects the color component of the first pixel with reference to the color component of the second pixel based on a luminance value of the second pixel; and

the third processing section calculates the feature value of the first pixel that depends on the noticeable color on the basis of the corrected color component of the first pixel based on a luminance value of the first pixel.