Image processing apparatus for vehicles

• US 5,987,174 A
• Filed: 12/24/1996
• Issued: 11/16/1999
• Est. Priority Date: 04/26/1995
• Status: Expired due to Fees

• Current Assignee: Hitachi, Ltd.
• Original Assignee: Hitachi, Ltd.
• Inventors: Ondo, Eiryo, Monji, Tatsuhiko, Tanaka, Yasunari, Nakamura, Kazuto, Takano, Kazuaki, Hanawa, Kazuhiko
• Primary Examiner(s): Mancuso, Joseph
• Assistant Examiner(s): Bali, Vikkram
• Application Number:
  US08/750,988
• Time in Patent Office: 1,057 Days
• Field of Search: 382/104, 382/153, 382/199, 382/103, 382/203, 382/298, 382/266, 382/162
• US Class Current: 382/199
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30256 Lane; Road marking
• CPC Class Codes: G06T 7/12 Edge-based segmentation
• CPC Class Codes: G06V 10/44 Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
• CPC Class Codes: G06V 10/56 relating to colour
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road

Abstract

An image pickup unit (2) mounted on a vehicle shoots an image in front thereof to attain image information. An edge pixel extraction section (40202 to 40206) scans the obtained image information for each line and then checks boundaries between a particular color and the other colors to extract pixels having the particular color as edge pixels. Moreover, a candidate line extraction section (40208 to 40212) references position coordinates of the extracted edge pixels such that assuming the edge pixels to be classified to configure straight lines, the edge pixels are coupled with each other to configure straight lines according to the classification so as to obtain the straight lines as candidate lines.

Thereafter, a line selection section (40212) selects a left line and a right line in front of the vehicle from the plural candidate lines, thereby attaining the left and right lines.

Claims

1. An image processing apparatus, comprising:

image pickup means mounted on a vehicle for shooting an image in front of the vehicle to attain image information;

edge pixel extraction means for scanning for each line the image information of an image in front of the vehicle obtained through the shooting operation by the image pickup means, checking boundaries between a particular color and other colors, and extracting pixels having the particular color as edge pixels;

candidate line extraction means for referencing position coordinates of the extracted edge pixels, assuming the edge pixels to be classified into straight lines, obtaining straight lines by coupling the edge pixels corresponding to respective results of the classification, and setting the obtained straight lines as candidate lines wherein multiple candidate lines are obtained for at least one of a left and a right line;

line selection means for selecting the left line and the right line in front of the vehicle from the candidate lines; and

wherein the line selection means includes means for selecting two of the candidate lines which have the largest numbers of edge pixels on the left and right sides of the image information of the vehicle as the left and right lines of the vehicle, respectively.

2. An image processing apparatus according to claim 1, wherein the line selection means includes means operative in an operation to select the left and right lines for selecting as the left and right lines two candidate lines satisfying a particular positional relationship.

3. An image processing apparatus according to claim 2, wherein the line selection means includes means operative in an operation to select the left and right lines by assuming a particular horizontal line which is parallel to a horizontal direction of the front image information and which has a particular length for selecting as the left and right lines two candidate lines arranged to be brought into contact with both ends of the horizontal line.

4. An image processing apparatus, comprising:

image pickup means mounted on a vehicle for shooting an image in front of the vehicle to attain image information;

edge pixel extraction means for scanning for each line the image information of an image in front of the vehicle obtained through the shooting operation by the image pickup means, checking boundaries between a particular color and other colors, and extracting pixels having the particular color as edge pixels;

candidate line extraction means for referencing position coordinates of the extracted edge pixels, assuming the edge pixels to be classified into straight lines, obtaining straight lines by coupling the edge pixels corresponding to respective results of the classification, and setting the obtained straight lines as candidate lines wherein multiple candidate lines are obtained for at least one of a left and a right line;

line selection means for selecting the left line and the right line in front of the vehicle from the candidate lines; and

wherein the line selection means includes means for selecting as one of the lines of the vehicle, namely, a selection line, one of the candidate lines having the largest number of edge pixels on the left or right side of the image information and selecting, in an operation to select the other line by assuming a particular horizontal line which is parallel to a horizontal direction of the front image information and which has a particular length, as the other line a candidate line arranged to be brought into contact with an end of the horizontal line when the selection line is most close to the other end thereof.

5. An image processing apparatus for identifying a line of vehicle disposed on a running road surface according to color image data of the road surface, comprising:

color image pickup means for shooting an image in front of a vehicle along a running direction thereof;

edge pixel extraction means for scanning for each line the image data of an image in front of the vehicle obtained through the shooting operation by the color image pickup means, checking boundaries between a particular color and other colors, obtaining pixels having the particular color as edge pixels, and extracting pixels satisfying a particular extraction color condition from the obtained edge pixels;

color sampling means for sampling color data of the decided pixels from a single screen of the image data;

extraction condition decision means for determining the extraction color condition according to the sampled color data; and

line image extraction means for attaining as a line of the vehicle a line image configured by coupling the extracted edge pixels into a straight line.

6. An image processing apparatus according to claim 5, wherein the extraction color condition means includes means for deciding the sample pixel positions according to road surface data representing the recognized line.

7. An image processing apparatus according to claim 5, further including:

means for deciding an illumination environment of the running road surface according to the sampled color data; and

means responsive to a result of the decision for controlling a state of an iris of an optical system disposed in the color image pickup means and a color conversion gain of an image data processing system thereof.

8. An image processing apparatus according to claim 5, further including means for updating the extraction color condition according to a state of the sampled color data and a number of pixels satisfying the extraction color condition.

9. An image processing apparatus according to claim 5, wherein the line selection means includes means for selecting, in an operation to select either one of the left and right lines by assuming a central perpendicular line of front image information, as a line of the vehicle, one of the candidate lines which has the largest number of edge pixels on the left or right side of the central perpendicular line.

10. An image processing apparatus according to claim 9, wherein the line selection means includes means operative in an operation to select the left and right lines for selecting as the left and right lines two candidate lines satisfying a particular positional relationship.

11. An image processing apparatus according to claim 10, wherein the line selection means includes means operative in an operation to select the left and right lines by assuming a particular horizontal line which is parallel to a horizontal direction of the front image information and which has a particular length for selecting as the left and right lines two candidate lines arranged to be brought into contact with both ends of the horizontal line.

12. An image processing apparatus according to claim 9, wherein the line selection means includes means for selecting as one of the lines of the vehicle, namely, a selection line by assuming a central perpendicular line of front image information, one of the candidate lines having the largest number of edge pixels on the left or right side of the central perpendicular line and selecting, in an operation to select the other line by assuming a particular horizontal line which is parallel to a horizontal direction of the front image information and which has a particular length, as the other line a candidate line arranged to be brought into contact with an end of the horizontal line when the selection line is brought into contact with the other end thereof.

13. An image processing apparatus, comprising:

image pickup means for shooting an image to attain image information;

image magnification process means for magnifying a size of the obtained image information;

edge pixel coordinate sense means for scanning for each line the magnified image information, checking boundaries between a particular color and other colors, and extracting pixels on the boundaries, and obtaining position coordinates of the extracted edge pixels;

edge pixel coordinate conversion means for converting the position coordinates of the extracted edge pixels into coordinates before the magnification process;

line image extraction means for referencing the converted coordinates of the edge pixels and coupling the edge pixels into a straight line, thereby attaining a line image; and

magnifying condition storage means for storing therein a zoom rate as a magnitude of magnification of the image information and a magnification center coordinate indicating a reference position for the execution of the magnification process of the image information, the magnification factor and the magnification center coordinate being defined for each horizontal line, wherein

the image magnification process means includes means for referencing storage contents in the magnifying condition storage means and magnifying for each horizontal line the size of the image information with the magnification center coordinate set to a center of magnification according to a magnification factor indicated by the zoom rate.

14. An image processing apparatus according to claim 13, wherein the image magnification process means includes means for magnifying the obtained image information only in the horizontal direction.

15. An image processing apparatus according to claim 13, wherein the magnifying condition storage means stores therein values of the zoom rate, the values continuously varying for each horizontal line.

16. An image processing apparatus according to claim 13, wherein the line extraction means includes means for obtaining, as the magnification center coordinate of each horizontal line, a center position of a vehicle running lane enclosed with the left and right lines in each horizontal line and storing the coordinate in the magnifying condition storage means.

17. An image processing apparatus according to claim 13, wherein the magnifying condition storage means beforehand stores therein values of the zoom rate, the value for horizontal lines existing above a particular horizontal line in the perpendicular direction being different from that for horizontal lines including the particular horizontal line and existing below the particular horizontal line in the perpendicular direction.

18. An image processing apparatus according to claim 14, wherein the magnifying condition storage means stores therein values of the zoom rate, the values continuously varying for each horizontal line.

19. An image processing apparatus according to claim 14, wherein the line extraction means includes means responsive to the coordinate data of extracted lines obtained by the process in a previous line extraction cycle for attaining a zoom rate and a magnification center coordinate for each horizontal line and storing the zoom rate and the magnification center coordinate in the magnifying condition storage means.

20. An image processing apparatus according to claim 14, wherein the line extraction means includes means for obtaining, according to a value obtained by dividing a predetermined screen width by a width of a vehicle running lane enclosed with the left and right lines in each horizontal line, the zoom rate for each horizontal line and storing the zoom rate in the magnifying condition storage means.

21. An image processing apparatus according to claim 15, wherein the line extraction means includes means for obtaining, as the magnification center coordinate of each horizontal line, a center position of a vehicle running lane enclosed with the left and right lines in each horizontal line and storing the coordinate in the magnifying condition storage means.

22. An image processing apparatus according to claim 14, wherein the magnifying condition storage means beforehand stores therein values of the zoom rate, the value for horizontal lines existing above a particular horizontal line in the perpendicular direction being different from that for horizontal lines including the particular horizontal line and existing below the particular horizontal line in the perpendicular direction.

23. An image processing apparatus, comprising:

image pickup means mounted on a vehicle for shooting an image in front thereof to attain image information;

image magnification process means for magnifying a size of the obtained image information;

edge pixel coordinate sense means for scanning for each line the magnified image information, checking boundaries between a particular color and other colors, and extracting pixels on the boundaries, and obtaining position coordinates of the extracted edge pixels;

edge coordinate conversion means for converting the position coordinates of the extracted edge pixels into coordinates before the magnification process; and

line extraction means for referencing the converted coordinates of the edge pixels and coupling the edge pixels into a straight line, thereby attaining the left and right lines and magnifying condition storage means for storing therein a zoom rate as a magnitude of magnification of the image information and a magnification center coordinate indicating a reference position for the execution of the magnification process of the image information, the zoom rate and the magnification center coordinate being defined for each horizontal line, wherein

the image magnification process means includes means for referencing storage contents in the magnifying condition storage means and magnifying for each horizontal line the size of the image information with the magnification center coordinate set to a center of magnification according to a magnification factor indicated by the zoom rate.

24. An image processing apparatus according to claim 23, wherein the line extraction means includes means responsive to the coordinate data of extracted lines obtained by the process in a previous line extraction cycle for attaining a zoom rate and a magnification center coordinate for each horizontal line and storing the zoom rate and the magnification center coordinate in the magnifying condition storage means.

25. An image processing apparatus according to claim 23, wherein the line extraction means includes means for obtaining, according to a value obtained by dividing a predetermined screen width by a width of a vehicle running lane enclosed with the left and right lines in each horizontal line, the zoom rate for each horizontal line and storing the zoom rate in the magnifying condition storage means.

26. An image processing apparatus according to claim 23, wherein the image magnification process means includes means for magnifying the obtained image information only in the horizontal direction.

27. An image processing apparatus according to claim 23, wherein the magnifying condition storage means stores therein values of the zoom rate, the values continuously varying for each horizontal line.

28. An image processing apparatus according to claim 23, wherein the line extraction means includes means for obtaining, as the magnification center coordinate of each horizontal line, a center position of a vehicle running lane enclosed with the left and right lines in each horizontal line and storing the coordinate in the magnifying condition storage means.

29. An image processing apparatus according to claim 23, wherein the magnifying condition storage means beforehand stores therein values of the zoom rate, the value for horizontal lines existing above a particular horizontal line in the perpendicular direction being different from that for horizontal lines including the particular horizontal line and existing below the particular horizontal line in the perpendicular direction.