Image recognition apparatus and image processing apparatus

• US 20020159616A1
• Filed: 03/09/2001
• Published: 10/31/2002
• Est. Priority Date: 09/29/1999
• Status: Active Grant

• Current Assignee: Fujitsu Ten Limited (DENSO Corporation)
• Original Assignee: Fujitsu Ten Limited (DENSO Corporation)
• Inventors: Ohta, Akihiro
• Granted Patent: US 6,763,125 B2
• Time in Patent Office: Days
• US Class Current: 382/104
• CPC Class Codes: G06T 7/97 Determining parameters from multiple pictures depth or shape recovery from multiple images G06T7/55; stereo camera calibration G06T7/85
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road

Abstract

In an image recognition apparatus, a left camera and a right camera are provided so as to produce a parallax. A road surface indication recognizing section recognizes road surface indications contained in a left input image that is supplied from the left camera. An edge detecting section detects remaining edges other than the edges of the recognized road surface indications among all the edges in the left input image. A matching section extracts a reference pattern in the left input image that includes a detected edge, and conducts matching between the reference pattern and a right input image that is supplied from the right camera using a pattern matching method. Based on each matching result with the reference pattern, an object recognizing section recognizes an object in the visual fields and determines a distance from the recognized object to the image recognition apparatus.

Claims

1. An image recognition apparatus comprising:

first and second cameras provided so as to produce a parallax, for shooting in respective visual fields;

edge detecting means for detecting edges in a first input image that is supplied from the first camera;

matching means for extracting, as a reference pattern, a portion in the first input image that includes at least part of a detected edge and for conducting matching between the reference pattern and a second input image that is supplied from the second camera using a pattern matching method; and

object recognizing means for recognizing an object in the visual fields of the first and second cameras, based on a matching result.

2. The image recognition apparatus of claim 1, further comprising:

road surface indication recognizing means f or recognizing road surface indications contained in at least one input image of the first input image and the second input image,

wherein the edge detecting means detects remaining edges other than edges of the road surface indications among all the edges in the first input image, from the first input image, based on a road surface indication recognition result.

3. The image recognition apparatus of claim 2, wherein the road surface indication recognizing means recognizes only the road surface indications in the first input image.

4. The image recognition apparatus of claim 2, wherein:

the first camera and the second camera are disposed on a mounting axial line that is parallel with a road surface in the visual fields so as to be separated from each other by a predetermined distance,

the road surface indication recognizing means;

(1) divides the at least one input image into a plurality of band-like regions extending in a reference direction corresponding to a direction parallel with the mounting axial line in the at least one input image,

(2) on the band-like region basis, carries out a search of density variation pattern in the reference direction to determine as a candidate portion for a road surface indication, a portion having one end whose density increases according to a predetermined reference increase pattern and another end whose density decreases according to a predetermined reference decrease pattern, and

(3) compares a width of the candidate portion for a road surface indication with a predetermined reference width, and when the width of the candidate portion for a road surface indication is smaller than or equal to the reference width, recognizes that the candidate portion in the at least one input image is part of a road surface indication, and

the reference width is predetermined based on the width of a road surface indication that is closest to the first camera and the second camera in a displayed image of a road surface.

5. The image recognition apparatus of claim 4, wherein the reference increase pattern is a density variation pattern of from a density corresponding to black to a density corresponding to white, and

the reference decrease pattern is a density variation pattern of from the density corresponding to white to the density corresponding to black.

6. The image recognition apparatus of claim 4, wherein the reference increase pattern is a density variation pattern of from a density corresponding to black to a density corresponding to white via a density corresponding to gray, and

the reference decrease pattern is a density variation pattern of from the density corresponding to white to the density corresponding to black via the density corresponding to gray.

7. The image recognition apparatus of claim 4, wherein the reference increase pattern is a density variation pattern of from a density corresponding to black via a density corresponding to gray to a density corresponding to white, and

the reference decrease pattern is a density variation pattern of from a density corresponding to white to a density corresponding to black.

8. The image recognition apparatus of claim 4, wherein the reference increase pattern is a density variation pattern of from a density corresponding to black to a density corresponding to white, and

the reference decrease pattern is a density variation pattern of from the density corresponding to white to the density corresponding to black via the density corresponding to gray.

9. The image recognition apparatus of claim 4, wherein the road surface indication recognizing means:

further determines a number of candidate portions that have a width smaller than or equal to the reference width and that are arranged consecutively in a direction approximately perpendicular to the reference direction, and

when the number of the candidate portions is greater than or equal to a predetermined reference number, recognizes that the candidate portions include parts of an image of a road surface indication in the at least one input image.

10. The image recognition apparatus of claim 9, wherein the road surface indication recognizing means:

judges whether an approximately band-like region that consists of the candidate portions that have a width smaller than or equal to the reference width and that are arranged consecutively in the direction approximately perpendicular to the reference direction in the predetermined number or more has an inclination that is within an allowable range that is predetermined based on an inclination of road surface indications in a displayed image of a road surface, and

only when the inclination of the approximately band-like region is within the allowable range, recognizes that the candidate portions constituting the approximately band-like region is a road surface indication.

11. The image recognition apparatus of claim 2, wherein the road surface indication recognizing means searches an entire area of the at least one input image for candidate portions for a road surface indication.

12. The image recognition apparatus of claim 2, wherein the road surface indication recognizing means searches, for candidate portions for a road surface indication, a prediction region of the at least one input image where a road surface indication is expected to exist.

13. The image recognition apparatus of claim 2, wherein the edge detecting means:

(1) judges, on a pixel basis, whether pixels in the first input image are located outside road surface indication portions therein, based on a road surface indication recognition result; and

(2) performs, only to each of pixels in the first input image that are located outside the road surface indication portions therein, a process for judging whether the pixel constitutes an edge in the first input image.

14. The image recognition apparatus of claim 1, wherein the first input image and the second input image each consist of a plurality of pixels and the matching means conducts matching between the reference pattern and the second input image, using color densities of the respective pixels as indices.

15. The image recognition apparatus of claim 14, wherein the matching means corrects, prior to the pattern matching, the densities of the respective pixels of at least one input image of the first input image and the second input image using an offset value commensurate with an individual, color-related difference between the first camera and the second camera.

16. The image recognition apparatus of claim 15, wherein the matching means learns the individual, color-related difference between the first camera and the second camera, based on the first input image and the second input image.

17. The image recognition apparatus of claim 1, wherein the first input image and the second input image each consist of a plurality of pixels and the matching means conducts matching between the reference pattern and the second input image, using color density variation amounts of the pixels as indices.

18. The image recognition apparatus of claim 14, wherein the matching means calculates a density difference between maximum and minimum densities of the pixels in the reference pattern and only when the calculated density difference is greater than or equal to a predetermined reference density difference, uses the reference pattern for the pattern matching,

the predetermined reference density difference being predetermined based on a minimum density difference of a reference pattern that enables effective matching.

19. The image recognition apparatus of claim 1, wherein the first camera and the second camera are disposed on a predetermined mounting axial line so as to be separated from each other by a predetermined distance,

in performing the pattern matching the matching means sets a search region as a subject of a process of the pattern matching in the second input image and compares, with a density distribution of the reference pattern, a density distribution of a portion in the second input image which portion includes at least one pixel of the search region and is congruous with the reference pattern,

a width of the search region in a direction that is perpendicular to a reference direction corresponding to a direction parallel with the mounting axial line in the second input image is greater than or equal to three pixels, and

a coordinate in the perpendicular direction of a center of the search region in the second input image is the same as a coordinate in the perpendicular direction of a center of the reference pattern in the first input image.

20. The image recognition apparatus of claim 1, wherein the first camera and the second camera are disposed on a predetermined mounting axial line so as to be separated from each other by a predetermined distance,

in performing the pattern matching the matching means sets a search region as a subject of a process of the pattern matching in the second input image and compares, with a density distribution of the reference pattern, a density distribution of a portion in the second input image which portion includes at least one pixel of the search region and is congruous with the reference pattern,

a width of the search region in a reference direction corresponding to a direction parallel with the mounting axial line in the second input image is a width commensurate with an upper limit parallax that is determined based on an arrangement of the first camera and the second camera, and

a coordinate in the reference direction of a center of the search region in the second input image is deviated from a coordinate in the reference direction of a center of the reference pattern in the first input image in the same direction as a deviation corresponding to a parallax of the second input image with respect to the first input image.

21. The image recognition apparatus of claim 1, wherein in performing the pattern matching the matching means:

(1) selects, one by one, a plurality of detected edges in the first input image in order from an edge having highest edge intensity;

(2) every time an edge is selected, sets, in the first input image, a reference pattern including at least part of the selected edge;

(3) sets, in the second input image, a search region as a processing subject of the pattern matching, based on the position of the set reference pattern in the first input image and a matching result of a reference pattern that has already been processed and the second input image; and

(4) compares a density distribution of the set reference pattern and a density distribution of a portion in the second input image that includes at least one pixel of the search region and is congruous with the set reference pattern.

22. An image processing apparatus comprising:

road surface image storing means for storing a displayed image of a road surface; and

road surface indication recognizing means for recognizing a displayed portion of a road surface indication in the road surface image,

wherein the road surface indication recognizing means;

(1) divides the road surface image into a plurality of band-like regions extending in a reference direction corresponding to a direction parallel with the road in the image;

(2) on the band-like region basis, carries out a search of density variation pattern in the reference direction to determine as a candidate portion for a road surface indication, a portion having one end whose density increases according to a predetermined reference increase pattern and another end whose density decreases according to a predetermined reference decrease pattern, and

(3) compares a width of the searched candidate portion for a road surface indication with a predetermined reference width, and when the width of the searched candidate portion for a road surface indication is smaller than or equal to the reference width, recognizes that the searched candidate portion is part of a road surface indication, and

the reference width is predetermined based on the width of a road surface indication that is closest to a camera that shot a road surface in a displayed image of the road surface.