VEHICLE PERIPHERY MONITORING DEVICE, VEHICLE PERIPHERY MONITORING PROGRAM AND VEHICLE PERIPHERY MONITORING METHOD

• US 20100103262A1
• Filed: 04/27/2007
• Published: 04/29/2010
• Est. Priority Date: 04/27/2007
• Status: Active Grant

• Current Assignee: Honda Motor Co., Ltd. (Honda Motor Company)
• Original Assignee: Honda Motor Co., Ltd. (Honda Motor Company)
• Inventors: Fardi, Basel
• Granted Patent: US 8,411,145 B2
• Time in Patent Office: Days
• US Class Current: 348/148
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: B60R 1/24 in front of the vehicle
• CPC Class Codes: B60R 1/30 providing vision in the non-visible spectrum, e.g. night or infrared vision
• CPC Class Codes: B60R 11/00 Arrangements for holding or mounting articles, not otherwise provided for
• CPC Class Codes: B60R 11/04 Mounting of cameras operative during drive; Arrangement of controls thereof relative to the vehicle
• CPC Class Codes: B60R 2300/103 using camera systems provided with artificial illumination device, e.g. IR light source
• CPC Class Codes: B60R 2300/205 using a head-up display
• CPC Class Codes: B60R 2300/301 combining image information with other obstacle sensor information, e.g. using RADAR/LIDAR/SONAR sensors for estimating risk of collision
• CPC Class Codes: B60R 2300/8093 for obstacle warning
• CPC Class Codes: G06F 18/256 of results relating to different input data, e.g. multimodal recognition
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10048 Infrared image
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/00 Image analysis
• CPC Class Codes: G06T 7/12 Edge-based segmentation
• CPC Class Codes: G06V 10/755 Deformable models or variational models, e.g. snakes or active contours
• CPC Class Codes: G06V 10/811 the classifiers operating on different input data, e.g. multi-modal recognition
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads

Abstract

A vehicle periphery monitoring device includes an image processing target area setting portion (21) configured to set an image processing target area in the captured image based on a distance detected by a radar between object and a vehicle; a directivity extracting portion (22) configured to perform a a differential filtering on each of pixels in the image processing target area in a predefined direction; an image processing target area correcting portion (23) configured to extract a feature region having a feature amount of a pedestrian'"'"'s head and to correct a range of the image processing target area based on a position of the feature region; a contour extracting portion (24) configured to extract a contour of the image portion included in the corrected image processing target area; and an object type discriminating portion (25) configured to discriminate a type of the monitoring object based on the extracted contour.

Claims

1. A vehicle periphery monitoring device which is provided with an imaging unit mounted in a vehicle to image a periphery of the vehicle and is configured to monitor the periphery of the vehicle on the basis of a captured image by the imaging unit, comprising:

an object detecting portion configured to detect an object located in an imaging area imaged by the imaging unit;

an image processing target area setting portion configured to set an image processing target area which may include an image portion of a monitoring object in the captured image on the basis of a detection result from the object detecting portion;

a contour extracting portion configured to extract a contour of the image portion included in the image processing target area; and

an object type discriminating portion configured to discriminate a type of the monitoring object in a real space corresponding to the image portion from which the contour has been extracted by the contour extracting portion on the basis of the extracted contour.

2. The vehicle periphery monitoring device according to claim 1, wherein

the object type discriminating portion discriminates whether or not the type of the monitoring object in the real space corresponding to the image portion from which the contour has been extracted is a predefined type by comparing a frequency component of the contour extracted by the contour extracting portion with a comparison pattern of frequency components set according to the monitoring object of the predefined type.

3. The vehicle periphery monitoring device according to claim 1, further includes a directivity extracting portion configured to perform a directivity extraction processing which performs a differential filtering on each of pixels in the image processing target area in a predefined direction, allocates a predefined first luminance to a pixel whose directivity has been recognized by the differential filtering, and allocates a second luminance having a difference of not smaller than a predefined level compared with the first luminance to a pixel whose directivity is not recognized,

wherein the contour extracting portion extracts the contour of the image portion in the image processing target area where the directivity extraction processing has been performed.

4. The vehicle periphery monitoring device according to claim 3, wherein the contour extracting portion extracts the contour of the image portion by setting an initial eclipse circling the image portion in the image processing target area where the directivity extraction processing has been performed and scaling down the circumference of the initial ellipse inward to a position of a pixel which has a luminance equal to a threshold set between the first luminance and the second luminance.

5. The vehicle periphery monitoring device according to claim 1, wherein

the contour extracting portion extracts the contour of the image portion by setting an initial eclipse circling the image portion in the image processing target area and performing a contour convergence processing on the initial ellipse according to Snakes algorithm,

the contour convergence processing is performed according to the Snakes algorithm by using a computation expression in which a multiplication term of an internal energy of Snakes energy where the Snakes energy and a contour positional variation and an external energy variation and an internal energy variation are presumed to be of zero at the time when a contour of a Snakes model of a discrete system is converged is multiplied to a multiplication term of the external energy.

6. The vehicle periphery monitoring device according to claim 3, wherein

the contour extracting portion sets an initial eclipse circling the image portion in the image processing target area where the directivity extraction processing has been performed, and extracts a contour of the image portion by scaling down the circumference of the initial ellipse inward to a position of a pixel which has a luminance equal to a threshold set between the first luminance and the second luminance;

the extracted contour is further converged by a contour convergence processing performed thereon according to Snakes algorithm; and

the contour convergence processing is performed according to the Snakes algorithm by using a computation expression in which a multiplication term of an internal energy of Snakes energy where the Snakes energy and a contour positional variation and an external energy variation and an internal energy variation are presumed to be of zero at the time when a contour of a Snakes model of a discrete system is converged is multiplied to a multiplication term of the external energy.

7. The vehicle periphery monitoring device according to claim 5, wherein

the contour extracting portion uses the following mathematical expressions (1) and (2) as the computation expression:

[Mathematical expression 1]

x^t=(I−

γ

A) {x^t-1−

γ

f_x(x^t-1, y^t-1)} 

 

(1)

[Mathematical expression 2]

y^t=(I−

γ

A) {y^t-1−

γ

f_y(x^t-1, y^t-1)} 

 

(2)

Herein,

(x^t, y^t);

coordinate of a contour dot at timing t;

(x^t-1, y^t-1);

a model expression of the contour at timing t−

1;

I;

a unit matrix;

A;

a transform matrix of internal energy;

γ

;

an iteration step (moving amount of the contour per step when converged inward);

f_x;

a computation expression of x components of the external energy; and

f_y;

a computation expression of y components of the external energy.

8. The vehicle periphery monitoring device according to claim 7, wherein

the iteration step γ

in the mathematical expressions (1) and (2) is configured to satisfy the following conditions defined by the mathematical expression (5) for the following mathematical expression (4) which is a Snakes expression where the discrete model of the contour is defined according to the following mathematical expression (3);

[Mathematical expression 3]

K_i=(x_i, y_i)^T=(x(ih), (ih))^T, i ∈

{0, 1, . . . , n−

1} 

 

(3)

Herein, Ki;

discrete model of the contour

9. The vehicle periphery monitoring device according to claim 1, further includes a radar configured to detect a distance between the vehicle and the object located in the imaging area imaged by the imaging unit,

wherein the object detecting portion detects the object located in the imaging area according to the distance detected by the radar.

10. A vehicle periphery monitoring program which causes a computer provided with an accessing portion configured to access a captured image by an imaging unit which is mounted in a vehicle to image a periphery of the vehicle to monitor the periphery of the vehicle on the basis of the captured image by the imaging unit, the program functioning the computer as:

an object detecting portion configured to detect an object located in an imaging area imaged by the imaging unit;

an image processing target area setting portion configured to set an image processing target area which may include an image portion of a monitoring object in the captured image on the basis of a detection result from the object detecting portion;

a contour extracting portion configured to extract a contour of the image portion included in the image processing target area; and

an object type discriminating portion configured to discriminate a type of the monitoring object in a real space corresponding to the image portion from which the contour has been extracted by the contour extracting portion on the basis of the extracted contour.

11. A vehicle periphery monitoring method which monitors a periphery of a vehicle according to a computer provided with an accessing portion configured to access a captured image by an imaging unit which is mounted in the vehicle to image the periphery of the vehicle to monitor the periphery of the vehicle on the basis of the captured image by the imaging unit, comprising steps of:

an object detecting step in which the computer detects an object located in an imaging area imaged by the imaging unit;

an image processing target area setting step in which the computer sets an image processing target area which may include an image portion of a monitoring object in the captured image on the basis of a detection result from the object detecting step;

a contour extracting step in which the computer extracts a contour of the image portion included in the image processing target area; and

an object type discriminating step in which the computer discriminates a type of the monitoring object in a real space corresponding to the image portion from which the contour has been extracted by the contour extracting step on the basis of the extracted contour.