VEHICLE PERIPHERY MONITORING DEVICE, VEHICLE PERIPHERY MONITORING PROGRAM, AND VEHICLE PERIPHERY MONITORING METHOD

• US 20100104138A1
• Filed: 04/27/2007
• Published: 04/29/2010
• Est. Priority Date: 04/27/2007
• Status: Active Grant

• Current Assignee: Arriver Software AB (Qualcomm, Inc.)
• Original Assignee: Honda Motor Co., Ltd. (Honda Motor Company)
• Inventors: Fardi, Basel
• Granted Patent: US 8,126,210 B2
• Time in Patent Office: Days
• US Class Current: 382/106
• CPC Class Codes: B60R 11/04 Mounting of cameras operative during drive; Arrangement of controls thereof relative to the vehicle
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/10048 Infrared image
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 2207/30261 Obstacle
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

A vehicle periphery monitoring device is provided with an image processing target area setting portion for setting an image processing target area (61) that may include an image portion of an monitoring object in a captured image (Im2) on the basis of a distance between an object and a vehicle; a feature region extracting portion for extracting a feature region with a feature amount of a head of a pedestrian in a search area (64) based on the image processing target area (61); an image processing target area correcting portion for correcting a range of the image processing target area from (61) to (62) on the basis of the position of a feature region (60a), an object type discriminating portion for discriminating a type of real space monitoring object corresponding to the image portion included in the corrected image processing target area (62).

Claims

1. A vehicle periphery monitoring device which is provided with a radar mounted in a vehicle and configured to detect a distance between the vehicle and an object located in a first monitoring range of a periphery of the vehicle and an imaging unit mounted in the vehicle and configured to photograph a second monitoring range overlapped with the first monitoring range, and is configured to monitor the periphery of the vehicle on the basis of a distance information detected by the radar and a captured image by the imaging unit, comprising:

an image processing target area setting portion configured to set an image processing target area which may include an image portion of a monitoring object in the captured image on the basis of the distance detected by the radar between an object located in the second monitoring range and the vehicle;

a feature region extracting portion configured to extract a feature region having a feature amount defined in accordance with a specified part of the monitoring object of a predefined type in a search area defined on the basis of the image processing target area;

an image processing target area correcting portion configured to correct a range of the image processing target area on the basis of a position of the feature region when the feature region is extracted by the feature region extracting portion; and

an object type discriminating portion configured to discriminate a type of the monitoring object in a real space corresponding to the image portion included in the image processing target area which has been corrected by the image processing target area correcting portion.

2. The vehicle periphery monitoring device according to claim 1, further includes a data storing portion configured to store an object size data representing a size of the monitoring object of the predefined type, wherein

the image processing target area setting portion detects a position corresponding to a contact portion between the object and a road surface in the captured image on the basis of the distance detected by the radar between the vehicle and the object located in the second monitoring range, and sets as the image processing target area a range determined by correcting the object size data according to the distance using the position as a reference.

3. The vehicle periphery monitoring device according to claim 1, wherein

the predefined type is a pedestrian;

the specified part is a head of the pedestrian; and

the object type discriminating portion discriminates whether the monitoring object in the real space corresponding to the image portion included in the image processing target area corrected by the image processing target area correcting portion is the pedestrian or not.

4. A vehicle periphery monitoring device which is provided with a radar mounted in a vehicle and configured to detect a distance between the vehicle and an object located in a first monitoring range of a periphery of the vehicle and an imaging unit mounted in the vehicle and configured to photograph a second monitoring range overlapped with the first monitoring range, and is configured to monitor the periphery of the vehicle on the basis of a distance information detected by the radar and a captured image by the imaging unit, comprising:

an image processing target area setting portion configured to set an image processing target area which may include an image portion of a monitoring object in the captured image on the basis of the distance detected by the radar between an object located in the second monitoring range and the vehicle;

a luminance distribution calculating portion configured to calculate a luminance distribution of a luminance calculation area defined on the basis of the image processing target area;

an image processing target area correcting portion configured to correct a range of the image processing target area on the basis of the luminance distribution calculated by the luminance distribution calculating portion; and

an object type discriminating portion configured to discriminate a type of the monitoring object in a real space corresponding to the image portion included in the image processing target area which has been corrected by the image processing target area correcting portion.

5. The vehicle periphery monitoring device according to claim 4, further includes a data storing portion configured to store an object size data representing a size of the monitoring object of a predefined type, wherein

the image processing target area setting portion detects a position corresponding to a contact portion between the object and a road surface in the captured image on the basis of the distance detected by the radar between the vehicle and the object located in the second monitoring range, and sets as the image processing target area a range determined by correcting the object size data according to the distance using the position as a reference.

6. The vehicle periphery monitoring device according to claim 4, wherein the luminance distribution calculating portion calculates a luminance projection in a horizontal direction and a vertical direction, respectively, in the luminance calculating area as the luminance distribution.

7. The vehicle periphery monitoring device according to claim 4, further includes a feature region extracting portion configured to extract a feature region having a feature amount defined in accordance with a specified part of the monitoring object of a predefined type in a search area defined on the basis of the image processing target area corrected by the image processing target area correcting portion, wherein

the image processing target area correcting portion configured to further correct a range of the image processing target area on the basis of a position of the feature region when the feature region is extracted by the feature region extracting portion.

8. A vehicle periphery monitoring program which causes a computer provided with an input portion configured to input a detection signal from a radar mounted in a vehicle and configured to detect a distance between the vehicle and an object located in a first monitoring range of a periphery of the vehicle, and an accessing portion configured to access a captured image from an imaging unit mounted in the vehicle and configured to photograph a second monitoring range overlapped with the first monitoring range, to monitor the periphery of the vehicle on the basis of a distance information detected by the radar and the captured image by the imaging unit, functioning the computer as:

an image processing target area setting portion configured to set an image processing target area which may include an image portion of a monitoring object in the captured image on the basis of the distance detected by the radar between an object located in the second monitoring range and the vehicle;

a feature region extracting portion configured to extract a feature region having a feature amount defined in accordance with a specified part of the monitoring object of a predefined type in a search area defined on the basis of the image processing target area;

an image processing target area correcting portion configured to correct a range of the image processing target area on the basis of a position of the feature region when the feature region is extracted by the feature region extracting portion; and

an object type discriminating portion configured to discriminate a type of the monitoring object in a real space corresponding to the image portion included in the image processing target area which has been corrected by the image processing target area correcting portion.

9. A vehicle periphery monitoring program which causes a computer provided with an input portion configured to input a detection signal from a radar mounted in a vehicle and configured to detect a distance between the vehicle and an object located in a first monitoring range of a periphery of the vehicle, and an accessing portion configured to access a captured image from an imaging unit mounted in the vehicle and configured to photograph a second monitoring range overlapped with the first monitoring range, to monitor the periphery of the vehicle on the basis of a distance information detected by the radar and the captured image by the imaging unit, functioning the computer as:

an image processing target area setting portion configured to set an image processing target area which may include an image portion of a monitoring object in the captured image on the basis of the distance detected by the radar between an object located in the second monitoring range and the vehicle;

a luminance distribution calculating portion configured to calculate a luminance distribution of a luminance calculation area defined on the basis of the image processing target area;

an image processing target area correcting portion configured to correct a range of the image processing target area on the basis of the luminance distribution calculated by the luminance distribution calculating portion; and

an object type discriminating portion configured to discriminate a type of the monitoring object in a real space corresponding to the image portion included in the image processing target area which has been corrected by the image processing target area correcting portion.

10. A vehicle periphery monitoring method which causes a computer provided with an input portion configured to input a detection signal from a radar mounted in a vehicle and configured to detect a distance between the vehicle and an object located in a first monitoring range of a periphery of the vehicle, and an accessing portion configured to access a captured image from an imaging unit mounted in the vehicle and configured to photograph a second monitoring range overlapped with the first monitoring range, to monitor the periphery of the vehicle on the basis of a distance information detected by the radar and the captured image by the imaging unit, comprising steps of:

an image processing target area setting step in which the computer sets an image processing target area which may include an image portion of a monitoring object in the captured image on the basis of the distance detected by the radar between an object located in the second monitoring range and the vehicle;

a feature region extracting step in which the computer extracts a feature region having a feature amount defined in accordance with a specified part of the monitoring object of a predefined type in a search area defined on the basis of the image processing target area;

an image processing target area correcting step in which the computer corrects a range of the image processing target area on the basis of a position of the feature region when the feature region is extracted in the feature region extracting step; and

an object type discriminating step in which the computer discriminates a type of the monitoring object in a real space corresponding to the image portion included in the image processing target area which has been corrected in the image processing target area correcting step.

11. A vehicle periphery monitoring method which causes a computer provided with an input portion configured to input a detection signal from a radar mounted in a vehicle and configured to detect a distance between the vehicle and an object located in a first monitoring range of a periphery of the vehicle, and an accessing portion configured to access a captured image from an imaging unit mounted in the vehicle and configured to photograph a second monitoring range overlapped with the first monitoring range, to monitor the periphery of the vehicle on the basis of a distance information detected by the radar and the captured image by the imaging unit, comprising steps of:

an image processing target area setting step in which the computer sets an image processing target area which may include an image portion of a monitoring object in the captured image on the basis of the distance detected by the radar between an object located in the second monitoring range and the vehicle;

a luminance distribution calculating step in which the computer calculates a luminance distribution of a luminance calculation area defined on the basis of the image processing target area;

an image processing target area correcting step in which the computer corrects a range of the image processing target area on the basis of the luminance distribution calculated in the luminance distribution calculating step; and

an object type discriminating step in which the computer discriminates a type of the monitoring object in a real space corresponding to the image portion included in the image processing target area which has been corrected in the image processing target area correcting step.