Vehicle surroundings monitoring apparatus, vehicle surroundings monitoring method, and vehicle surroundings monitoring program

• US 7,671,725 B2
• Filed: 03/13/2007
• Issued: 03/02/2010
• Est. Priority Date: 03/24/2006
• Status: Active Grant

• Current Assignee: Arriver Software AB (Qualcomm, Inc.)
• Original Assignee: Honda Motor Co., Ltd. (Honda Motor Company)
• Inventors: Sugimoto, Yoichi, Nishigaki, Morimichi, Tsuji, Takayuki, Nagaoka, Nobuharu
• Primary Examiner(s): Wu; Daniel
• Assistant Examiner(s): Lu; Shirley
• Application Number:
  US11/717,565
• Publication Number:
  US 20070222566A1
• Time in Patent Office: 1,085 Days
• Field of Search: 340/435, 340/944, 340/470
• US Class Current: 340/435
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition

Abstract

A vehicle surroundings monitoring apparatus, a vehicle surroundings monitoring method, and a vehicle surroundings monitoring program which can rapidly determine an object such as a pedestrian to be avoided which must be avoided from coming into contact with the vehicle from an image of the surroundings of the vehicle and can provide information to a driver or control the vehicle behaviors. The vehicle surroundings monitoring apparatus includes an object extraction process unit (11) which extracts objects existing around a vehicle (10) from images taken by infrared cameras (2R, 2L), a pedestrian extraction process unit (12) which extracts a pedestrian from the extracted objects, a posture determination process unit (13) which determines the posture of the extracted pedestrian, an object-to-be-avoided determination process unit (14) which determines whether the extracted object is an object to be avoided which must be avoided from coming into contact with the vehicle (10) by executing a determination algorithm including at least a first determination process on the posture of the pedestrian determined by the posture determination process unit (13), and a vehicle equipment control process unit (15) which controls equipment of the vehicle (10) at least according to the determination result of the object-to-be-avoided determination process unit (14).

Claims

1. A vehicle surroundings monitoring apparatus which monitors the surroundings of a vehicle from an image taken by a camera mounted on the vehicle, comprising:

an object extraction process unit which extracts an object existing around the vehicle from the image taken by the camera;

a pedestrian extraction process unit which extracts a pedestrian from the object extracted by the object extraction process unit;

a posture determination process unit which determines the posture of the pedestrian extracted by the pedestrian extraction process unit;

an object-to-be-avoided determination process unit which determines whether the object extracted by the object extraction process unit is an object to be avoided which must be avoided from coming into contact with the vehicle by executing a determination algorithm including at least a first determination process on the posture of the pedestrian determined by the posture determination process unit; and

a vehicle equipment control process unit which controls equipment of the vehicle at least according to a determination result of the object-to-be-avoided determination process unit.

2. The vehicle surroundings monitoring apparatus according to claim 1, wherein the equipment controlled by the vehicle equipment control process unit includes a device capable of calling attention of a driver of the vehicle.

3. The vehicle surroundings monitoring apparatus according to claim 1, wherein the equipment controlled by the vehicle equipment control process unit includes a device capable of controlling traveling behaviors of the vehicle.

4. The vehicle surroundings monitoring apparatus according to claim 1, further comprising:

a relative position detection process unit which sequentially detects relative positions of the object extracted by the object extraction process unit to the vehicle; and

a moving direction feature value calculation process unit which calculates a moving direction feature value representing a moving direction of the object relative to the vehicle based on the time series of the relative position of the object detected by the relative position detection process unit at longer time intervals than the execution period of the determination process of the posture determination process unit,

wherein the determination algorithm which is executed by the object-to-be-avoided determination process unit includes a second determination process on the moving direction feature value of the object calculated by the moving direction feature value calculation process unit as well as the first determination process;

the first determination process is performed to determine whether the posture of the pedestrian satisfies a predetermined first requirement; and

the second determination process is performed to determine whether the moving direction feature value of the object satisfies a predetermined second requirement, and

wherein the object-to-be-avoided determination process unit determines that the object is the object to be avoided when the determination result of the second determination process on the moving direction feature value of the object satisfies the second requirement or when the determination result of the second determination process does not satisfy the second requirement and further the determination result of the first determination process on the posture of the pedestrian satisfies the first requirement.

5. The vehicle surroundings monitoring apparatus according to claim 1, wherein the vehicle equipment control process unit controls the vehicle equipment in different modes between when the determination result of the second determination process on the moving direction feature value of the object satisfies the second requirement and when the determination result of the second determination process does not satisfy the second requirement and further the determination result of the first determination process on the posture of the pedestrian satisfies the first requirement.

6. The vehicle surroundings monitoring apparatus according to claim 1, wherein the posture determination process unit determines a degree of symmetry of the pedestrian in the vehicle width direction in the image with the symmetry considered as a feature value representing the posture of the pedestrian.

7. The vehicle surroundings monitoring apparatus according to claim 1, wherein the posture determination process unit determines a degree of tilt of a trunk axis of the pedestrian with the tilt of the trunk axis considered as a feature value representing the posture of the pedestrian.

8. The vehicle surroundings monitoring apparatus according to claim 1, further comprising:

a binarization process unit which extracts a binary area, in which a luminance value of a pixel in the image is equal to or higher than a predetermined threshold value, by binarizing the image taken by the camera; and

a run length data creation process unit which creates run length data of the binary area extracted by the binarization process unit,

wherein the object extraction process unit extracts objects existing around the road based on the run length data created by the run length data creation process unit, and

wherein the posture determination process unit determines the posture of the pedestrian extracted by the pedestrian extraction process unit based on the run length data created by the run length data creation process unit.

9. The vehicle surroundings monitoring apparatus according to claim 8, wherein the posture determination process unit includes:

a process unit which calculates point sequence data consisting of pixel coordinates of midpoints of respective lines of the run length data from the run length data created by the run length data creation process unit;

a process unit which calculates an approximate straight line approximate to the calculated point sequence data; and

a process unit which determines the posture of the pedestrian based on the calculated approximate straight line.

10. The vehicle surroundings monitoring apparatus according to claim 9, wherein the posture determination process unit determines the posture of the pedestrian based on an angle between the approximate straight line and a vertical axis in the image.

11. A vehicle surroundings monitoring apparatus having a computer including an interface circuit for accessing data of an image taken by a camera mounted on a vehicle, the apparatus monitoring the surroundings of the vehicle by arithmetic processing with the computer,

wherein the computer performs:

an object extraction process of extracting objects existing around the vehicle from the image taken by the camera;

a pedestrian extraction process of extracting a pedestrian from the objects extracted in the object extraction process;

a posture determination process of determining the posture of the pedestrian extracted in the pedestrian extraction process;

an object-to-be-avoided determination process of determining whether the object extracted in the object extraction process is an object to be avoided which must be avoided from coming into contact with the vehicle by executing a determination algorithm including at least a first determination process on the posture of the pedestrian determined in the posture determination process; and

a vehicle equipment control process of controlling equipment of the vehicle at least according to a determination result of the object-to-be-avoided determination process.

12. A vehicle surroundings monitoring method for monitoring the surroundings of a vehicle from an image taken by a camera mounted on the vehicle, comprising:

an object extraction step of extracting an object existing around the vehicle from the image taken by the camera;

a pedestrian extraction step of extracting a pedestrian from the object extracted in the object extraction step;

a posture determination step of determining the posture of the pedestrian extracted in the pedestrian extraction step;

an object-to-be-avoided determination step of determining whether the object extracted in the object extraction step is an object to be avoided which must be avoided from coming into contact with the vehicle by executing a determination algorithm including at least a first determination process on the posture of the pedestrian determined in the posture determination step; and

a vehicle equipment control step of controlling equipment of the vehicle at least according to a determination result of the object-to-be-avoided determination step.

13. A vehicle surroundings monitoring program for causing a computer to perform processing of monitoring the surroundings of a vehicle from an image taken by a camera mounted on the vehicle, the program causing the computer to perform:

an object extraction process of extracting an object existing around the vehicle from the image taken by the camera;

a pedestrian extraction process of extracting a pedestrian from the object extracted in the object extraction process;

a posture determination process of determining the posture of the pedestrian extracted in the pedestrian extraction process;

an object-to-be-avoided determination process of determining whether the object extracted in the object extraction process is an object to be avoided which must be avoided from coming into contact with the vehicle by executing a determination algorithm including at least a first determination process on the posture of the pedestrian determined in the posture determination process; and

a vehicle equipment control process of controlling equipment of the vehicle at least according to a determination result of the object-to-be-avoided determination process.