Vehicle surroundings monitoring apparatus

• US 20070291987A1
• Filed: 06/06/2007
• Published: 12/20/2007
• Est. Priority Date: 06/16/2006
• Status: Active Grant

• Current Assignee: Arriver Software AB (Qualcomm, Inc.)
• Original Assignee: Honda Motor Co., Ltd. (Honda Motor Company)
• Inventors: Saka, Masakazu, Koike, Hiroyuki, Taniguchi, Fuminori
• Granted Patent: US 8,005,266 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition

Abstract

A vehicle surroundings monitoring apparatus capable of recognizing the type of object existing around a vehicle with accuracy. The vehicle surroundings monitoring apparatus includes a process unit, which extracts a first image portion HP_—1 likely to be the head of an object and sets a reference mask area MASK_C including HP_—1, a left-hand mask area MASK_L near the left side of the reference mask area MASK_C, and a right-hand mask area MASK_R near the right side of the reference mask area MASK_C, and a process unit which searches for a second image portion presumed to be a leg image within a lower search area AREA_3 when a difference between an average luminance AVE_C of the reference mask area MASK_C and an average luminance AVE_L of the left-hand mask area MASK_L or a difference between the average luminance AVE_C and an average luminance AVE_R of the right-hand mask area MASK_R is equal to or greater than a predetermined level and which recognizes that the object type is “pedestrian” in the case where a plurality of second image portions are detected.

Claims

1. A vehicle surroundings monitoring apparatus which monitors the surroundings of a vehicle by using an image captured by a camera mounted on the vehicle, comprising:

a first image portion extraction process unit which extracts a first image portion likely to be an image of a first predetermined portion in a predetermined type of object in a real space from the captured image;

a reference mask area setting process unit which sets a predetermined area including the first image portion as a reference mask area in the captured image;

a lower search area setting process unit which sets a lower search area under the reference mask area in the captured image;

a second image portion search process unit which searches for a second image portion having a preset feature value corresponding to a second predetermined portion in the predetermined type of object within the lower search area; and

an object type determination process unit which determines the type of object in the real space corresponding to the first image portion according to whether the second image portion search process unit detects a plurality of the second image portions.

2. A vehicle surroundings monitoring apparatus according to claim 1, wherein:

the predetermined type is “

pedestrian” and

the first predetermined portion is a pedestrian'"'"'s head; and

the object type determination process unit determines that the type of object in the real space corresponding to the first image portion is “

pedestrian”

in the case where the second image portion search process unit detects the plurality of the second image portions.

3. A vehicle surroundings monitoring apparatus according to claim 2, wherein the second predetermined portion is a pedestrian'"'"'s leg.

4. A vehicle surroundings monitoring apparatus according to claim 3, further comprising a comparative mask area setting process unit which sets a left-hand mask area near the left side of the reference mask area and a right-hand mask area near the right side of the reference mask area in the captured image, wherein:

the lower search area setting process unit sets the lower search area under the reference mask area, the left-hand mask area, and the right-hand mask area;

the object type determination process unit determines that the type of object in the real space corresponding to the first image portion is “

pedestrian”

in the case where a luminance difference between the reference mask area and the left-hand mask area is equal to or greater than a first predetermined level and a luminance difference between the reference mask area and the right-hand mask area is equal to or greater than a second predetermined level; and

the object type determination process unit causes the second image portion search process unit to search for the second image portion within the lower search area in the case where the luminance difference between the reference mask area and the left-hand mask area is smaller than the first predetermined level or in the case where the luminance difference between the reference mask area and the right-hand mask area is smaller than the second predetermined level and, in the case where a plurality of the second image portions are detected, it determines that the type of object in the real space corresponding to the first image portion is “

pedestrian.”

5. A vehicle surroundings monitoring apparatus according to claim 3, wherein:

the object type determination process unit determines that the type of object in the real space corresponding to the first image portion is “

pedestrian”

in the case where a luminance distribution in the reference mask area satisfies a predetermined condition; and

the object type determination process unit causes the second image portion search process unit to search for the second image portion within the lower search area unless the luminance distribution in the reference mask area satisfies the predetermined condition and, in the case where a plurality of the second image portions are detected, it determines that the type of object in the real space corresponding to the first image portion is “

pedestrian.”

6. A vehicle surroundings monitoring apparatus according to claim 3, further comprising:

a real space distance recognition process unit which recognizes a distance between a real space position corresponding to the image portion included in the captured image and the vehicle; and

a comparative mask area setting process unit which sets a left-hand mask area near the left side of the reference mask area and a right-hand mask area near the right side of the reference mask area in the captured image,

wherein;

the lower search area setting process unit sets the lower search area under the reference mask area, the left-hand mask area, and the right-hand mask area;

the object type determination process unit determines that the type of object in the real space corresponding to the first image portion is “

pedestrian”

in the case where a difference between a reference distance, which is a distance between the real space position corresponding to the first image portion and the vehicle, and a left-hand comparative distance, which is a distance between the real space position corresponding to the image portion of the left-hand mask area and the vehicle, is equal to or greater than a first predetermined distance and in the case where a difference between the reference distance and a right-hand comparative distance, which is a distance between a real space position corresponding to the image portion of the right-hand mask area and the vehicle, is equal to or greater than a second predetermined distance; and

the object type determination process unit causes the second image portion search process unit to search for the second image portion within the lower search area in the case where the difference between the reference distance and the left-hand comparative distance is smaller than the first predetermined distance or in the case where the difference between the reference distance and the right-hand comparative distance is smaller than the second predetermined distance and, in the case where a plurality of the second image portions are detected, it determines that the type of object in the real space corresponding to the first image portion is “

pedestrian.”

7. A vehicle equipped with the vehicle surroundings monitoring apparatus according to claim 1.

8. A vehicle surroundings monitoring apparatus which monitors the surroundings of a vehicle by using an image captured by a camera mounted on the vehicle, comprising:

a first image portion extraction process unit which extracts a first image portion likely to be an image of a first predetermined portion in a predetermined type of object in a real space from the captured image;

a reference mask area setting process unit which sets a predetermined area including the first image portion as a reference mask area in the captured image;

a lower search area setting process unit which sets a lower search area under the reference mask area in the captured image;

a third image portion search process unit which searches for a third image portion having a third feature value preset according to a case where the predetermined type of a plurality of objects exist within the lower search area; and

an object type determination process unit which determines the type of object in the real space corresponding to the first image portion according to whether the third image portion search process unit detects the third image portion.

9. A vehicle surroundings monitoring apparatus according to claim 8, wherein:

the predetermined type is “

pedestrian” and

the first predetermined portion is a pedestrian'"'"'s head;

the third feature value is applicable to a case where the image portion includes an image of legs of a plurality of pedestrians; and

the object type determination process unit determines that the type of object in the real space corresponding to the first image portion is “

pedestrian”

in the case where the third image portion search process unit detects the third image portion.

10. A vehicle equipped with the vehicle surroundings monitoring apparatus according to claim 8.

11. A vehicle surroundings monitoring apparatus which monitors the surroundings of a vehicle by using a computer which is provided in the apparatus and includes an interface circuit for accessing an image captured by a camera mounted on the vehicle,

wherein the computer performs:

a first image portion extraction process for extracting a first image portion likely to be an image of a first predetermined portion in a predetermined type of object in a real space from the captured image;

a reference mask area setting process for setting a predetermined area including the first image portion as a reference mask area in the captured image;

a lower search area setting process for setting a lower search area under the reference mask area in the captured image;

a second image portion search process for searching for a second image portion having a preset feature value corresponding to a second predetermined portion in the predetermined type of object within the lower search area; and

an object type determination process for determining the type of object in the real space corresponding to the first image portion according to whether a plurality of the second image portions are detected in the second image portion search process.

12. A vehicle equipped with the vehicle surroundings monitoring apparatus according to claim 11.

13. A vehicle surroundings monitoring program which causes a computer having an interface circuit for accessing an image captured by a camera mounted on the vehicle to perform a function of monitoring the surroundings of the vehicle, the program causing the computer to function as:

a first image portion extraction process unit which extracts a fist image portion likely to be an image of a first predetermined portion in a predetermined type of object in a real space from the captured image;

a reference mask area setting process unit which sets a predetermined area including the first image portion as a reference mask area in the captured image;

a lower search area setting process unit which sets a lower search area under the reference mask area in the captured image;

a second image portion search process unit which searches for a second image portion having a preset feature value corresponding to a second predetermined portion in the predetermined type of object within the lower search area; and

an object type determination process unit which determines the type of object in the real space corresponding to the first image portion according to whether the second image portion search process unit detects a plurality of the second image portions.

14. A vehicle surroundings monitoring method of monitoring the surroundings of a vehicle by using a computer having an interface circuit for accessing an image captured by a camera mounted on the vehicle, the method comprising:

a first image portion extraction step of causing the computer to extract a first image portion likely to be an image of a first predetermined portion in a predetermined type of object in a real space;

a reference mask area setting step of causing the computer to set a predetermined area including the first image portion as a reference mask area in the captured image;

a lower search area setting step of causing the computer to set a lower search area under the reference mask area in the captured image;

a second image portion search step of causing the computer to search for a second image portion having a preset feature value corresponding to a second predetermined portion in the predetermined type of object within the lower search area; and

an object type determination step of causing the computer to determine the type of object in the real space corresponding to the first image portion according to whether a plurality of the second image portions are detected in the second image portion search step.