Vehicle surroundings monitoring apparatus, vehicle, vehicle surroundings monitoring method, and vehicle surroundings monitoring program

• US 7,957,556 B2
• Filed: 04/13/2007
• Issued: 06/07/2011
• Est. Priority Date: 04/25/2006
• Status: Active Grant

• Current Assignee: Arriver Software AB (Qualcomm, Inc.)
• Original Assignee: Honda Motor Co., Ltd. (Honda Motor Company)
• Inventors: Hattori, Hiroshi, Takahashi, Akio, Aimura, Makoto
• Primary Examiner(s): Le; Vu
• Assistant Examiner(s): LIEW, ALEX KOK SOON
• Application Number:
  US11/787,050
• Publication Number:
  US 20070248245A1
• Time in Patent Office: 1,516 Days
• Field of Search: 382/103, 701/28
• US Class Current: 382/103
• CPC Class Codes: B60W 30/095 Predicting travel path or likelihood of collision
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands

Abstract

An object such as a person is extracted from images captured by cameras (2R, 2L) to determine the degree of vertical symmetry of the object (step 33). In the case where the degree of vertical symmetry is determined to be high, the object is determined to be other than a living body (step 33a). In the case where the degree of vertical symmetry is determined to be low, the object is determined likely to be a living body (step 33b). In the case where determined likely to be a living body, the object is further determined to be a predetermined kind of living body (for example, a person) or not based on the image of the object (steps 34 and 35). This allows an extremely reliable and simple determination of whether the type of the object extracted from the image captured by the camera is a living body such as a person.

Claims

1. A vehicle surroundings monitoring apparatus comprising:

an object extraction process unit which extracts an object existing in an imaging area of a camera mounted on a vehicle from an image captured by the camera;

an object type determination process unit which determines the type of the extracted object on the basis of the image of the object in the captured image; and

a vertical symmetry determination process unit which determines whether a degree of vertical symmetry of the extracted object is high or low on the basis of the image of the object in the captured image,

wherein the object type determination process unit is constructed to determine the type of the extracted object with a distinction between a predetermined kind of living body including at least a person and other object types, and includes

a first-type determination process unit constructed to determine the type of the object to be other than a living body in the case where the degree of vertical symmetry of the object is determined to be high by the vertical symmetry determination process unit, and further includes

a second-type determination process unit constructed to determine whether the object is the predetermined kind of living body, on the basis of the image of the object in the captured image, for objects other than those each determined not to be a living body by the first-type determination process unit among the extracted objects, when the degree of vertical symmetry of the object is determined to be low by the vertical symmetry determination process unit.

2. The vehicle surroundings monitoring apparatus according to claim 1, wherein:

the camera is an infrared camera and the captured image is a grayscale image;

the object extraction process unit includes a process unit which extracts an area having a luminance value equal to or greater than a predetermined value as a binary object from the captured image, and a process unit which extracts an area including the binary object in the grayscale image as the image of a grayscale object on the basis of at least the binary object and the luminance value data of the grayscale image; and

the object whose degree of vertical symmetry is determined by the vertical symmetry determination process unit is the grayscale object.

3. The vehicle surroundings monitoring apparatus according to claim 1, further comprising:

an object-to-be-avoided determination process unit which determines whether the object extracted by the object extraction process unit is an object which must be avoided from coming into contact with the vehicle at least on the basis of the determination result of the object type determination process unit; and

a vehicle equipment control process unit which controls predetermined equipment mounted on the vehicle at least according to the determination result of the object-to-be-avoided determination process unit.

5. A vehicle equipped with the vehicle surroundings monitoring apparatus according to claim 1.

9. The vehicle surroundings monitoring apparatus according to claim 1, wherein:

said vertical symmetry determination process unit is operable to set symmetry determination mask areas, one each at upper and lower parts of the image, to determine luminance distribution in each of the mask areas and to compare a difference in luminance distributions of said mask areas with a predetermined threshold value, for determining whether the degree of vertical symmetry of the extracted object is high or low compared to said predetermined threshold value; and

a vertical length of each of the symmetry determination mask areas is less than one half of a vertical length of the image; and

wherein a width of each of the symmetry determination mask areas is equal to a width of the image.

10. The vehicle surroundings monitoring apparatus according to claim 1, wherein:

said vertical symmetry determination process unit is operable to set symmetry determination mask areas, one each at upper and lower parts of the image, to determine luminance distribution in each of the mask areas and to compare a difference in luminance distributions of said mask areas with a predetermined threshold value, for determining whether the degree of vertical symmetry of the extracted object is high or low compared to said predetermined threshold value; and

a vertical length of each of the symmetry determination mask areas by the vertical symmetry determination process unit is one half of a vertical length of the image; and

wherein a width of each of the symmetry determination mask areas is equal to a width of the image.

4. A vehicle surroundings monitoring apparatus having a computer, wherein the computer performs:

an object extraction process for extracting an object existing in an imaging area of a camera mounted on a vehicle from an image captured by the camera;

an object type determination process for determining the type of the extracted object with a distinction between a predetermined kind of living body including at least a person and other object types; and

a vertical symmetry determination process for determining whether a degree of vertical symmetry of the extracted object is high or low on the basis of the image of the object in the captured image,

wherein the object type determination process performed by the computer includes

a first-type determination process for determining the type of the object to be other than a living body in the case where the degree of vertical symmetry of the object is determined to be high by the determination result of the vertical symmetry determination process; and

a second-type determination process for determining whether the object is the predetermined kind of living body, on the basis of the image of the object in the captured image, for objects other than those each determined not to be a living body by the first-type determination process among the extracted objects, when the degree of vertical symmetry of the object is determined to be low by the determination result of the vertical symmetry determination process.

6. A vehicle equipped with the vehicle surroundings monitoring apparatus according to claim 4.

7. A vehicle surroundings monitoring method performed by a vehicle surroundings monitoring apparatus arranged in a vehicle, said method comprising:

a step of extracting an object existing in an imaging area of a camera mounted on a vehicle from an image captured by the camera;

a step of determining the type of the extracted object with a distinction between a predetermined kind of living body including at least a person and other object types; and

a step of determining whether a degree of vertical symmetry of the extracted object is high or low on the basis of the image of the object in the captured image,

wherein the step of determining the type of the extracted object includes

a first-type step of determining the type of the object to be other than a living body in the case where the degree of vertical symmetry is determined to be high by the step of determining the degree of vertical symmetry; and

a second-type step of determining whether the object is the predetermined kind of living body, on the basis of the image of the object in the captured image, for objects other than those each determined not to be a living body by the first-type step among the extracted objects, when the degree of vertical symmetry of the object is determined to be low by the step of determining the degree of vertical symmetry.

8. A vehicle surroundings monitoring program embodied on a computer-readable medium, and having the functions of causing a computer to perform:

a process for extracting an object existing in an imaging area of an imaging device mounted on a vehicle from an image captured by the imaging device;

a process for determining the type of the extracted object on the basis of the image of the object in the captured image;

a process for determining whether a degree of vertical symmetry of the extracted object is high or low on the basis of the image of the object in the captured image;

wherein

said process for determining the type of the extracted object determines with a distinction between a predetermined kind of living body including at least a person and other object types;

said process for determining the type of the extracted object comprising

a first-type determination process for determining the type of the object to be other than a living body in the case where the degree of vertical symmetry of the object is determined to be high by the determination result of the vertical symmetry determination process; and

a second-type determination process for determining whether the object is the predetermined kind of living body, on the basis of the image of the object in the captured image, for objects other than those each determined not to be a living body by the first-type determination process unit among the extracted objects, when the degree of vertical symmetry of the object is determined to be low by the determination result of the vertical symmetry determination process.