Vehicle environment monitoring apparatus

• US 20080260207A1
• Filed: 10/16/2007
• Published: 10/23/2008
• Est. Priority Date: 10/31/2006
• Status: Active Grant

• Current Assignee: Arriver Software AB (Qualcomm, Inc.)
• Original Assignee: Honda Motor Co., Ltd. (Honda Motor Company)
• Inventors: Nagaoka, Nobuharu, Hashimoto, Hideki
• Granted Patent: US 7,925,050 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06T 7/0002 Inspection of images, e.g. flaw detection
• CPC Class Codes: G06T 7/32 using correlation-based methods
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads

Abstract

A vehicle environment monitoring apparatus capable of extracting an image of a monitored object in an environment around a vehicle by separating the same from the background image with a simple configuration having a single camera mounted on the vehicle is provided. The apparatus includes a first image portion extracting processing unit to extract first image portions (A1, A2) considered to be the head of a pedestrian from a currently picked up image and a previously picked up image by an infrared camera, a mask area setting processing unit to set mask areas (M1(0,0), M1(1,0), . . . , M1(5,8)) around the first image portion (A1) in the currently picked up image, and an object extracting processing unit to carry out pattern matching for the previously picked up image by a comparison pattern obtained through affine transformation of each mask area at a change rate (Rate) between the first image portions (A1, A2), and to set an area (Ar1) including the first image portion (A2) and a second image portion (M2(1,3), M2(2,3), . . . , M2(3,6)) where a displacement amount between the position (black point) corresponding to the centroid of the mask area and the matching position is smaller than a predetermined threshold value to be an image area of the monitored object.

Claims

1. A vehicle environment monitoring apparatus detecting a monitored object in an environment around a vehicle from an image picked up by a single camera mounted on the vehicle, comprising:

a first image portion extracting processing unit to extract a first image portion having a predetermined feature value from each of a first picked-up image and a second picked-up image picked up by the camera at different time points;

a second image portion extracting processing unit to extract, from each of the first and second picked-up images, a second image portion having a change rate of not greater than a predetermined level in relative positional relation with the first image portion between the first and second picked-up images; and

an object detecting processing unit to detect an area including the first and second image portions in the first or second picked-up image as an image area of the same monitored object in the environment around the vehicle.

2. The vehicle environment monitoring apparatus according to claim 1, further comprising:

a real space distance grasping processing unit to grasp a distance between the vehicle and a position in a real space corresponding to the image area including the first and second image portions; and

an object type determining processing unit to calculate a size of the object in the real space corresponding to the image area including the first and second image portions using the distance grasped by the real space distance grasping processing unit, and to determine a type of the object based on the size.

3. The vehicle environment monitoring apparatus according to claim 2, wherein the object type determining processing unit calculates at least one of a height and a width of the object in the real space as a size of the object in the real space corresponding to the image area including the first and second image portions.

4. The vehicle environment monitoring apparatus according to claim 1, further comprising a mask area setting processing unit to set a mask area in the first picked-up image in a predetermined position around the first image portion, wherein

the second image portion extracting processing unit extracts, as the second image portion, an image area in the second picked-up image where the degree of correlation with a feature value of the mask area is not less than a predetermined level and where an amount of displacement from a reference position with respect to the first image portion in the second picked-up image, corresponding to the position of the mask area with respect to the first image portion in the first picked-up image, is not greater than a predetermined value.

5. The vehicle environment monitoring apparatus according to claim 4, wherein at least one of an area upward of the first image portion and an area downward of the first image portion is defined as the mask area.

6. The vehicle environment monitoring apparatus according to claim 4, wherein at least one of an area leftward of the first image portion and an area rightward of the first image portion is defined as the mask area.

7. The vehicle environment monitoring apparatus according to claim 4, wherein

the camera is an infrared camera,

the first image portion extracting processing unit extracts the first image portion from an infrared gray scale image picked up by the camera, and

the second image portion extracting processing unit extracts the second image portion by correlation processing with the mask area in the gray scale image.

8. The vehicle environment monitoring apparatus according to claim 4, further comprising:

a real space distance grasping processing unit to grasp a distance between the vehicle and a position in a real space corresponding to the image area including the first and second image portions; and

an object type determining processing unit to calculate a size of the object in the real space corresponding to the image area including the first and second image portions using the distance grasped by the real space distance grasping processing unit, and to determine a type of the object based on the size.

9. The vehicle environment monitoring apparatus according to claim 8, wherein the object type determining processing unit calculates at least one of a height and a width of the object in the real space as the size of the object in the real space corresponding to the image area including the first and second image portions.

10. A vehicle having the vehicle environment monitoring apparatus as recited in claim 1 mounted thereon.

11. A vehicle environment monitoring apparatus including a computer having an interface circuit for accessing data of an image picked up by a single camera mounted on a vehicle, and detecting a monitored object in an environment around the vehicle by the computer, the computer executing:

first image portion extracting processing of extracting a first image portion having a predetermined feature value from each of a first picked-up image and a second picked-up image picked up by the camera at different time points;

second image portion extracting processing of extracting, from each of the first and second picked-up images, a second image portion having a change rate of not greater than a predetermined level in relative positional relation with the first image portion between the first and second picked-up images; and

object detecting processing of detecting an area including the first and second image portions in the first or second picked-up image as an image area of the same monitored object in the environment around the vehicle.

12. A vehicle having the vehicle environment monitoring apparatus as recited in claim 11 mounted thereon.

13. A vehicle environment monitoring program for causing a computer having an interface circuit for accessing data of an image picked up by a single camera mounted on a vehicle to implement a function of detecting a monitored object in an environment around the vehicle from the image, the program causing the computer to serve as:

a first image portion extracting processing unit to extract a first image portion having a predetermined feature value from each of a first picked-up image and a second picked-up image picked up by the camera at different time points;

a second image portion extracting processing unit to extract, from each of the first and second picked-up images, a second image portion having a change rate of not greater than a predetermined level in relative positional relation with the first image portion between the first and second picked-up images; and

an object detecting processing unit to detect an area including the first and second image portions in the first or second picked-up image as an image area of the same monitored object in the environment around the vehicle.

14. A vehicle environment monitoring method for detecting a monitored object in an environment around a vehicle by a computer having an interface circuit for accessing data of an image picked up by a single camera mounted on the vehicle, the method comprising:

a first image portion extracting step, executed by the computer, of extracting a first image portion having a predetermined feature value from each of a first picked-up image and a second picked-up image picked up by the camera at different time points;

a second image portion extracting step, executed by the computer, of extracting, from each of the first and second picked-up images, a second image portion having a change rate of not greater than a predetermined level in relative positional relation with the first image portion between the first and second picked-up images; and

an object detecting step, executed by the computer, of detecting an area including the first and second image portions in the first or second picked-up image as an image area of the same monitored object in the environment around the vehicle.