Apparatus for vehicle surroundings monitoring and method thereof

• US 20050111698A1
• Filed: 11/16/2004
• Published: 05/26/2005
• Est. Priority Date: 11/20/2003
• Status: Abandoned Application

• Current Assignee: Nissan Motor Co., Ltd.
• Original Assignee: Nissan Motor Co., Ltd.
• Inventors: Kawai, Akio
• Application Number:
  US10/988,531
• Publication Number:
  US 20050111698A1
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: B60R 1/24 in front of the vehicle
• CPC Class Codes: B60R 1/30 providing vision in the non-visible spectrum, e.g. night or infrared vision
• CPC Class Codes: B60R 2300/103 using camera systems provided with artificial illumination device, e.g. IR light source
• CPC Class Codes: B60R 2300/106 using night vision cameras
• CPC Class Codes: B60R 2300/205 using a head-up display
• CPC Class Codes: B60R 2300/30 characterised by the type of image processing
• CPC Class Codes: B60R 2300/302 combining image information with GPS information or vehicle data, e.g. vehicle speed, gyro, steering angle data
• CPC Class Codes: B60R 2300/305 merging camera image with lines or icons
• CPC Class Codes: B60R 2300/404 triggering from stand-by mode to operation mode
• CPC Class Codes: B60R 2300/60 characterised by monitoring and displaying vehicle exterior scenes from a transformed perspective
• CPC Class Codes: B60R 2300/8033 for pedestrian protection
• CPC Class Codes: B60R 2300/8053 for bad weather conditions or night vision
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition
• CPC Class Codes: H04N 5/33 Transforming infrared radiation cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20

Abstract

An aspect of the present invention provides a vehicle surroundings monitoring device that includes an object extracting unit configured to extract objects that emit infrared rays from a photographed infrared image, a pedestrian candidate extracting unit configured to extract pedestrian candidates based on the shape of the images of objects extracted by the object extracting unit, and a structure exclusion processing unit configured to exclude structures from the pedestrian candidates based on the gray levels of the images of the pedestrian candidates.

Claims

1. A vehicle surroundings monitoring device, comprising:

an object extracting unit configured to extract objects that emit infrared rays from a photographed infrared image;

a pedestrian candidate extracting unit configured to extract pedestrian candidates based on the shape of the images of objects extracted by the object extracting unit; and

a structure exclusion processing unit configured to exclude structures from the pedestrian candidates based on the gray levels of the images of the pedestrian candidates.

2. The vehicle surroundings monitoring device as claimed in claim 1, wherein

the pedestrian candidate extracting unit comprises:

a rectangle setting unit configured to set rectangular frames circumscribing the images of objects extracted by the object extracting unit;

a vertical to horizontal dimension ratio calculating unit configured to calculate the vertical to horizontal dimension ratios of the rectangular frames set by the rectangle setting unit; and

a pedestrian determining unit configured to determine that an object is a passenger candidate when the vertical to horizontal dimension ratio of the corresponding frame is within a prescribed range of numerical values.

3. The vehicle surroundings monitoring device as claimed in claim 1, wherein

the pedestrian determining unit determines that an object is a passenger candidate when the vertical to horizontal dimension ratio is in the range from 4:

1 to 4;

3.

4. The vehicle surroundings monitoring device as claimed in claim 1, wherein

the structure exclusion processing unit comprises:

an average gray level calculating unit configured to calculate the average value of the gray level distribution of an image of a pedestrian candidate;

a gray level dispersion calculating unit configured to calculate the dispersion value of the gray level distribution of an image of a pedestrian candidate; and

a structure determining unit configured to determine that the image of a pedestrian candidate is a structure and exclude the image from the pedestrian candidates when the average gray level value of the image of the pedestrian candidate is equal to or larger than a prescribed value or when the gray level dispersion value of the image of the pedestrian candidate is equal to or below a prescribed value.

5. The vehicle surroundings monitoring device as claimed in claim 1, wherein

the structure exclusion processing unit comprises:

an average gray level calculating unit configured to calculate the average value of the gray level distribution of an image of a pedestrian candidate;

a gray level dispersion calculating unit configured to calculate the dispersion value of the gray level distribution of an image of a pedestrian candidate; and

a structure determining unit configured to determine that the image of a pedestrian candidate is a structure when the average gray level value of the image of the pedestrian candidate is equal to or larger than a prescribed value and the gray level dispersion value of the image of the pedestrian candidate is equal to or below a prescribed value.

6. The vehicle surroundings monitoring device as claimed in claim 1, further comprising an image processing unit electrically coupled to an infrared camera, the image processing unit configured to obtain an infrared image from the infrared camera and store the infrared image;

and

wherein the object extracting unit is configured to extract objects using an infrared image acquired by the mage processing unit.

7. The vehicle surroundings monitoring device as claimed in claim 6, further comprising a display device provided in front of the driver'"'"'s seat of the vehicle and configured to display an infrared image photographed by the infrared camera;

and

wherein the display control unit is configured to emphasize the images of the pedestrian candidates that have not been determined to be structures by the structure exclusion processing unit.

8. The vehicle surroundings monitoring device as claimed in claim 3, wherein

the display control unit is configured to emphasize the images of the pedestrian candidates that have not been determined to be structures by enclosing said images in frames drawn with a dotted line, broken line, chain line, or a solid bold line.

9. The vehicle surroundings monitoring device as claimed in claim 6, further comprising a vehicle speed sensor configured to detect the speed of the vehicle in which the vehicle speed surroundings monitoring device is installed;

and

wherein the display control unit is configured to display the infrared image on the display device when the vehicle speed is equal to or above a prescribed value.

10. A vehicle surroundings monitoring device, comprising:

an object extracting means for extracting objects that emit infrared rays from a photographed infrared image;

a pedestrian candidate extracting means for extracting pedestrian candidates based on the shape of the images of objects extracted by the object extracting unit; and

a structure exclusion processing means for excluding structures from the pedestrian candidates based on the gray levels of the images of the pedestrian candidates.

11. A vehicle surroundings monitoring method, comprising:

emitting infrared rays from a vehicle;

receiving infrared rays reflected from objects existing in the vicinity of the vehicle and creating an infrared image;

extracting from the infrared image those objects that reflect a quantity of infrared rays equal to or exceeding a prescribed quantity;

extracting the images of pedestrian candidates based on the shapes of the images of the extracted objects;

determining if the pedestrian candidates are structures based on the gray levels of the images of the pedestrian candidates; and

determining that the pedestrian candidates that have not been determined to be structures are pedestrians.

12. The vehicle surroundings monitoring method as claimed in claim 11, wherein

the procedure of extracting the images of pedestrian candidates based on the shapes of the images of the extracted objects, comprises:

setting a rectangular frame circumscribing the images of objects extracted by the object extracting unit;

calculating the vertical to horizontal dimension ratios of the rectangular frames set by the rectangle setting unit; and

determining that the objects whose images are circumscribed by rectangular frames having vertical to horizontal dimension ratios within a prescribed range of numerical values are pedestrian candidates.

13. The vehicle surroundings monitoring method as claimed in claim 12, wherein

the vertical to horizontal dimension ratio is in the range from 4:

1 to 4;

3

14. The vehicle surroundings monitoring method as claimed in claim 11, wherein

the procedure of the determining if the pedestrian candidates are structures based on the gray levels of the images of the pedestrian candidates, comprises:

calculating the average value of the gray level distribution of an image of a pedestrian candidate;

calculating the dispersion value of the gray level distribution of an image of a pedestrian candidate; and

determining that the image of a pedestrian candidate is a structure and exclude the image from the pedestrian candidates when the average gray level value of the image of the pedestrian candidate is equal to or larger than a prescribed value or when the gray level dispersion value of the image of the pedestrian candidate is equal to or below a prescribed value.

15. The vehicle surroundings monitoring method as claimed in claim 11, wherein

the procedure of the determining if the pedestrian candidates are structures based on the gray levels of the images of the pedestrian candidates, comprises

calculating the average value of the gray level distribution of an image of a pedestrian candidate;

calculating the dispersion value of the gray level distribution of an image of a pedestrian candidate; and

determining that the image of a pedestrian candidate is a structure when the average gray level value of the image of the pedestrian candidate is equal to or larger than a prescribed value and the gray level dispersion value of the image of the pedestrian candidate is equal to or below a prescribed value.

16. The vehicle surroundings monitoring method as claimed in claim 11, further comprising:

emphasized displaying images of the pedestrian candidates that have not been determined to be structures.

17. The vehicle surroundings monitoring method as claimed in claim 16, wherein

the emphasized displaying is carried out by enclosing said images in frames drawn with a dotted line, broken line, chain line, or a solid bold line.

18. The vehicle surroundings monitoring device as claimed in claim 16, wherein

the emphasized displaying is carried out when the vehicle speed is equal to or above a prescribed value.