Vehicle surroundings monitoring device

• US 9,064,293 B2
• Filed: 06/15/2010
• Issued: 06/23/2015
• Est. Priority Date: 06/15/2010
• Status: Active Grant

• Current Assignee: Mitsubishi Electric Corporation
• Original Assignee: Mitsubishi Electric Corporation
• Inventors: Mitsugi, Tatsuya, Inoue, Satoru, Suzuki, Ryotaro, Shimizu, Hirokazu
• Primary Examiner(s): Yenke, Brian
• Application Number:
  US13/580,410
• Publication Number:
  US 20120320211A1
• Time in Patent Office: 1,834 Days
• Field of Search: 348/148, 340/435, 340/436, 382/103
• US Class Current: 1/1
• CPC Class Codes: B60R 2300/301 combining image information with other obstacle sensor information, e.g. using RADAR/LIDAR/SONAR sensors for estimating risk of collision
• CPC Class Codes: B60R 2300/602 with an adjustable viewpoint
• CPC Class Codes: G06T 3/00 Geometric image transformations in the plane of the image
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads

Abstract

A vehicle surroundings monitoring device carries out a position mapping process of expressing a positional relationship in the detection ranges of distance sensors 2 for detecting an obstacle 100 by coordinates on a camera image captured by a camera unit 3 having an image capture range in the surroundings of a vehicle, derives transformation parameters used for carrying out a mapping transformation in such a way that the obstacle detected by the distance sensors 2 is placed at a predetermined position on the camera image on which the position mapping process has been carried out, and carries out an image transformation on the camera image by using the transformation parameters in such a way that the obstacle is placed at the predetermined position.

Claims

1. A vehicle surroundings monitoring device comprising:

a position mapping unit that carries out a position mapping process of expressing a positional relationship in a detection range in surroundings of a vehicle of a distance sensor that detects an obstacle by coordinates on a camera image captured by a camera having an image capture range in said surroundings of the vehicle;

a mapping transformer that derives transformation parameters used for carrying out a mapping transformation in such a way that said obstacle detected by said distance sensor is placed at a predetermined position on said camera image on which the position mapping process has been carried out by said position mapping unit;

a camera image transformer that carries out an image transformation on said camera image by using the transformation parameters acquired by said mapping transformer in such a way that said obstacle is placed at said predetermined position;

an extra-position-mapping determination unit that determines that an obstacle is not existing in said camera image on which the position mapping process has been carried out and the obstacle is existing in the detection range by said position mapping unit; and

a guidance processor that provides guidance showing that the obstacle is existing out of said camera image based on a determination of the extra-position-mapping determination unit.

2. The vehicle surroundings monitoring device according to claim 1, wherein said camera image transformer returns its output to said position mapping unit to repeatedly carry out the image transformation on said camera image in such a way that said obstacle detected by said distance sensor is placed at said predetermined position on said camera image, and said guidance processor provides guidance showing that said obstacle does not appear on said camera image when said obstacle does not appear on the said camera image even though said camera image transformer has repeatedly carried out the image transformation on said camera image a predetermined number of times.

3. The vehicle surroundings monitoring device according to claim 1, wherein said camera image transformer carries out an image transformation process of enlarging or reducing the camera image.

4. The vehicle surroundings monitoring device according to claim 1, wherein said camera image transformer carries out an image transformation process of transforming the camera image into a bird'"'"'s-eye view.

5. The vehicle surroundings monitoring device according to claim 1, wherein said camera image transformer carries out an image transformation process of changing a reference point of the camera image.

6. The vehicle surroundings monitoring device according to claim 1, wherein said camera image transformer carries out the image transformation while switching between a screen mode in which to correct distortion of said camera image and a screen mode in which to carry out an image transformation process of transforming the camera image into a bird'"'"'s-eye view.

7. The vehicle surroundings monitoring device according to claim 1, wherein when said camera image is displayed as a rectangular image, said mapping transformer carries out the mapping transformation in such a way that the obstacle detected by said distance sensor is placed at a position close to one of sides of said rectangular image except a side nearest to the vehicle.

8. The vehicle surroundings monitoring device according to claim 1, wherein when said camera image is displayed as a rectangular image, said mapping transformer carries out the mapping transformation in such a way that the obstacle detected by said distance sensor is placed at a center of a side of said rectangular image which is furthest from the vehicle.

9. The vehicle surroundings monitoring device according to claim 1, wherein said mapping transformer carries out the mapping transformation in such a way that the obstacle detected by said distance sensor is placed at a center of said camera image.

10. The vehicle surroundings monitoring device according to claim 1, wherein said vehicle surroundings monitoring device includes an image superimposer that superimposes an image showing the position of the obstacle on the image on which the image transformation has been carried out by said camera image transformer.

11. The vehicle surroundings monitoring device according to claim 1, wherein said vehicle surroundings monitoring device includes a compositor that creates an image in which both an output of said camera and an output of said camera image transformer are displayed on an identical screen, and an image superimposer that superimposes both the camera image captured by said camera and the image on which the image transformation has been carried out by said camera image transformer on the image created by said compositor.

12. The vehicle surroundings monitoring device according to claim 1, wherein said vehicle surroundings monitoring device includes an attention attracting sound generator that provides guidance about the image transformation by voice when said camera image transformer carries out the image transformation.

13. The vehicle surroundings monitoring device according to claim 1, wherein said guidance processor superimposes an image showing that said obstacle does is not existing on said camera image on which the image transformation has been carried out by said camera image transformer to notify to that effect.

14. The vehicle surroundings monitoring device according to claim 1, wherein said guidance processor notifies that said obstacle is not existing on said camera image by guiding voice.

15. The vehicle surroundings monitoring device according to claim 1, wherein said camera unit starts a capturing operation when being triggered by detection of an obstacle by said distance sensor.

16. The vehicle surroundings monitoring device according to claim 1, wherein the position mapping process includes transforming a position vector of a detected obstacle into coordinates on the camera image captured by the camera.