Vehicle surroundings monitoring device, and image production method/program

• US 20030222983A1
• Filed: 05/29/2003
• Published: 12/04/2003
• Est. Priority Date: 05/31/2002
• Status: Active Grant

• Current Assignee: Matsushita Electric Industrial Company Limited (Panasonic Holdings Corporation)
• Original Assignee: Matsushita Electric Industrial Company Limited (Panasonic Holdings Corporation)
• Inventors: Nobori, Kunio, Sato, Satoshi, Nakagawa, Masamichi
• Granted Patent: US 7,110,021 B2
• Time in Patent Office: Days
• US Class Current: 348/148
• CPC Class Codes: G01S 11/12 using electromagnetic waves other than radio waves
• CPC Class Codes: G01S 5/16 using electromagnetic waves other than radio waves
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06T 7/97 Determining parameters from multiple pictures depth or shape recovery from multiple images G06T7/55; stereo camera calibration G06T7/85
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

A synthesized image showing the situation around a vehicle is produced from images taken with cameras capturing the surroundings of the vehicle and presented to a display. A position computation section computes position information and the reliability of the position information, for a plurality of points in the camera images. An image variable-synthesis section produces a first synthesized image from the camera images using the position information. An image fixed-synthesis section produces a second synthesized image by road surface projection, for example, without use of the ,..position information. A synthesis scheme selection section selects either one of the first and second synthesized images according to the reliability as the synthesized image to be presented.

Claims

1. A vehicle surroundings monitoring device for producing a synthesized image showing the situation around a vehicle from images taken with a camera capturing the surroundings of the vehicle and presenting the synthesized image, the device comprising:

a position computation section for computing position information and reliability of the position information, for a plurality of points in the camera images;

an image variable-synthesis section for producing a first synthesized image from the camera images using the position information;

an image fixed-synthesis section for producing a second synthesized image from the camera images by a predetermined method without use of the position information; and

a synthesis scheme selection section for selecting either one of the first and second synthesized images according to the reliability as the synthesized image to be presented.

2. The device of claim 1, wherein the synthesis scheme selection section selects a synthesized image by additional use of the position information.

3. The device of claim 2, wherein the synthesis scheme selection section extracts a point belonging to a region used for image synthesis among the plurality of points by referring to the position information, and

the reliability of the extracted point is used for selection of a synthesized image.

4. The device of claim 1, wherein the image variable-synthesis section and the image fixed-synthesis section perform image synthesis so that the first and second synthesized images match with each other in the position of the road surface.

5. The device of claim 1, wherein the synthesis scheme selection section selects image parts from the first and second synthesized images according to the reliability, in place of selecting either one of the fist and second synthesized images, and produces the synthesized image to be presented from the selected image parts.

6. The device of claim 1, wherein the synthesis scheme selection section superimposes on the selected synthesized image a predetermined indication corresponding to a feature of the synthesized image.

7. The device of claim 1, wherein the vehicle is provided with an obstacle sensor, and

the synthesis scheme selection section selects a synthesized image by additional use of distance information indicating a distance value from an obstacle obtained from the obstacle sensor.

8. The device of claim 7, wherein the synthesis scheme selection section evaluates the accuracy of the position information of the obstacle by referring to the distance information and the position information, and when determining that the accuracy is insufficient, does not select a synthesized image according to the reliability, but selects the second synthesized image.

9. The device of claim 1, wherein the vehicle is provided with moving direction detection means for detecting the moving direction of the vehicle, and

the synthesis scheme selection section selects a synthesized image by additional use of the moving direction obtained from the moving direction detection means.

10. The device of claim 9, wherein the synthesis scheme selection section specifies a point belonging to an image region located in the moving direction among the plurality of points, and

the reliability of the specified point is used for selection of a synthesized image.

11. An image production method for producing a synthesized image showing the situation around a vehicle from images taken with a camera capturing the surroundings of the vehicle, the method comprising the steps of:

(1) computing position information and reliability of the position information, for a plurality of points in the camera images;

(2) comparing the reliability computed in the step (1) with a predetermined reference;

(3) producing the synthesized image from the camera images using the position information computed in the step (1) if the reliability is determined higher than the predetermined reference in the step (2); and

(4) producing the synthesized image from the camera images by a predetermined method without use of the position information if the reliability is determined lower than the predetermined reference in the step (2).

12. The method of claim 11, wherein the step (1) comprises:

tracking a corresponding point in first and second camera images;

computing three-dimensional position coordinates of the specified corresponding point from original image coordinates of the point in the first and second camera images, and determining the original image coordinates and the three-dimensional position coordinates as the position information of the corresponding point;

inversely computing image coordinates of the corresponding point in the first and second camera images from the three-dimensional position coordinates; and

determining the reliability of the position information using the inversely-computed image coordinates and the original image coordinates.

13. A program enabling a computer to execute a method for producing a synthesized image showing the situation around a vehicle from images taken with a camera capturing the surroundings of the vehicle, the method comprising the steps of:

(1) computing position information and reliability of the position information, for a plurality of points in the camera images;

(2) comparing the reliability computed in the step (1) with a predetermined reference;

(3) producing the synthesized image from the camera images using the position information computed in the step (1) if the reliability is determined higher than the predetermined reference in the step (2); and

(4) producing the synthesized image from the camera images by a predetermined method without use of the position information if the reliability is determined lower than the predetermined reference in the step (2).