Vehicle surroundings monitoring device, and image production method/program

• US 7,110,021 B2
• Filed: 05/29/2003
• Issued: 09/19/2006
• Est. Priority Date: 05/31/2002
• Status: Expired due to Term

• Current Assignee: Matsushita Electric Industrial Company Limited (Panasonic Holdings Corporation)
• Original Assignee: Matsushita Electric Industrial Company Limited (Panasonic Holdings Corporation)
• Inventors: Nobori, Kunio, Sato, Satoshi, Nakagawa, Masamichi, Nakata, Mikiya
• Primary Examiner(s): Dastouri, Mehrdad
• Assistant Examiner(s): CZEKAJ, DAVID J
• Application Number:
  US10/447,523
• Publication Number:
  US 20030222983A1
• Time in Patent Office: 1,209 Days
• Field of Search: 348/143, 348/148, 348/45, 348/51, 348/52, 348/53, 348/54, 348/55, 348/56, 348/57, 348/58, 348/59, 348/60, 701/301, 701/211, 701/200, 340/937
• US Class Current: 348/148
• CPC Class Codes: G01S 11/12 using electromagnetic waves other than radio waves
• CPC Class Codes: G01S 5/16 using electromagnetic waves other than radio waves
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06T 7/97 Determining parameters from multiple pictures depth or shape recovery from multiple images G06T7/55; stereo camera calibration G06T7/85
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

A synthesized image showing the situation around a vehicle is produced from images taken with cameras capturing the surroundings of the vehicle and presented to a display. A position computation section computes position information and the reliability of the position information, for a plurality of points in the camera images. An image variable-synthesis section produces a first synthesized image from the camera images using the position information. An image fixed-synthesis section produces a second synthesized image by road surface projection, for example, without use of the position information. A synthesis scheme selection section selects either one of the first and second synthesized images according to the reliability as the synthesized image to be presented.

Claims

1. A vehicle surroundings monitoring device for producing a synthesized image showing the situation around a vehicle using images taken with at least one camera capturing the surroundings of the vehicle and presenting the synthesized image at a display unit, the device comprising:

a position computation section operable to compute position information for a plurality of points in the images taken with the at least one camera, and operable to compute a precision of the position information, wherein said precision represents a degree of a measurement error;

an image variable-synthesis section operable to produce a first synthesized image having a predetermined viewpoint which is different form a view point of the camera, based on the images taken with the at least one camera and the position information;

an image fixed-synthesis section operable to produce a second synthesized image having a predetermined view point which is different from a view point of the camera, based on the images taken with the at least one camera by a predetermined method without use of the position information; and

a synthesis scheme selection section operable to select either the first synthesized image or the second synthesized image based on the precision as the synthesized image to be presented at the display unit.

2. The device of claim 1, wherein the synthesis scheme selection section selects a synthesized image by additional use of the position information.

3. The device of claim 2, wherein the synthesis scheme selection section extracts a point belonging to a region used for image synthesis among the plurality of points by referring to the position information, and

the precision of the extracted point is used for selection of a synthesized image.

4. The device of claim 1, wherein the image variable-synthesis section and the image fixed-synthesis section perform image synthesis so that the first and second synthesized images match with each other in the position of the road surface.

5. The device of claim 1, wherein the synthesis scheme selection section selects image parts from the first and second synthesized images according to the precision, in place of selecting either one of the first and second synthesized images, and produces the synthesized image to be presented from the selected image parts.

6. the device of claim 1, wherein the synthesis scheme selection section superimposes on the selected synthesized image a predetermined indication corresponding to a feature of the synthesized image.

7. The device of claim 1, wherein the vehicle is provided with an obstacle sensor, and

the synthesis scheme selection section selects a synthesized image by additional use of distance information indicating a distance value from an obstacle obtained from the obstacle sensor.

8. The device of claim 7, wherein the synthesis scheme selection section evaluates the accuracy of the position information of the obstacle by referring to the distance information and the position information, and when determining that the accuracy is insufficient, does not select a synthesized image according to the precision, but selects the second synthesized image.

9. The device of claim 1, wherein the vehicle is provided with moving direction detection means for detecting the moving direction of the vehicle, and

the synthesis scheme selection section selects a synthesized image by additional use of the moving direction obtained from the moving direction detection means.

10. The device of claim 9, wherein the synthesis scheme selection specifies a point belonging to an image region located in the moving direction among the plurality of points, and

the precision of the specified point is used for selection of a synthesized image.

11. A vehicle surroundings monitoring device according to claim 1, wherein the position computation section computes the precision by which position information is calculated, including determining a degree at which computed position information includes errors.

12. An image production method for producing a synthesized image showing the situation around a vehicle using images taken with at least one camera capturing the surroundings of the vehicle, the method comprising:

computing position information for a plurality of points in the images taken with the at least one camera, and precision of the position information, wherein said precision represents a degree of a measurement error;

comparing the precision with a predetermined reference;

producing a first synthesized image viewed from a predetermined viewpoint which is different from a viewpoint of the at least one camera, based on the images taken with the at least one camera and position information if the precision is determined higher than the predetermined reference; and

producing a second synthesized image viewed from a predetermined viewpoint which is different from a viewpoint of the at least one camera, based on the images taken with the at least one camera by a predetermined method without use of the position information if the precision is determined lower than the predetermined reference.

13. The method of claim 12, wherein the step (1) comprises:

tracking a corresponding point in first and second camera images;

computing three-dimensional position coordinates of the specified corresponding point from original image coordinates of the point in the first and second camera images, and determining the original image coordinates and the three-dimensional position coordinates as the position information of the corresponding point;

inversely computing image coordinates of the corresponding point in the first and second camera images from the three-dimensional position coordinates; and

determining the precision of the position information using the inversely-computed image coordinates and the original image coordinates.

14. A program enabling a computer to execute a method for producing a synthesized image showing the situation around a vehicle from images taken with a camera capturing the surroundings of the vehicle, the method comprising the steps of:

(1) computing position information and precision of the position information, for a plurality of points in the camera images;

(2) comparing the precision computed in the step (1) with a predetermined reference;

(3) producing the synthesized image viewed from a predetermined viewpoint using the camera images and the position information computed in the step (1) if the precision is determined higher than the predetermined reference in the step (2); and

(4) producing the synthesized image from the camera images by a predetermined method without use of the position information if the precision is determined lower than the predetermined reference in the step (2).

15. A vehicle surroundings monitoring device for producing a synthesized image showing the situation around a vehicle, said the device comprising:

at least one camera operable to capture the surroundings of the vehicle;

a position computation section operable to compute position information for a plurality of points in images taken with the at least one camera, and operable to compute a measurement error for the position information;

an image variable-synthesis section operable to produce a first synthesized image having a predetermined view point which is different from a view point at which the images were captured by the camera, based on the images taken with the at least one camera and position information;

an image fixed-synthesis section operable to produce a second synthesized image having a predetermined view point which is different from a view point at which the images were captured by the camera, based on the images taken with the at least one camera by a predetermined method without use of the position information;

a synthesis scheme section operable to select as the synthesized image either the first synthesized image or the second synthesized image based on the measurement error of the position information; and

a display unit operable to display the synthesized image.