Vehicle surroundings display device and image providing system

• US 7,176,959 B2
• Filed: 09/09/2002
• Issued: 02/13/2007
• Est. Priority Date: 09/07/2001
• Status: Active Grant

• Current Assignee: Matsushita Electric Industrial Company Limited (Panasonic Holdings Corporation)
• Original Assignee: Matsushita Electric Industrial Company Limited (Panasonic Holdings Corporation)
• Inventors: Nobori, Kunio, Matsukawa, Yoshihiko, Sato, Satoshi, Nakagawa, Masamichi, Okamoto, Shusaku, Morimura, Atsushi
• Primary Examiner(s): Vo; Tung
• Assistant Examiner(s): SENFI, BEHROOZ M
• Application Number:
  US10/237,514
• Publication Number:
  US 20030076414A1
• Time in Patent Office: 1,618 Days
• Field of Search: 348/148, 348/135, 348/113, 348/116, 348/136, 348/140, 348/143, 348/169, 348/139, 348/141, 348/142, 382/106, 382/105, 382/104, 382/286, 382/199, 382/309, 382/276, 382/254
• US Class Current: 348/148
• CPC Class Codes: G01S 11/12 using electromagnetic waves other than radio waves
• CPC Class Codes: G01S 17/931 of land vehicles

Abstract

In the vehicle surroundings display device of the invention, an image processor generates an image showing the situation around a vehicle. When an obstacle is detected by an obstacle detection means, the image processor determines an image missing region using the position of the border of the obstacle on the vehicle side detected by an obstacle position detection means and fills the image missing region with substitute image data. This eliminates unnaturalness of a display image the user may feel and enables the user to accurately recognize the position of a neighboring obstacle that is important in driving operation.

Claims

1. A vehicle surroundings display device comprising:

a camera for taking an image of the surroundings of a vehicle;

obstacle detection means for detecting an obstacle in the surroundings of the vehicle;

an image processor for generating a surroundings image representing the situation around the vehicle from the image taken with the camera; and

image missing region detection means for detecting an image missing region in the surroundings image when the obstacle is detected by the obstacle detection means, the image missing region failing to obtain image data from the camera due to existence of the obstacle,

wherein the image missing region detection means includes object position detection means for detecting a border position of the obstacle, which is a position of a border on the vehicle side of an outline of the obstacle, and determines the image missing region using the detected border position,

wherein the image processor fills at least part of the image missing region with substitute image data when the image missing region is detected.

2. The device of claim 1, wherein the image processor uses a border image including at least the border portion of the obstacle on the vehicle side for generation of the substitute image data.

3. The device of claim 2, wherein the image processor places the border image on the surroundings image so as to align with the border position detected by the obstacle position detection means,

presumes a region occupied by the obstacle in the surroundings image by aligning with the border position, and

generates the substitute image data by pixel interpolation using pixel data of the border image for a portion of the presumed obstacle region other than the border image.

4. The device of claim 3, wherein when the obstacle is an axisymmetric object, the image processor assumes a symmetry axis of the obstacle in the surroundings image, inverts the border image with respect to the symmetry axis to provide an inverted border image, and generates an image between the border image and the inverted border image in the obstacle region by the pixel interpolation using pixel data of the border image.

5. The device of claim 1, further comprising an obstacle image database for sorting image data of objects as candidates of obstacles,

wherein the image processor determines the type of the detected obstacle, reads image data of the determined type of obstacle from the obstacle image database, places the read image data on the surroundings image to alight with the border position detected by the obstacle position detection means as the substitute image data.

6. The device of claim 5, wherein the image processor determines the type of the obstacle based on the image from the camera.

7. The device of claim 1, wherein when the obstacle is a vehicle, the obstacle position detection means detects a contact position of a tire of the vehicle with the road surface in the image form the camera as the border position.

8. The device of claim 1, wherein when the obstacle is a vehicle, the obstacle position detection means detects a line of a plane including a number plate of the vehicle in contact with the road surface in the image from the camera as the border position.

9. The device of claim 1, wherein the image processor generates the surroundings image by performing virtual viewpoint transform based on the image form the camera.

10. An image providing system comprising:

a camera for taking an image;

detection means for detecting an object in an image-provided range;

an image processor for generating a provided image showing the situation in the image-provided range form the image taken with the camera; and

image missing region detection means for detecting an image missing region in the provided image when an object is detected by the detection means, the image missing region failing to obtain image data from the camera due to existence of the object,

wherein the image missing region detection means includes object position detection means for detecting a border position of the object, which is a position of a border on the camera side of an outline of the object, and determines the image missing region using the detected border position,

wherein the image processor fills at least part of the image missing region with substitute image data when the image region is detected.

11. The system of claim 10, wherein the camera takes an image of the surroundings of a vehicle.

12. The system of claim 10, wherein the camera takes an image of the inside or surroundings of a building.