Drive supporting device

• US 7,218,758 B2
• Filed: 03/01/2002
• Issued: 05/15/2007
• Est. Priority Date: 03/28/2001
• Status: Active Grant

• Current Assignee: Matsushita Electric Industrial Company Limited (Panasonic Holdings Corporation)
• Original Assignee: Matsushita Electric Industrial Company Limited (Panasonic Holdings Corporation)
• Inventors: Mizusawa, Kazufumi, Ishii, Hirofumi, Okada, Tsuyoshi
• Primary Examiner(s): Ahmed; Samir
• Assistant Examiner(s): Tabatabai; Abolfazl
• Application Number:
  US10/472,919
• Publication Number:
  US 20040105579A1
• Time in Patent Office: 1,901 Days
• Field of Search: 382/104, 340/435, 340/932.2, 340/937, 340/988, 701/1, 701/23, 701/80, 701/82, 701/202, 701/209, 701/210, 701/211
• US Class Current: 382/104
• CPC Class Codes: B60R 1/27 providing all-round vision, e.g. using omnidirectional cameras
• CPC Class Codes: B60R 1/31 providing stereoscopic vision
• CPC Class Codes: B60R 2300/305 merging camera image with lines or icons
• CPC Class Codes: B60R 2300/607 from a bird's eye viewpoint
• CPC Class Codes: B60R 2300/806 for aiding parking
• CPC Class Codes: B60R 2300/8093 for obstacle warning
• CPC Class Codes: G01C 11/06 by comparison of two or more pictures of the same area
• CPC Class Codes: G01C 3/085 with electronic parallax measurement
• CPC Class Codes: G01S 11/12 using electromagnetic waves other than radio waves
• CPC Class Codes: G01S 5/16 using electromagnetic waves other than radio waves
• CPC Class Codes: G06T 15/00 3D [Three Dimensional] image rendering
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 5/80 Geometric correction
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G06V 20/586 of parking space
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

The object of the invention is to provide driving support apparatus which allows the driver to intuitively and more accurately recognize the situation around an obstacle thus reducing a burden of the driver. Driving support apparatus according to the invention has a plurality of image pickup means (1001, 3001) mounted on a mobile unit, conversion means (3008) for converting an image picked up by said image pickup means (3001) to an image seen from a virtual viewpoint above said image pickup means (1001, 3001) or an image obtained through orthogonal projection from above based on a road surface model, 3D map creation means (3007) for detecting three-dimensional information on that other than that on the road surface based on the parallax between the images picked up by the plurality of image pickup means, 3D image composition means (3008) for correcting the distortion of a figure in said viewpoint-converted image based on said detected three-dimensional information, and display means (3009) for displaying said distortion-corrected image.

Claims

1. Driving support apparatus comprising:

image pickup means mounted on a mobile unit;

conversion means for converting one or more images picked up by said image pickup means to an image seen from a virtual viewpoint above said image pickup means or an image orthogonal-projected from above based on a road surface model;

detection means for detecting three-dimensional information other than that on the road surface based on a parallax between images picked up by said image pickup means;

distortion correction means for correcting distortion of a figure in an image, for which said viewpoint conversion is performed, based on said detected three-dimensional information; and

display means for displaying an image for which said distortion-corrected is performed.

8. Driving support apparatus according to one of claims 1 through 7,

wherein a plurality of the image pickup means mounted on said mobile unit are arranged at predetermined intervals in a vertical direction of said mobile unit, in a vertical direction to a road surface model, or in a direction from one of the image pickup means to said virtual viewpoint or a line of sight orthogonal projected.

2. Driving support apparatus comprising:

image pickup means mounted on a mobile unit;

conversion means for converting one or more images picked up by said image pickup means to an image seen from a virtual viewpoint above a position of said image pickup means or an image orthogonal-projected from above based on a road surface model; and

obstacle area detection means for detecting an area where a parallax between images picked up by said image pickup means does not coincide with a parallax on the road surface model as an obstacle area.

3. Driving support apparatus according to claim 2, comprising:

overlay means for overlaying said detected area in an image, for which said viewpoint-converted is performed, based on said road surface model; and

display means for displaying said composite image.

4. Driving support apparatus according to claim 3,

wherein said overlay means detects a distance from said image pickup means and a height from a road surface based on a parallax of said detected area, changes a display of areas according to the distance and the height, and then overlays said detected area in an image, for which said viewpoint-converted is performed, based on said road surface model.

5. Driving support apparatus according to claim 3,

wherein said overlay means detects a distance from said image pickup means based on a parallax of said detected area, and overlays said detected area and character which shows said detected distance in an image, for which said viewpoint-converted is performed, based on said road surface model.

6. Driving support apparatus comprising:

image pickup means mounted on a mobile unit;

conversion means for converting one or more images picked up by said image pickup means to an image seen from a virtual viewpoint above a position of said image pickup means or an image orthogonal-projected from above based on a road surface model;

obstacle area detection means for detecting an unmatched area between said converted images as an obstacle area;

overlay means for overlaying said obstacle area in said converted image; and

display means for displaying said composite image.

7. Driving support apparatus comprising:

image pickup means mounted on a mobile unit;

intermediate image conversion means for converting one or more images picked up by said image pickup means to an intermediate image in which a distance and an angle from said image pickup means are coordinates based on a road surface model;

obstacle area detection means for detecting an unmatched area between said converted images, compares two images in said area to estimate an actual distance, corrects a distance and a position of said area in said converted image with the estimated distance, and outputs the corrected area as an obstacle area;

overlay means for overlaying said obstacle area in said converted image;

conversion means for converting said composite image to an ordinary coordinate image of a road surface; and

display means for displaying said converted image.

9. Driving support apparatus comprising:

image pickup means mounted on a mobile unit;

conversion means for respectively converting one or more images picked up by said image pickup means to an image seen from a virtual viewpoint above a position of said image pickup means or an image orthogonal-projected from above based on a road surface model; and

display means for displaying said converted image,

wherein said conversion means determines a strength based on a size of each pixel on the road surface model on the screen of said image pickup-means and an angle to the road surface, and varies a brightness and a color of pixels based on the strength.

10. Driving support apparatus comprising:

image pickup means mounted on a mobile unit;

intermediate image conversion means for converting one or more images picked up by said image pickup means to an intermediate image in which a distance or a height and an angle from said image pickup means are coordinates based on a road surface model and a cylinder model;

obstacle area detection means for detecting an unmatched area between said converted images, compares two images in said area to estimate an actual distance, corrects a distance and a position of said area in said converted image with the estimated distance, and outputs the corrected area as an obstacle area;

overlay means for overlaying said obstacle area in said converted image,

conversion means for converting said composite image to an ordinary coordinate image of a road surface; and

display means for displaying said converted image.

12. Driving support apparatus according to claim 7, 10 or 11, comprising:

storage means for storing search range data.

13. Driving support apparatus according to claim 7, 10 or 11, wherein at least three units of said image pickup means are arranged on the same axis.

14. Driving support apparatus according to claim 7, 10 or 11,

wherein three or more units of said image pickup means in total including two units of said image pickup means on a vertical axis and two units of image pickup means on a horizontal axis, around the single imaging pickup means, are arranged.

15. Driving support apparatus according to claim 7, 10 or 11,

wherein said image pickup means includes two or more image pickup means having different resolutions.

11. Driving support apparatus comprising:

image pickup means mounted on a mobile unit;

intermediate image conversion means for converting an image picked up by said image pickup means to an intermediate image in which a plane symmetrical with respect to a straight line linking between said image pickup means as a axis is a projection plane;

obstacle area detection means for detecting an unmatched area between said converted images, compares two images in said area to estimate an actual distance, corrects a distance and a position of said area in said converted image with the estimated distance, and outputs the corrected area as an obstacle area;

overlay means for overlaying said obstacle area in said converted image,

conversion means for converting said composite image to an ordinary coordinate image of a road surface; and

display means for displaying said converted image.

16. Driving support apparatus comprising:

image pickup means mounted on a mobile unit;

conversion means for converting an image picked up by said image pickup means to a viewpoint-converted image seen from a virtual viewpoint which is different from a position of said image pickup means; and

display means for displaying an image converted by said conversion means,

wherein said image pickup means picks up a plurality of images having a predetermined parallax and said display means displays an image which is corrected based on said parallax.

17. Distance information extraction apparatus comprising:

image pickup means for picking up a plurality of images from a plurality of observation points;

conversion means for converting said plurality of images to intermediate images projected onto a plane having orthogonal coordinates;

means for defining one of said orthogonal coordinates as a direction of rotation angle about an axis passing through said plurality of observation points and aligning a search direction to obtain a parallax between said intermediate images in a direction of another coordinate of said orthogonal coordinates; and

measurement means for measuring a distance to a target in said image based on said parallax.