Vehicle circumferential image providing device and vehicle circumferential image providing method

• US 8,243,994 B2
• Filed: 04/25/2007
• Issued: 08/14/2012
• Est. Priority Date: 05/09/2006
• Status: Active Grant

• Current Assignee: ITEL Telecomunicazioni S.r.l., Nissan Motor Co., Ltd.
• Original Assignee: Nissan Motor Co., Ltd.
• Inventors: Suzuki, Masayasu, Akatsuka, Takeshi
• Primary Examiner(s): Johns, Andrew W
• Application Number:
  US12/298,837
• Publication Number:
  US 20090257659A1
• Time in Patent Office: 1,938 Days
• Field of Search: 382/104, 382/199, 382/284, 382/293, 382/294, 382/295, 348/148, 340/937, 340/435, 340/436, 340/437
• US Class Current: 382/104
• CPC Class Codes: B60R 1/27 providing all-round vision, e.g. using omnidirectional cameras
• CPC Class Codes: B60R 11/04 Mounting of cameras operative during drive; Arrangement of controls thereof relative to the vehicle
• CPC Class Codes: B60R 2300/102 using 360 degree surveillance camera system
• CPC Class Codes: B60R 2300/303 using joined images, e.g. multiple camera images
• CPC Class Codes: B60R 2300/607 from a bird's eye viewpoint
• CPC Class Codes: B60R 2300/8093 for obstacle warning
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: G06T 7/33 using feature-based methods
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums

Abstract

In a vehicle circumferential image providing device 1, an image converting unit 53 generates an overhead image 205 from images picked up with plural camera modules 10. An image detecting unit 54 performs edge detection in an overhead image 205. Further, the image detecting unit 54 determines continuity of edge lines connecting between connected parts in the overhead image 205, among the detected edge lines. When the edge lines striding over the connected parts in the overhead image 205 are discontinuous, the image detecting unit 54 sets edge lines at the far side of the own vehicle to match edge lines at the near side of the own vehicle, thereby setting the edge lines continuous at the connected parts.

Claims

1. A vehicle circumferential image providing device configured to provide an image at a circumference of a vehicle to a driver, the vehicle circumferential image providing device comprising:

a plurality of imaging units configured to image the circumference of the vehicle, wherein the plurality of imaging units are configured to image in mutually different directions;

a first image processing unit configured to coordinate convert images at the circumference of the vehicle imaged by the plurality of imaging units using a ground surface as a reference surface, and generate an overhead image by combining the coordinate-converted images;

an edge detecting unit configured to detect edges in the overhead image generated by the first image processing unit;

a determining unit configured to determine continuity of edge lines extending over connected parts of the coordinate-converted images, among edge lines detected by the edge detecting unit; and

a second image processing unit configured to perform imaging processing by setting edge lines at a far side of the vehicle to match edge lines at a near side of the vehicle, thereby setting the edge lines to be continuous at the connected parts, when the edge lines extending over the connected parts of the coordinate-converted images are discontinuous.

2. The vehicle circumferential image providing device according to claim 1, wherein the second image processing unit is configured such that, when a plurality of edge lines extending over the coordinate-converted images are present, and edge lines at the far side of the vehicle are discontinuous among the plurality of edge lines, the second image processing unit sets the edge lines at the far side of the vehicle to be continuous at the connected parts while maintaining the continuity of the edge lines at the near side of the vehicle.

3. The vehicle circumferential image providing device according to claim 2, further comprising a unit configured to determine whether a plurality of edge lines extending over the connected parts of the coordinate-converted images constitute the same object, wherein the second image processing unit is configured such that, when it is determined that the plurality of edge lines extending over the connected parts of the coordinate-converted images constitute the same object, the second image processing unit specifies edge lines present at the near side of the vehicle and edge lines present at the far side of the vehicle.

4. The vehicle circumferential image providing device according to claim 1, wherein the second image processing unit is configured such that, when the edge lines extending over the coordinate-converted images are discontinuous, and the edge lines have inflected points, the second image processing unit sets the edge lines to be continuous at the connected parts so that the edge lines do not exceed a tangent line connecting points at which an object represented by the edge lines contacts the ground surface.

5. The vehicle circumferential image providing device according to claim 1, wherein the second image processing unit is configured such that, when a plurality of edge lines extending over the coordinate-converted images are present, and the edge lines at the near side of the vehicle are continuous among the plurality of edge lines, the second image processing unit sets the edge lines at the far side of the vehicle to be continuous at the connected parts while maintaining the continuity of the edge lines at the near side of the vehicle.

6. The vehicle circumferential image providing device according to claim 1, wherein the second image processing unit is configured to perform at least one of a shift process of shifting an edge, a thinning process of thinning a pixel, and a compression process of compressing an image, to set edges to be continuous at the connected parts.

7. The vehicle circumferential image providing device according to claim 6, wherein the second image processing unit is configured to perform a process of filling a predetermined color into a part losing pixel information in an overhead image, by performing at least one of the shift process, the thinning process, and the compression process.

8. The vehicle circumferential image providing device according to claim 1, wherein the edge detecting unit is configured such that, when a speed of the vehicle is equal to or higher than a predetermined speed, the edge detecting unit performs edge detection in a region farther from the vehicle in the overhead image than from the vehicle when a speed of the vehicle is less than the predetermined speed.

9. The vehicle circumferential image providing device according to claim 1, wherein the edge detecting unit performs edge detection in a region of a proceeding direction of the vehicle in the overhead image.

10. A vehicle circumferential image providing method for providing an image at a circumference of a vehicle to a driver, the vehicle circumferential image providing method comprising:

a step of imaging mutually different directions of the circumference of the vehicle;

a step of coordinate-converting the images at the circumference of the vehicle imaged using a ground surface as a reference surface, and generating an overhead image by combining the coordinate-converted images;

a step of detecting edges in the generated overhead image;

a step of determining continuity of edge lines extending over connected parts of the coordinate-converted images, among detected edge lines; and

a step of performing imaging processing by setting edge lines at a far side of the vehicle to match edge lines at a near side of the vehicle, thereby setting the edge lines to be continuous at the connected parts, when the edge lines extending over the connected parts of the coordinate-converted images are discontinuous.