Method for arranging cameras in a vehicle surroundings monitoring system

• US 7,502,048 B2
• Filed: 10/15/2002
• Issued: 03/10/2009
• Est. Priority Date: 10/15/2001
• Status: Active Grant

• Current Assignee: Panasonic Corporation (Panasonic Holdings Corporation)
• Original Assignee: Panasonic Corporation (Panasonic Holdings Corporation)
• Inventors: Morimura, Atsushi, Mizusawa, Kazufumi, Yoshida, Takashi, Nakagawa, Masamichi, Iisaka, Atsushi, Okamoto, Shusaku
• Primary Examiner(s): Dastouri; Mehrdad
• Assistant Examiner(s): HALLENBECK-HUBER, JEREMIAH CHARLES
• Application Number:
  US10/271,411
• Publication Number:
  US 20030085999A1
• Time in Patent Office: 2,338 Days
• Field of Search: 348/148
• US Class Current: 348/148
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: B60R 1/27 providing all-round vision, e.g. using omnidirectional cameras
• CPC Class Codes: B60R 2300/102 using 360 degree surveillance camera system
• CPC Class Codes: B60R 2300/105 using multiple cameras
• CPC Class Codes: B60R 2300/303 using joined images, e.g. multiple camera images
• CPC Class Codes: B60R 2300/305 merging camera image with lines or icons
• CPC Class Codes: B60R 2300/607 from a bird's eye viewpoint
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

The invention provides a vehicle surroundings monitoring system for synthesizing and displaying images captured by a plurality of cameras to provide images that, near the border between partial images, are more accurate and more readily understood by the user than has been the case conventionally. In the synthesized image, a partial image captured by a first camera and a partial image captured by a second camera are adjacent to one another at their border. Near this border, the capturing directions of the first and the second cameras both substantially match the direction of the border. Consequently, images of objects near the border extend along the border, and thus do not disappear on the synthesized image.

Claims

1. A vehicle surroundings monitoring system comprising:

a plurality of cameras, including at least a first camera and a second camera, for capturing images of surroundings of a vehicle, the first and second cameras being arranged apart from each other; and

an image processing unit operable to receive camera images captured by the plurality of cameras, and operable to generate a virtual viewpoint synthesized image in which the surroundings of the vehicle are looked down upon from above and each of the plurality of cameras is represented as a separate and distinct point, based on a first partial image according to the first camera and a second partial image according to the second camera, the first partial image being a portion of the image captured by the first camera, the second partial image being a portion of the image captured by the second camera, the first and second partial images being represented in the virtual viewpoint synthesized image as areas that share a common boundary line substantially parallel to and coincident with a line drawn between the separate and distinct points of the first and second cameras so that an image of objects near the common boundary line between the first and second partial images does not disappear on the synthesized image.

2. The vehicle surroundings monitoring system according to claim 1, wherein in the synthesized image, a blend image in which the first and the second partial images are mixed is arranged at the border between the first partial image and the second partial image.

3. The vehicle surroundings monitoring system according to claim 1, wherein the first camera is arranged at a front left or right end portion of the vehicle facing right or left obliquely forward, and

the second camera is arranged at a left or right side portion of the vehicle facing left or right obliquely forward.

4. The vehicle surroundings monitoring system according to claim 3, wherein the plurality of cameras includes:

a third camera arranged at a left or right side portion of the vehicle facing left or right obliquely rearward; and

a fourth camera arranged at a rear left or right end portion of the vehicle facing right or left obliquely rearward.

5. The vehicle surroundings monitoring system according to claim 1, wherein the first camera is arranged at a front right or left end portion of the vehicle facing left or right obliquely forward, and

the second camera is arranged at a front left or right end portion of the vehicle facing left or right obliquely rearward.

6. The vehicle surroundings monitoring system according to claim 5, wherein the plurality of cameras includes:

a third camera arranged at a rear left or right end portion of the vehicle facing right or left obliquely rearward; and

a fourth camera arranged at a rear right or left end portion of the vehicle facing right or left obliquely forward.

7. The vehicle surroundings monitoring system of claim 1, wherein the first and second cameras are installed on a straight line exactly matching the border portion.

8. The vehicle surroundings monitoring system according to claim 1, wherein the image processing unit performs geometric transformation of the first and second partial images into the virtual viewpoint synthesized image using at least one of a group of models consisting of road surface plane model, cylindrical model and quasi-cylindrical model.

9. The vehicle surroundings monitor system according to claim 1 wherein one of the following alternatives applies:

a) the first camera is arranged at a front left or right end portion of the vehicle facing right or left obliquely forward, and

the second camera is arranged at a left or right side portion of the vehicle facing left or right obliquely forward;

b) the first camera is arranged at a front right or left end portion of the vehicle facing left or right obliquely forward, and

the second camera is arranged at a front left or right end portion of the vehicle facing left or right obliquely rearward;

c) the first camera is arranged at a rear left or right end portion of the vehicle facing right or left obliquely rearward, and

the second camera is arranged at a left or right side portion of the vehicle facing left or right obliquely rearward;

d) the first camera is arranged at a rear left or right end portion of the vehicle facing left or right obliquely forward, and

the second camera is arranged at a rear side portion of the vehicle facing left or right obliquely rearward;

e) the first camera is arranged at a front left or right end portion of the vehicle facing right or left obliquely forward, and

the second camera is arranged at a rear left or right end portion of the vehicle facing left or right obliquely forward;

f) the first camera is arranged at a rear left or right end portion of the vehicle facing left or right obliquely forward, and

the second camera is arranged at a rear right or left end portion of the vehicle facing left or right obliquely rearward;

g) the first camera is arranged at a rear left or right end portion of the vehicle facing right or left obliquely rearward, and

the second camera is arranged at a front left or right end portion of the vehicle facing left or right obliquely rearward.

10. A method for adjusting a vehicle surroundings monitoring system, wherein the vehicle surroundings monitoring system comprises a plurality of cameras for capturing images of surroundings of the vehicle, and an image processing portion that synthesizes partial images obtained from images captured by the plurality of cameras, and displays the synthesized image on a display device as a virtual viewpoint synthesized image in which the surroundings of the vehicle are looked down upon from above and each of the plurality of cameras is represented as a separate and distinct point in the virtual viewpoint synthesized image;

the method comprising;

a step of arranging first and second cameras, of the plurality of cameras, for capturing partial images arranged adjacent to one another in the virtual viewpoint synthesized image, so that the first and second cameras are apart from each other and the captured regions of the first and the second cameras overlap one another, the first partial image being a portion of the image captured by the first camera, the second partial image being a portion of the image captured by the second camera; and

a step of adjusting at least one of the first camera, the second camera, and the image processing portion, so that, a border portion between opposing sides of a first partial image according to the first camera and a second partial image according to the second camera

forms a straight line which substantially matches a straight line connecting the separate and distinct point of the first camera and the second camera in the virtual viewpoint synthesized image so that an image of objects near the common boundary line between the first and second partial images does not disappear on the synthesized image.

11. A vehicle surroundings monitoring system comprising:

a plurality of cameras, including at least a first camera and a second camera, for capturing images of surroundings of a vehicle, the first and second cameras being arranged apart from each other; and

an image processing unit operable to receive camera images captured by the plurality of cameras, and operable to generate a virtual viewpoint synthesized image in which the surroundings of the vehicle are looked down upon from above based on camera images, for displaying on a display device, wherein and each of the plurality of cameras is represented as a separate and distinct point within said virtual viewpoint synthesized image;

wherein the virtual viewpoint synthesized image includes a first partial image according to the first camera and a second partial image according to the second camera, the first partial image being a portion of the image captured by the first camera, the second partial image being a portion of the image captured by the second camera, said partial images being cut and stitched together to define a seam line, and

wherein said seam line substantially matches a straight line connecting separate and distinct points of the first camera and the second camera in the virtual viewpoint synthesized image so that an image of objects near the common boundary line between the first and second partial images does not disappear on the synthesized image.