Image synthesizing device and method

• US 20030063774A1
• Filed: 08/30/2002
• Published: 04/03/2003
• Est. Priority Date: 10/01/2001
• Status: Active Grant

• Current Assignee: Nissan Motor Co., Ltd.
• Original Assignee: Nissan Motor Co., Ltd.
• Inventors: Oizumi, Ken
• Granted Patent: US 7,123,748 B2
• Time in Patent Office: Days
• US Class Current: 382/104
• CPC Class Codes: G06T 2200/32 involving image mosaicing
• CPC Class Codes: G06T 7/30 Determination of transform parameters for the alignment of images, i.e. image registration

Abstract

An image synthesizing device is operatively coupled to a plurality of cameras to accurately express the color and brightness of a synthesized image photographed. The image synthesizing device is configured to synthesize the images of the cameras by selecting with respect to an observation point on a surface being photographed the camera positioned closest to the observation point, designating the color and brightness values of the observation point photographed by that camera as the color and brightness values of the observation point, and conducting the same processing for the entire surface being photographed. Thus, among the cameras, the image synthesizing device selects the camera that is closest to the observation point and uses the color and brightness values of that camera. This arrangement results in selecting a camera that photographs a small area of the photographed surface per unit pixel. Consequently, a synthesized image having good color and brightness precision is obtained.

Claims

1. An image synthesizing device comprising:

a plurality of cameras configured to obtain images; and

an image processing device configured to receive the images from the cameras,

the image processing device being further configured to synthesizes the images of the cameras by

determining a plurality of observation points for a reference surface being photographed by the cameras,

selecting one of the cameras for each of the observation points as a basis camera such that the basis camera for each of the observation points is the closest of the cameras to the observation points on the reference surface being photographed by the cameras, and

designating color and brightness values photographed by the basis camera selected for each of the observation points as color and brightness values for that one of the observation points.

2. The image synthesizing device as recited in claim 1, wherein

the image processing device is further configured to

synthesize the images of the cameras as a synthesized image of a virtual camera with a predetermined number of pixels through computer processing,

establish correspondence between each of the observation points and each of the pixels of the virtual camera based on relative position and camera characteristic of the virtual camera, and

designate the color and brightness values of the observation points as color and brightness values of the pixels of the virtual camera.

3. An image synthesizing device comprising:

a plurality of cameras configured to obtain images; and

an image processing device configured to receive the images from the cameras,

the image processing device being further configured to synthesizes the images of the cameras by

determining a plurality of observation points for a reference surface being photographed by the cameras,

synthesizing color and brightness values with respect to the observation points on the reference surface being photographed by each of the cameras by using strengths Sn calculated as a function f(Ln) of distances Ln between the observation points and the cameras to calculate synthesized color and brightness values for the observation points,

designating the synthesized color and brightness values as the color and brightness values of the observation points.

4. The image synthesizing device as recited in claim 3, wherein

the image processing device is further configured to

synthesize the images of the cameras as a synthesized image of a virtual camera with a predetermined number of pixels through computer processing,

establish correspondence between each of the observation points and each of the pixels of the virtual camera based on relative position and camera characteristic of the virtual camera, and

designate the synthesized color and brightness values of the observation points as color and brightness values of the pixels of the virtual camera.

5. An image synthesizing device comprising:

a plurality of cameras configured to obtain images; and

an image processing device configured to receive the images from the cameras,

the image processing device being further configured to synthesizes the images of the cameras by

determining a plurality of observation points for a reference surface being photographed by the cameras,

selecting one of the cameras for each of the observation points as a basis camera such that the basis camera for each of the observation points has an angle formed between the reference surface being photographed and a straight line joining one of the observation points and that closest to being perpendicular, and

designating color and brightness values photographed by the basis camera selected for each of the observation points as color and brightness values for that one of the observation points.

6. The image synthesizing device as recited in claim 5, wherein

the image processing device is further configured to

synthesize the images of the cameras as a synthesized image of a virtual camera with a predetermined number of pixels through computer processing,

establish correspondence between each of the observation points and each of the pixels of the virtual camera based on relative position and camera characteristic of the virtual camera, and

designate the color and brightness values of the observation points as color and brightness values of the pixels of the virtual camera.

7. An image synthesizing device comprising:

a plurality of cameras configured to obtain images; and

an image processing device configured to receive the images from the cameras,

the image processing device being further configured to synthesizes the images of the cameras by

determining a plurality of observation points for a reference surface being photographed by the cameras,

synthesizing color and brightness values with respect to the observation points on the reference surface being photographed by each of the cameras by using strengths Sn calculated as a function f(α

n) of angles α

n between the reference surface being photographed and straight lines joining the observation points and the cameras to calculate synthesized color and brightness values for the observation points, and

designating the synthesized color and brightness values as the color and brightness values of the observation points.

8. The image synthesizing device as recited in claim 7, wherein

the image processing device is further configured to

synthesize the images of the cameras as a synthesized image of a virtual camera with a predetermined number of pixels through computer processing,

establish correspondence between each of the observation points and each of the pixels of the virtual camera based on relative position and camera characteristic of the virtual camera, and

designate the synthesized color and brightness values of the observation points as color and brightness values of the pixels of the virtual camera.

9. An image synthesizing device comprising:

a plurality of cameras configured to obtain images; and

an image processing device configured to receive the images from the cameras,

the image processing device being further configured to synthesizes the images of the cameras by

determining a plurality of observation points for a reference surface being photographed by the cameras,

synthesizing color and brightness values with respect to the observation points on the reference surface being photographed by each of the cameras by using strengths Sn calculated as a function f(Ln, α

n) of distances Ln between the observation points and the cameras and angles an between the reference surface being photographed and straight lines joining the observation points and the cameras, and

designating the synthesized color and brightness values as the color and brightness values of the observation points.

10. The image synthesizing device as recited in claim 9, wherein

the image processing device is further configured to

create a synthesized image obtained by the synthesizing of the images of the cameras as the image of a virtual camera by establishing through computer processing a virtual camera for creating the synthesized image,

establish the correspondence between each of the observation points and each pixel based on the position and camera characteristic of the virtual camera, and

designate synthesized color and brightness values found for each pixel as the color and brightness values of a pixel corresponding to the virtual camera.

11. An image synthesizing method comprising:

obtaining images photographed by a plurality of cameras;

determining a plurality of observation points for a reference surface being photographed by the cameras,

selecting one of the cameras for each of the observation points as a basis camera such that the basis camera for each of the observation points is the closest of the cameras to the observation points on the reference surface being photographed by the cameras, and

designating color and brightness values photographed by the basis camera selected for each of the observation points as color and brightness values for that one of the observation points.

12. The image synthesizing method as recited in claim 11, further comprising

synthesizing the images of the cameras as a synthesized image of a virtual camera with a predetermined number of pixels through computer processing,

establishing correspondence between each of the observation points and each of the pixels of the virtual camera based on relative position and camera characteristic of the virtual camera;

and

designating the color and brightness values of the observation points as color and brightness values of the pixels of the virtual camera..

13. An image synthesizing method comprising:

obtaining images photographed by a plurality of cameras;

determining a plurality of observation points for a reference surface being photographed by the cameras;

synthesizing color and brightness values with respect to the observation points on the reference surface being photographed by each of the cameras by using strengths Sn calculated as a function f(Ln) of distances Ln between the observation points and the cameras to calculate synthesized color and brightness values for the observation points; and

designating the synthesized color and brightness values as the color and brightness values of the observation points.

14. The image synthesizing method as recited in claim 13, further comprising

synthesizing the images of the cameras as a synthesized image of a virtual camera with a predetermined number of pixels through computer processing,

establishing correspondence between each of the observation points and each of the pixels of the virtual camera based on relative position and camera characteristic of the virtual camera;

and

designating the synthesized color and brightness values of the observation points as color and brightness values of the pixels of the virtual camera..

15. An image synthesizing method comprising:

obtaining images photographed by a plurality of cameras;

determining a plurality of observation points for a reference surface being photographed by the cameras;

selecting one of the cameras for each of the observation points as a basis camera such that the basis camera for each of the observation points has an angle formed between the reference surface being photographed and a straight line joining one of the observation points and that closest to being perpendicular; and

designating color and brightness values photographed by the basis camera selected for each of the observation points as color and brightness values for that one of the observation points.

16. The image synthesizing method as recited in claim 15, further comprising

synthesizing the images of the cameras as a synthesized image of a virtual camera with a predetermined number of pixels through computer processing,

establishing correspondence between each of the observation points and each of the pixels of the virtual camera based on relative position and camera characteristic of the virtual camera;

and

designating the color and brightness values of the observation points as color and brightness values of the pixels of the virtual camera..

17. An image synthesizing method comprising:

obtaining images photographed by a plurality of cameras;

determining a plurality of observation points for a reference surface being photographed by the cameras;

synthesizing color and brightness values with respect to the observation points on the reference surface being photographed by each of the cameras by using strengths Sn calculated as a function f(α

n) of angles an between the reference surface being photographed and straight lines joining the observation points and the cameras to calculate synthesized color and brightness values for the observation points; and

designating the synthesized color and brightness values as the color and brightness values of the observation points.

18. The image synthesizing method as recited in claim 17, further comprising

synthesizing the images of the cameras as a synthesized image of a virtual camera with a predetermined number of pixels through computer processing,

establishing correspondence between each of the observation points and each of the pixels of the virtual camera based on relative position and camera characteristic of the virtual camera;

and

designating the synthesized color and brightness values of the observation points as color and brightness values of the pixels of the virtual camera..

19. An image synthesizing method comprising:

obtaining images photographed by a plurality of cameras;

determining a plurality of observation points for a reference surface being photographed by the cameras;

synthesizing color and brightness values with respect to the observation points on the reference surface being photographed by each of the cameras by using strengths Sn calculated as a function f(Ln, α

n) of distances Ln between the observation points and the cameras and angles α

n between the reference surface being photographed and straight lines joining the observation points and the cameras; and

designating the synthesized color and brightness values as the color and brightness values of the observation points.

20. The image synthesizing method as recited in claim 19, further comprising

synthesizing the images of the cameras as a synthesized image of a virtual camera with a predetermined number of pixels through computer processing,

establishing correspondence between each of the observation points and each of the pixels of the virtual camera based on relative position and camera characteristic of the virtual camera;

and

designating the synthesized color and brightness values of the observation points as color and brightness values of the pixels of the virtual camera..

21. An image synthesizing device comprising:

at least first and second means for capturing and generating at least first and second images, respectively; and

means for processing the at least first and second images from the at least first and second means,

the means for processing being further configured to synthesizes the at least first and second images by

determining a plurality of observation points for a reference surface being captured by the at least first and second means,

determining color and brightness values for at least one of the at least first and second means for each of the observation points, and

using the color and brightness values of at least one of the at least first and second means to designate color and brightness values for the observation points, the color and brightness values for the observation points being based on positions of the at least first and second means to relative the observation points.