Image processing method and apparatus for generating panoramic image

• US 20040228544A1
• Filed: 11/26/2003
• Published: 11/18/2004
• Est. Priority Date: 11/29/2002
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Suzuki, Masahiro, Katayama, Akihiro, Sakagawa, Yukio, Kotake, Daisuke, Endo, Takaaki
• Granted Patent: US 7,388,982 B2
• Time in Patent Office: Days
• US Class Current: 382/284
• CPC Class Codes: G06T 15/205 Image-based rendering
• CPC Class Codes: G06T 2207/20212 Image combination
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: G06T 5/007 Dynamic range modification applied in cameras using an electronic image sensor H04N23/741, H04N23/743
• CPC Class Codes: G06T 5/90 Dynamic range modification of images or parts thereof
• CPC Class Codes: G06T 7/30 Determination of transform parameters for the alignment of images, i.e. image registration
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture

Abstract

There are disclosed an image processing method and apparatus for generating a panoramic image in which the hue or brightness discontinuities generated at the seams of source images are reduced. According to the image processing method of this invention, upon compositing source images (221, 222) having an overlap portion, the hue or brightness of each source image is converted (H211, H221) using a reference image which has overlap portions with both the source images to be composited, and the converted images (241, 242) are composited, thereby reducing the hue or brightness discontinuities generated at the seam of images.

Claims

1. An image processing method for generating a composite image by compositing a plurality of source images, each of which has an overlapping portion with at least one another source image, comprising:

a reference image acquisition step of acquiring a reference image which has overlapping portions with both of one of the source images, and another source image to be composited with that source image;

a conversion generation step of generating conversions based on a hue or brightness level of the reference image for the one of the source images, and the other source image to be composited with that source image;

a conversion step of converting the one of the source images, and the other source image to be composited with that source image on the basis of the generated conversions; and

a composition step of generating the composite image by compositing the one of the source images, and the other source image to be composited with that source image, which have been converted.

2. An image processing method for generating a panoramic image by compositing first and second images, comprising:

acquiring the first and second images, and a reference image which has overlapping portions with both of the first and second images;

calculating a first conversion condition required to adjust a hue or brightness level of the first image to a hue or brightness level of the reference image;

calculating a second conversion condition required to adjust a hue or brightness level of the second image to the hue or brightness level of the reference image; and

generating the panoramic image by compositing an image obtained by converting the first image on the basis of the first conversion condition, and an image obtained by converting the second image on the basis of the second conversion condition.

3. The method according to claim 2, wherein the reference image, and the first and second images are images sensed at the same time.

4. The method according to claim 2, wherein the panoramic image is generated by further compositing a third image.

5. A computer readable storage medium storing a computer program that makes a computer execute an image processing method for generating a composite image by compositing a plurality of source images, each of which has an overlapping portion with at least one another source image, said computer program comprising:

a program of a reference image acquisition step of acquiring a reference image which has overlapping portions with both of one of the source images, and another source image to be composited with that source image;

a program of a conversion generation step of generating conversions based on a hue or brightness level of the reference image for the one of the source images, and the other source image to be composited with that source image;

a program of a conversion step of converting the one of the source images, and the other source image to be composited with that source image on the basis of the generated conversions; and

a program of a composition step of generating the composite image by compositing the one of the source images, and the other source image to be composited with that source image, which have been converted.

6. An image processing apparatus for generating a composite image by compositing a plurality of source images, each of which has an overlapping portion with at least one another source image, comprising:

reference image acquisition unit adapted to acquire a reference image which has overlapping portions with both of one of the source images, and another source image to be composited with that source image;

conversion generation unit adapted to generate conversions based on a hue or brightness level of the reference image for the one of the source images, and the other source image to be composited with that source image;

conversion unit adapted to convert the one of the source images, and the other source image to be composited with that source image on the basis of the generated conversions; and

composition unit adapted to generate the composite image by compositing the one of the source images, and the other source image to be composited with that source image, which have been converted.

7. A computer readable storage medium storing a computer program that makes a computer execute an image processing method for generating a panoramic image by compositing first and second images, said computer program comprising:

a program of a step of acquiring the first and second images, and a reference image which has overlapping portions with both of the first and second images;

a program of a step of calculating a first conversion condition required to adjust a hue or brightness level of the first image to a hue or brightness level of the reference image;

a program of a step of calculating a second conversion condition required to adjust a hue or brightness level of the second image to the hue or brightness level of the reference image; and

a program of a step of generating the panoramic image by compositing an image obtained by converting the first image on the basis of the first conversion condition, and an image obtained by converting the second image on the basis of the second conversion condition.

8. An image processing apparatus for generating a panoramic image by compositing first and second images, comprising:

acquiring unit adapted to acquire the first and second images, and a reference image which has overlapping portions with both of the first and second images;

first calculating unit adapted to calculate a first conversion condition required to adjust a hue or brightness level of the first image to a hue or brightness level of the reference image;

second calculating unit adapted to calculate a second conversion condition required to adjust a hue or brightness level of the second image to the hue or brightness level of the reference image; and

generating unit adapted to generate the panoramic image by compositing an image obtained by converting the first image on the basis of the first conversion condition, and an image obtained by converting the second image on the basis of the second conversion condition.

9. An image processing method for generating a full-view panoramic image by compositing a plurality of images each of which has overlapping portions with neighboring images, comprising:

acquiring, when an arbitrary one of the plurality of images is defined as a first image, and images to be composited as right and left neighboring images of the first image are defined as second and third images, a first reference image having overlapping portions with both the first and second images, and a second reference image having overlapping portions with both the first and third images;

calculating first and second conversion conditions required to adjust a hue or brightness level of the first image to hue or brightness levels of the first and second reference images;

generating a converted image by converting a left predetermined region of the first image on the basis of the first conversion condition and converting a right predetermined region of the first image on the basis of the second conversion condition; and

generating a full-view panoramic image by compositing the converted images generated for all of the plurality of images.

10. The method according to claim 9, wherein the conversions based on the first and second conversion conditions are weighted depending on distances from right and left ends of the first image.

11. The method according to claim 10, wherein the first and second reference images are generated from one image.

12. A computer readable storage medium storing a computer program that makes a computer execute an image processing method for generating a full-view panoramic image by compositing a plurality of images each of which has overlapping portions with neighboring images, said computer program comprising:

a program of a step of acquiring, when an arbitrary one of the plurality of images is defined as a first image, and images to be composited as right and left neighboring images of the first image are defined as second and third images, a first reference image having overlapping portions with both the first and second images, and a second reference image having overlapping portions with both the first and third images;

a program of a step of calculating first and second conversion conditions required to adjust a hue or brightness level of the first image to hue or brightness levels of the first and second reference images;

a program of a step of generating a converted image by converting a left predetermined region of the first image on the basis of the first conversion condition and converting a right predetermined region of the first image on the basis of the second conversion condition; and

a program of a step of generating a full-view panoramic image by compositing the converted images generated for all of the plurality of images.

13. An image processing method for generating a full-view panoramic image by compositing a plurality of images each of which has overlapping portions with neighboring images, comprising:

acquiring unit adapted to, when an arbitrary one of the plurality of images is defined as a first image, and images to be composited as right and left neighboring images of the first image are defined as second and third images, acquire a first reference image having overlapping portions with both the first and second images, and a second reference image having overlapping portions with both the first and third images;

calculating unit adapted to calculate first and second conversion conditions required to adjust a hue or brightness level of the first image to hue or brightness levels of the first and second reference images;

first generating unit adapted to generate a converted image by converting a left predetermined region of the first image on the basis of the first conversion condition and converting a right predetermined region of the first image on the basis of the second conversion condition; and

second generating unit adapted to generate a full-view panoramic image by compositing the converted images generated for all of the plurality of images.