Apparatus and method for generating panoramic image based on image quality

• US 10,075,635 B2
• Filed: 01/13/2016
• Issued: 09/11/2018
• Est. Priority Date: 01/15/2015
• Status: Expired due to Fees

• Current Assignee: Electronics and Telecommunications Research Institute
• Original Assignee: Electronics and Telecommunications Research Institute
• Inventors: Seok, Joo Myoung, Ahn, Sang Woo, Cho, Yong Ju, Cha, Ji Hun
• Primary Examiner(s): Aghevli, Reza
• Application Number:
  US14/994,430
• Publication Number:
  US 20160212338A1
• Time in Patent Office: 972 Days
• Field of Search: None
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture

Abstract

Disclosed are a method and an apparatus for generating a panoramic image based on an image quality. The method of generating a panoramic image includes extracting a matching point to connect a base image captured by a first camera and a reference image captured by a second camera, geometrically transforming the reference image by determining a homography between the base image and the reference image, determining a change in image quality of the geometrically transformed reference image, and generating a panoramic image in which the geometrically transformed reference image is connected to the base image based on the determined change in image quality.

Claims

1. A method of generating a panoramic image, comprising:

extracting a matching point to connect a base image captured by a first camera and a reference image captured by a second camera;

geometrically transforming the reference image by determining a homography between the base image and the reference image based on the matching point;

determining a change in image quality of the geometrically transformed reference image by

inversely transforming the geometrically transformed reference image,

setting a target comparison area for comparison in the base image and the reference image based on a location of the matching point with respect to the base image and the reference image,

comparing an image quality of the set target comparison area in the base image or the reference image to an image quality of a target comparison area in the inversely transformed reference image, and

estimating an image quality degradation occurring due to the geometrical transforming of the reference image based on a result of the comparing; and

generating a panoramic image in which the geometrically transformed reference image is connected to the base image based on the determined change in image quality.

2. The method of claim 1, wherein the setting of the target comparison area comprises:

determining a first horizontal distance based on a direction in which the matching point is located from a center of the base image;

determining a second horizontal distance based on a direction in which the matching point is located from a center of the reference image;

determining a vertical distance based on a distance between the matching point and an upper end or a lower end of the base image; and

setting the target comparison area by applying the first horizontal distance, the second horizontal distance, and the vertical distance based on the matching point.

3. The method of claim 1, wherein the generating of the panoramic image comprises:

displaying matching points in response to the extracting of the matching points from the base image and the reference image;

displaying a change in image quality of the reference image geometrically transformed based on a matching point selected from the matching points by a user; and

generating a panoramic image in which the reference image geometrically transformed based on the matching point selected by the user is connected to the base image, in response to the selecting of the matching point displaying the change in image quality.

4. A method of measuring a change in image quality occurring through generation of a panoramic image using a base image and a reference image, the method comprising:

inversely transforming a geometrically transformed reference image based on a homography between a base image captured by a first camera and a reference image captured by a second camera;

setting a target comparison area for comparison in the inversely transformed reference image, and the base image or the reference image based on a location of a matching point to connect the base image and the reference image with respect to the base image and the reference image;

comparing an image quality of the set target comparison area in the base image or the reference image to an image quality of the set target comparison area in the inversely transformed reference image; and

estimating an image quality degradation occurring due to the geometrical transforming of the reference image based on a result of the comparing.

5. The method of claim 4, wherein the setting of the target comparison area comprises:

determining a first horizontal distance based on a direction in which the matching point is located in the base image;

determining a second horizontal distance based on a direction in which the matching point is located in the reference image;

determining a vertical distance based on a distance between the matching point and an upper end or a lower end of the base image; and

setting the target comparison area by applying the first horizontal distance, the second horizontal distance, and the vertical distance based on the matching point.

6. The method of claim 5, wherein, when a right area of the base image is connected to a left area of the reference image, the determining of the first horizontal distance comprises measuring a distance from the matching point to a rightmost end of the base image and determining the measured distance to be the first horizontal distance.

7. The method of claim 5, wherein, when a right area of the base image is connected to a left area of the reference image, the determining of the second horizontal distance comprises measuring a distance from the matching point to a leftmost end of the reference image and determining the measured distance to be the second horizontal distance.

8. The method of claim 5, wherein the determining of the vertical distance comprises:

setting, to be a first vertical distance, a shorter distance of a distance from the matching point to an upper end of the base image and a distance from the matching point to a lower end of the base image;

setting, to be a second vertical distance, a distance having a direction different from a direction of the first vertical distance of a distance from the matching point to an upper end of the reference image and a distance from the matching point to a lower end of the reference image; and

setting the first vertical distance and the second vertical distance in combination, to be the vertical distance.

9. The method of claim 4, wherein the inversely transformed reference image is an image resulting from the inverse transforming of the homography between the base image and the reference image.

10. An apparatus for generating a panoramic image, comprising:

a processor configured to

extract a matching point to connect a base image captured by a first camera and a reference image captured by a second camera,

geometrically transform the reference image by determining a homography between the base image and the reference image based on the matching point,

determine a change in image quality of the geometrically transformed reference image by

inversely transforming the geometrically transformed reference image,

setting a target comparison area for comparison in the base image and the reference image based on based on a location of the matching point with respect to the base image and the reference image,

comparing an image quality of the set target comparison area in the base image or the reference image to an image quality of a target comparison area in the inversely transformed reference image, and

estimating an image quality degradation occurring due to the geometrical transforming of the reference image based on a result of the comparing, and

generate a panoramic image in which the geometrically transformed reference image is connected to the base image based on the determined change in image quality.

11. The apparatus of claim 10, wherein the processor comprises:

a matching point extractor configured to extract the matching point to connect the base image captured by the first camera and the reference image captured by the second camera;

a reference image transformer configured to geometrically transform the reference image by determining the homography between the base image and the reference image based on the matching point;

an image quality change determiner configured to determine the change in image quality of the geometrically transformed reference image; and

a panoramic image generator configured to generate the panoramic image in which the geometrically transformed reference image is connected to the base image based on the determined change in image quality.

12. The apparatus of claim 11, wherein the image quality change determiner comprises:

an inverse transformer configured to inversely transform the geometrically transformed reference image;

a target comparison area setter configured to set the target comparison area for comparison in the base image and the reference image based on the location of the matching point with respect to the base image and the reference image; and

an image quality change measurer configured to compare the image quality of the set target comparison area in the base image or the reference image to the image quality of the target comparison area in the inversely transformed reference image, and estimate the image quality degradation occurring due to the geometrical transforming of the reference image based on the result of the comparing.

13. The apparatus of claim 12, wherein the target comparison area setter is configured to determine a first horizontal distance based on a direction in which the matching point is located from a center of the base image, determine a second horizontal distance based on a direction in which the matching point is located from a center of the reference image, determine a vertical distance based on a distance between the matching point and an upper end or a lower end of the base image, and set the target comparison area by applying the first horizontal distance, the second horizontal distance, and the vertical distance based on the matching point.

14. The apparatus of claim 11, wherein the panoramic image generator comprises:

an image display configured to display matching points in response to the extracting of the matching points from the base image and the reference image;

a measurement result display configured to display a change in image quality of the reference image geometrically transformed based on a matching point selected from the matching points by a user; and

an image generator configured to generate a panoramic image in which the reference image geometrically transformed based on the matching point selected by the user is connected to the base image, in response to the selecting of the matching point displaying the change in image quality.

15. The apparatus of claim 10, further comprising a memory configured to store instructions,

wherein the processor is further configured to execute the instructions to configure the processor to

extract the matching point to connect the base image captured by the first camera and the reference image captured by the second camera,

geometrically transform the reference image by determining the homography between the base image and the reference image based on the matching point,

determine the change in image quality of the geometrically transformed reference image, and

generate the panoramic image in which the geometrically transformed reference image is connected to the base image based on the determined change in image quality.