Image mosaicing apparatus for mitigating curling effect

• US 20090141043A1
• Filed: 12/18/2007
• Published: 06/04/2009
• Est. Priority Date: 11/30/2007
• Status: Abandoned Application

• Current Assignee: Hitachi, Ltd.
• Original Assignee: Hitachi, Ltd.
• Inventors: Guo, Tao, Koga, Masashi
• Application Number:
  US12/000,853
• Publication Number:
  US 20090141043A1
• Time in Patent Office: Days
• US Class Current: 345/629
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: G09G 3/001 using specific devices not provided for in groups G09G3/02 - G09G3/36, e.g. using an intermediate record carrier such as a film slide; Projection systems; Display of non-alphanumerical information, solely or in combination with alphanumerical information, e.g. digital display on projected diapositive as background slide projectors per se G03B23/00 = 42 HP

Abstract

The mosaicing curling effect which causes a mosaic strip to curl is effectively mitigated in a system for generating a mosaic image from sequential images which partially overlap with one another. The system has a matrix estimating unit, an adjustment element calculating unit, and an image projecting unit. The matrix estimating unit estimates a transformation matrix for transformation between two adjacent images taken out from among the sequential images. The adjustment element calculating unit calculates an adjustment element of the estimated transformation matrix by decomposing the transformation matrix into a rotational component and a perspective component. The image projecting unit projects each image on the mosaic plane using the transformation matrix to which the calculated adjustment element has been applied.

Claims

1. An image mosaicing system for generating a mosaic image by compositing a plurality of sequential images which partially overlap with one another, comprising:

a storage unit which stores the plurality of sequential images;

an input unit which obtains control information on the plurality of sequential images stored in the storage unit;

a display unit which displays computation results as a mosaic image;

a processing unit which performs computation;

a memory which stores information for the computation; and

an output unit which outputs the generated mosaic image,

wherein the processing unit includes;

a matrix calculating module which calculates a transformation matrix for transformation between two adjacent images obtained from among the plurality of sequential images;

an adjustment element generating module which generates an adjustment element of the estimated transformation matrix; and

an image projecting module which projects each image on a mosaic plane by using the transformation matrix to which the calculated adjustment element has been applied,

wherein the matrix calculating module is configured to;

obtain two adjacent images from among the plurality of sequential images;

extract corresponding features from the two adjacent images; and

estimate a transformation matrix for transformation between the two adjacent images in order to minimize a total deviation between the corresponding features,

wherein the adjustment element generating module is configured to;

generate an adjustment element of the estimated transformation matrix;

obtain first control information, which is for projecting a first image included in a mosaic strip onto a predetermined mosaic plane, and second control information, which is for projecting a last image included in the mosaic strip onto the mosaic plane;

estimate a first transformation matrix, which transforms the first image onto the mosaic plane, and a last transformation matrix, which transforms the last image onto the mosaic plane by referring to the obtained control information;

estimate a second transformation matrix, which transforms the last image to an image on the mosaic plane by referring to the first transformation matrix estimated by the adjustment element generating module and the transformation matrix estimated by the matrix calculating module for transformation between images;

decompose a difference between the estimated last transformation matrix and the estimated second transformation matrix into rotational components and perspective components; and

calculate adjustment elements of the decomposed rotational components and adjustment elements of the decomposed perspective components, and

wherein the image projecting module is configured to;

estimate a transformation matrix for transforming each image included in the mosaic strip to an image on the mosaic plane by using the first transformation matrix estimated by the adjustment element generating module and the transformation matrix estimated by the matrix calculating module for transformation between images, and by applying the calculated adjustment elements; and

project the image onto the mosaic plane by referring to the transformation matrix estimated for each image.

2. The image mosaicing system according to claim 1,

wherein the first control information is information for projecting the first image to geo-register, and

wherein the second control information is information for projecting the last image to geo-register, and is used to spread errors accumulated in projecting images onto the mosaic plane over the entire images included in the mosaic strip.

3. The image mosaicing system according to claim 1, wherein the adjustment element generating module is designed to:

calculate an orientation angle A1 of the last image transformed by the last transformation matrix;

calculate an orientation angle A2 of the last image transformed by the second transformation matrix;

calculate a difference between the calculated orientation angles to obtain a rotation angle A=A2−

A1;

calculate an increment of the rotation angle A′

=A/(N−

1) by dividing the obtained rotation angle A by N−

1, N being number of images included in the mosaic strip; and

calculate an increment of a perspective matrix by referring to the obtained rotation angle increment as the adjustment elements of the rotational components.

4. The image mosaicing system according to claim 3, wherein the adjustment element generating module is configured to:

retrieve coordinates of four vertices of the image;

determine an intersection point of two diagonals of the image as an origin;

determine an arbitrary axis that runs through the origin; and

calculate an angle between one of the two diagonals and the determined axis with respect to the determined origin to obtain the orientation angle of the image.

5. The image mosaicing system according to claim 1, wherein the adjustment element generating module calculates the perspective components using the following Expression 1:

[(1−

t)Θ

A]⊕

(tΘ

B) 

 

(1)

tε

[0,1]

where the matrix A represents a rotation matrix which rotates the calculated rotation angle A, the matrix B represents the second transformation matrix, and the operators are expressed by the following Expressions 2 and 3;

sΘ

A=e^{s log A} 

 

(2)

A⊕

B=e^{log A+log B} 

 

(3)

6. An image mosaicing method of generating a mosaic image by compositing a plurality of sequential images which partially overlap with one another with the use of a computer, comprising:

a first step of calculating a transformation matrix for transformation between two adjacent images obtained from among the plurality of sequential images;

a second step of generating an adjustment element of the estimated transformation matrix; and

a third step of projecting each image on a mosaic plane by using the transformation matrix to which the calculated adjustment element has been applied,

wherein the first step includes the steps of;

obtaining two adjacent images from among the plurality of sequential images;

extracting corresponding features from the two adjacent images; and

estimating a transformation matrix for transformation between the two adjacent images in order to minimize a total deviation between the corresponding features,

wherein the second step includes the steps of;

generating an adjustment element of the estimated transformation matrix;

obtaining first control information, which is for projecting a first image included in a mosaic strip onto a predetermined mosaic plane, and second control information, which is for projecting a last image included in the mosaic strip onto the mosaic plane;

estimating a first transformation matrix, which transforms the first image onto the mosaic plane, and a last transformation matrix, which transforms the last image onto the mosaic plane by referring to the obtained control information to;

estimating a second transformation matrix, which transforms the last image to an image on the mosaic plane by referring to the first transformation matrix estimated in the second step and the transformation matrix estimated in the first step for transformation between images to;

decomposing a difference between the estimated last transformation matrix and the estimated second transformation matrix into rotational components and perspective components; and

calculating adjustment elements of the decomposed rotational components and adjustment elements of the decomposed perspective components, and

wherein the third step includes the steps of;

estimating a transformation matrix for transforming each image included in the mosaic strip to an image on the mosaic plane by using the first transformation matrix estimated in the second step and the transformation matrix estimated in the first step for transformation between images, and by further applying the calculated adjustment elements; and

project the image onto the mosaic plane by referring to the transformation matrix estimated for each image.

7. The image mosaicing method according to claim 6,

wherein the first control information is information for projecting the first image to geo-register, and

wherein the second control information is information for projecting the last image to geo-register, and is used to spread errors accumulated in projecting images onto the mosaic plane over the entire images included in the mosaic strip.

8. The image mosaicing method according to claim 6, wherein the step of calculating adjustment elements separately for the rotational components and the perspective components includes:

calculating an orientation angle A1 of the last image transformed by the last transformation matrix;

calculating an orientation angle A2 of the last image transformed by the second transformation matrix;

calculating a difference between the calculated orientation angles to obtain a rotation angle A=A2−

A1;

calculating an increment of the rotation angle A′

=A/(N−

1) by dividing the obtained rotation angle A by N−

1, N being number of images included in the mosaic strip; and

calculate an increment of a perspective matrix by referring to the obtained rotation angle increment as the adjustment elements of the rotational components.

9. The image mosaicing method according to claim 8, wherein the step of calculating the orientation angle of the last image includes:

retrieving coordinates of four vertices of a transformed image;

determining an intersection point of two diagonals of the image as an origin;

determining an arbitrary axis that runs through the origin;

calculating an angle between one of the two diagonals and the determined axis; and

calculating the orientation angle between one of the two diagonals and the determined axis with respect to the determined origin to obtain the orientation angle of the image.

10. The image mosaicing method according to claim 6, wherein the step of calculating adjustment elements separately for the rotational components and the perspective components comprises calculating the perspective components by the following Expression 1:

[(1−

t)Θ

A]⊕

(tΘ

B) 

 

(1)

tε

[0,1]

where the matrix A represents a rotation matrix which rotates the calculated rotation angle A, the matrix B represents the second transformation matrix, and the operators are expressed by the following Expressions 2 and 3;

sΘ

A=e^{s log A} 

 

(2)

A⊕

B=e^{log A+log B} 

 

(3).