METHOD OF REGISTERING AND ALIGNING MULTIPLE IMAGES

• US 20080123996A1
• Filed: 10/31/2007
• Published: 05/29/2008
• Est. Priority Date: 05/13/2005
• Status: Abandoned Application

• Current Assignee: Semiconductor Insights Incorporated (Informa PLC)
• Original Assignee: Semiconductor Insights Incorporated (Informa PLC)
• Inventors: Martincevic, Vladimir, Abt, Jason, Zavadsky, Vyacheslav, Keyes, Edward, Braverman, Mark
• Application Number:
  US11/931,465
• Publication Number:
  US 20080123996A1
• Time in Patent Office: Days
• US Class Current: 382/294
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: G06V 10/16 using multiple overlapping images; Image stitching

Abstract

A method of registering and vertically aligning multiply-layered images into a mosaic is described. The method comprises performing an iterative process of vertical alignment of layers into a mosaic using a series of defined alignment correspondence pairs and global registration of images in a layer using a series of defined registration correspondence points and then redefining the identified alignment correspondence pairs and/or registration correspondence points until a satisfactory result is obtained. Optionally, an initial global registration of each layer could be performed initially before commencing the alignment process. The quality of the result could be determined using a least squares error minimization or other technique.

Claims

1. A method of creating a horizontally registered and vertically aligned multiple-layered mosaic of images of an object, comprising the steps of:

a. for each layer of detail of the portion of the object to be imaged;

i. capturing the detail of the layer in a series of captured images;

ii. performing pair-wise registration of the captured images of the layer into a mosaic; and

iii. performing global registration of the mosaic of the layer, into an optimised mosaic layer;

b. identifying at least one alignment correspondence pair, located on an image on a first optimised mosaic layer and on at least one image on a separate optimised mosaic layer; and

c. iteratively;

i. performing alignment and global registration of all of the optimised mosaic layers into a substantially aligned mosaic, using the identified alignment correspondence pairs associated with each optimised mosaic layer; and

ii. redefining the identified alignment correspondence pairs;

until the alignment of the layers is optimized;

wherein the step of redefining the identified alignment correspondence pairs comprises inspecting the results of the registration performed;

wherein the step of inspecting comprises calculating and assigning a minimum energy function to error in the alignment performed; and

wherein the step of calculating and assigning a minimum energy function comprises applying a least squares energy minimization algorithm.

2. A method according to claim 1, wherein the step of applying a least squares energy minimization algorithm comprises the steps of:

a. grouping data concerning registration error into a plurality of groups according to reliability;

b. minimizing weighted least squares energy value obtained from application of the least squares energy minimizing algorithm, taking into account the most reliable group; and

c. minimizing energy value obtained from application of the least squares energy minimizing algorithm in the least reliable groups while ensuring no increase in the energy value of the most reliable group.

3. A method of creating a horizontally registered and vertically aligned multiple-layered mosaic of images of an object, comprising the steps of:

a. for each layer of detail of the portion of the object to be imaged;

i. capturing the detail of the layer in a series of captured images;

ii. performing pair-wise registration of the captured images of the layer into a mosaic; and

iii. performing global registration of the mosaic of the layer, into an optimised mosaic layer;

b. identifying at least one alignment correspondence pair, located on an image on a first optimised mosaic layer and on at least one image on a separate optimised mosaic layer; and

c. iteratively;

i. performing alignment and global registration of all of the optimised mosaic layers into a substantially aligned mosaic, using the identified alignment correspondence pairs associated with each optimised mosaic layer; and

ii. redefining the identified alignment correspondence pairs;

until the alignment of the layers is optimized;

wherein the step of redefining the identified alignment correspondence pairs comprises inspecting the results of the registration performed;

wherein the step of inspecting comprises calculating and assigning a minimum energy function to error in the alignment performed; and

wherein the step of calculating and assigning a minimum energy comprises applying a 95% order statistical energy function.

4. A method of creating a horizontally registered and vertically aligned multiple-layered mosaic of images of an object, comprising the steps of:

a. Capturing the detail of each of the layers in a series of images;

b. Performing pair-wise registration of the captured images of each of the layers into a mosaic;

c. for a plurality of layers of detail of the object to be imaged;

i. identifying at least one registration correspondence point, each associated with two images, and identifying at least one alignment correspondence pair, located on an image on a first layer and on at least one image on a different one of the plurality of layers;

ii. iteratively;

a) performing global registration of the captured images of each of the plurality of layers into a mosaic, using the identified registration correspondence points for the layer and performing alignment of all of the plurality of layers into a mosaic, using the identified alignment correspondence pairs;

b) redefining the identified registration correspondence points for each of the plurality of layers and the identified alignment correspondence pairs;

until the registration and alignment of the plurality of layers is optimised; and

iii. adding at least one layer to the plurality of layers;

until the alignment of all of the layers of detail of the portion of the object is imaged and optimized.

5. A method according to claim 4, wherein the step of performing pair-wise registration the captured images into a mosaic before the iteration step.

6. A method according to claim 4, wherein the step of identifying comprises the step of identifying registration correspondence points followed by the step of identifying alignment correspondence pairs.

7. A method according to claim 4, wherein the step of identifying comprises the step of identifying alignment correspondence pairs followed by the step of identifying registration correspondence pairs.

8. A method according to claim 4, wherein the step of redefining the identified registration correspondence points comprises discarding previously identified registration correspondence points as being unreliable.

9. A method according to claim 8, wherein the step of discarding previously identified registration correspondence points comprises discarding previously identified registration correspondence points as constituting outliers.

10. A method according to claim 8, wherein the step of discarding previously identified registration correspondence points comprises discarding previously identified registration correspondence points as constituting errors.

11. A method according to claim 4, wherein the step of redefining the identified registration correspondence points comprises identifying additional registration correspondence points.

12. A method according to claim 4, wherein the step of redefining the identified alignment correspondence pairs comprises discarding previously identified alignment correspondence pairs as being unreliable.

13. A method according to claim 12, wherein the step of discarding previously identified alignment correspondence points as constituting outliers.

14. A method according to claim 12, wherein the step of discarding previously identified alignment correspondence points as constituting errors.

15. A method according to claim 4, wherein the step of redefining the identified alignment correspondence pairs comprises identifying additional alignment correspondence points.

16. A method of creating a horizontally registered and vertically aligned multiple-layered mosaic of images of a portion of an object, comprising the steps of:

(b) for each layer of detail of the portion of the object to be imaged;

(i) capturing the detail of the layer in a series of captured images;

(ii) performing pair-wise registration of the captured images of the layer into a mosaic;

(iii) identifying at least on registration correspondence point, associated with at least two images;

(iv) performing global registration of the mosaic of the layer, into an optimized mosaic layer using the identified registration correspondence points associated with the layer; and

(v) redefining the identified registration correspondence points for the layer;

(c) Identifying at least one alignment correspondence pair, located on an image of a first optimised mosaic layer and on at least one image on a separate optimized mosaic layer; and

(d) Iteratively;

(i) performing alignment and global registration of all of the optimised mosaic layers into a substantially aligned mosaic, using the identified alignment correspondence pairs associated with each optimized mosaic layer; and

(ii) redefining the identified alignment correspondence pairs and the identified registration correspondence points;

until the alignment of the layers is optimized.