Alignment of an ordered stack of images from a specimen

• US 9,159,130 B2
• Filed: 10/14/2010
• Issued: 10/13/2015
• Est. Priority Date: 10/22/2009
• Status: Active Grant

• Current Assignee: Koninklijke Philips Electronics N.V. (Koninklijke Philips N.V.)
• Original Assignee: Koninklijke Philips N.V.
• Inventors: Kneepkens, Rik E. J.
• Primary Examiner(s): Kassa, Yosef
• Application Number:
  US13/502,530
• Publication Number:
  US 20120207367A1
• Time in Patent Office: 1,825 Days
• Field of Search: 382/275, 382/284, 382/293, 382/294, 382/295, 382/296, 382/128, 382/130, 382/132, 358/540, 358/450, 128/922, 128/923
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20021 Dividing image into blocks, subimages or windows
• CPC Class Codes: G06T 2207/30024 Cell structures in vitro; Tissue sections in vitro
• CPC Class Codes: G06T 7/30 Determination of transform parameters for the alignment of images, i.e. image registration
• CPC Class Codes: G06T 7/33 using feature-based methods
• CPC Class Codes: G06T 7/337 involving reference images or patches
• CPC Class Codes: G06V 20/695 Preprocessing, e.g. image segmentation

Abstract

The present invention relates to the field of alignment of an ordered stack of images from a sliced specimen. According to the present method and apparatus, the ordered stack of images is aligned by successively determining, for at least two already aligned images of the ordered stack, the respective misalignments with an unaligned image which is to be aligned next, selecting from the at least two aligned images as a reference image that aligned image with which the unaligned image has the smallest amount of misalignment, and aligning the unaligned image with the selected reference image. This is intended to provide a robust and computationally cheap aligning method and apparatus.

Claims

1. A method for aligning an ordered stack of images from a specimen, comprising the successive steps of:

determining, for at least two already aligned images of the ordered stack, the respective misalignments with an unaligned image which is to be aligned next in the ordered stack;

discarding the unaligned image to be aligned next from the ordered stack of images if all of the determined misalignments amounts are above a certain threshold;

replacing the discarded image with an interpolated image generated by an interpolation of the aligned images;

from the at least two aligned images, selecting as a reference image that aligned image with which the unaligned image has the smallest amount of misalignment; and

aligning the unaligned image with the selected reference image;

wherein all the above steps are performed via a processor.

2. The method of claim 1, wherein the respective misalignments are determined by motion estimation so that each misalignment is described by an estimated displacement field.

3. The method of claim 2, wherein the displacement field comprises at least one displacement vector describing how to shift pixels in the image to be aligned in order to align this image to the selected reference image.

4. The method of claim 2, wherein the aligned images and the unaligned image to be aligned next are divided into blocks and wherein the respective displacement fields are obtained by a block matching algorithm.

5. The method of claim 1, wherein the respective amounts of misalignment are calculated in terms of a respective matching error between the respective aligned image and the unaligned image to be aligned next, wherein the matching errors are calculated based on a predetermined matching criterion, and wherein the aligned image having the smallest matching error is selected as the reference image.

6. The method of claim 1, wherein the aligning of the unaligned image with the selected reference image comprises transforming the unaligned image based on transformation parameters describing how the unaligned image has to be transformed in order to achieve alignment with the selected reference image, and wherein for each aligning step only the respective unaligned image which is currently being aligned is being transformed.

7. The method of claim 1, wherein before determining the respective misalignments at least one of the images is converted into another signal space so that all images are in comparable signal space.

8. The method of claim 1, wherein the respective misalignments are determined by object recognition.

9. The method of claim 1, wherein the respective misalignments are determined by motion estimation so that each misalignment is described by an estimated displacement field, and wherein, if large amounts of misalignment repeatedly occur, the misalignments are then determined by object recognition instead of motion estimation.

10. The method of claim 1, wherein an aligned image will not be used for further misalignment determining steps if its amount of misalignment in the current misalignment determining step is above a certain threshold as compared to remaining amounts of misalignment.

11. An apparatus for aligning an ordered stack of images from a specimen, comprising the following devices:

a determiner for determining for at least two already aligned images of the ordered stack the respective misalignments with an unaligned image which is to be aligned next in the ordered stack;

a discarding processor for discarding the unaligned image to be aligned next from the ordered stack of images if all of the determined misalignments amounts are above a certain threshold;

a replacing processor for replacing the discarded image with an interpolated image generated by an interpolation of aligned images;

a selector for selecting, from the at least two aligned images, as a reference image that aligned image with which the unaligned image has the smallest amount of misalignment; and

an aligner for aligning the unaligned image with the selected reference image.

12. The apparatus of claim 11, further comprising a motion estimator for determining the respective misalignments by motion estimation so that each misalignment is described by an estimated displacement field.

13. The apparatus of claim 12, wherein the displacement field comprises at least one displacement vector describing how to shift pixels in the image to be aligned in order to align this image to the selected reference image.

14. A non-transitory computer readable medium storing a program, for controlling an apparatus for aligning an ordered stack of images from a specimen, which, when executed by at least one processor, causes the at least one processor to carry out successive steps of:

determining, for at least two already aligned images of the ordered stack, the respective misalignments with an unaligned image which is to be aligned next in the ordered stack;

discarding the unaligned image to be aligned next from the ordered stack of images if all of the determined misalignments amounts are above a certain threshold;

replacing the discarded image with an interpolated image generated by an interpolation of the aligned images;

from the at least two aligned images, selecting as a reference image that aligned image with which the unaligned image has the smallest amount of misalignment; and

aligning the unaligned image with the selected reference image.