Coregistering Images of Needle Biopsies Using Multiple Weighted Landmarks

• US 20140228707A1
• Filed: 02/11/2013
• Published: 08/14/2014
• Est. Priority Date: 02/11/2013
• Status: Active Grant

• Current Assignee: Definiens AG (Astrazeneca PLC)
• Original Assignee: Definiens AG (Astrazeneca PLC)
• Inventors: Schmidt, Guenter, Binnig, Gerd, Brieu, Nicolas, Goerner, Melanie
• Granted Patent: US 9,060,672 B2
• Time in Patent Office: Days
• US Class Current: 600/567
• CPC Class Codes: A61B 10/0233 Pointed or sharp biopsy instruments
• CPC Class Codes: A61B 5/00 Measuring for diagnostic purposes radiation diagnosis A61B6/00; diagnosis by ultrasonic, sonic or infrasonic waves A61B8/00; Identification of persons
• CPC Class Codes: G02B 21/367 providing an output produced by processing a plurality of individual source images, e.g. image tiling, montage, composite images, depth sectioning, image comparison
• CPC Class Codes: G06T 2200/24 involving graphical user interfaces [GUIs]
• CPC Class Codes: G06T 2207/10056 Microscopic image
• CPC Class Codes: G06T 2207/20072 Graph-based image processing
• CPC Class Codes: G06T 2207/20101 Interactive definition of point of interest, landmark or seed
• CPC Class Codes: G06T 2207/30024 Cell structures in vitro; Tissue sections in vitro
• CPC Class Codes: G06T 2207/30172 Centreline of tubular or elongated structure
• CPC Class Codes: G06T 7/344 involving models
• CPC Class Codes: G06V 20/695 Preprocessing, e.g. image segmentation

Abstract

A method for coregistering images involves defining middle paths through image objects depicting tissue slices of needle biopsies. First landmarks are defined on a first middle path through a first image object in a first digital image of a first tissue slice, and second landmarks are defined on a second middle path through a second image object of a second digital image of a second tissue slice. Individual first landmarks are associated with individual second landmarks. A first pixel in the first object is coregistered with a second pixel in the second object using multiple first and second landmarks. The first image is displayed in a first frame on a graphical user interface, and the second image is displayed in a second frame such that the first pixel is centered in the first frame, the second pixel is centered in the second frame, and the images have the same orientations.

Claims

1. A method comprising:

defining a first path in a first image object of a first digital image of a first tissue slice;

defining a second path in a second image object of a second digital image of a second tissue slice;

defining first landmarks on the first path;

defining second landmarks on the second path;

associating individual first landmarks with individual second landmarks;

coregistering a first pixel in the first image object with a second pixel in the second object using multiple first landmarks and multiple second landmarks; and

displaying on a graphical user interface the first digital image in a first frame and the second digital image in a second frame, wherein the first pixel is positioned at a first reference location inside the first frame and the second pixel is positioned at a second reference location inside the second frame, and wherein the first reference location relative to the first frame corresponds to the second reference location relative to the second frame.

2. The method of claim 1, wherein the first path passes through the first image object halfway between opposite boundaries of the first image object.

3. The method of claim 1, wherein the first path resembles an outer contour of the first image object.

4. The method of claim 1, further comprising:

defining an additional first landmark not on the first path but within the first image object at a location determined based on a local direction of the first path.

5. The method of claim 1, wherein the displaying on the graphical user interface is performed before the coregistering the first pixel with the second pixel.

6. The method of claim 1, further comprising:

selecting the second digital image;

shifting the second digital image based on user input while the first digital image does not move relative to the first frame; and

coregistering the first pixel at the first reference location with a third pixel that is positioned at the second reference location as a result of the shifting.

7. The method of claim 1, further comprising:

selecting the second digital image;

rotating the second digital image based on user input while the first digital image does not move relative to the first frame; and

coregistering the first digital image with the second digital image as the second digital image is positioned after the rotating.

8. A method comprising:

defining a first middle path through a first image object of a first digital image of a first tissue slice;

defining a second middle path through a second image object of a second digital image of a second tissue slice;

defining first landmarks on the first middle path;

defining second landmarks on the second middle path;

associating individual first landmarks with individual second landmarks;

coregistering a first pixel in the first image object with a second pixel in the second object using multiple first landmarks and multiple second landmarks; and

displaying on a graphical user interface the first digital image in a first frame and the second digital image in a second frame, wherein the first pixel is centered in the first frame and the second pixel is centered in the second frame.

9. The method of claim 8, further comprising:

shifting the second digital image within the second frame in response to a user shifting the first digital image relative to the first frame, wherein the second digital image is shifted such that each pixel of the second digital image that is centered in the second frame corresponds to a coregistered pixel of the first digital image that is centered in the first frame.

10. The method of claim 9, wherein each coregistered pixel of the second digital image is determined based only on those second landmarks that are associated with first landmarks located on the first image object.

11. The method of claim 9, wherein each coregistered pixel of the second digital image is determined using first landmarks that are weighted based on their distances to the first pixel.

12. The method of claim 8, further comprising:

shifting the second digital image within the second frame in response to a user shifting the first frame over the first digital image, wherein the second digital image is shifted relative to the second frame such that each pixel of the second digital image that is centered in the second frame corresponds to a coregistered pixel of the first digital image that is centered in the first frame.

13. The method of claim 8, wherein the first landmarks include an additional landmark that is located outside of the first image object.

14. The method of claim 8, wherein the coregistering is performed such that a spatial relationship between the second pixel and multiple second landmarks corresponds to the spatial relationship between the first pixel and multiple first landmarks.

15. The method of claim 14, wherein the spatial relationship between the first pixel and the multiple first landmarks is defined by assigning a larger weighting to those first landmarks that are closer to the first pixel.

16. The method of claim 8, wherein the first tissue slice is taken from a needle biopsy.

17. The method of claim 8, wherein the second digital image is displayed in the second frame in a coregistered orientation to how the first digital image is displayed in the first frame.

18. The method of claim 8, wherein less than the entire first digital image is displayed in the first frame, and less than the entire second digital image is displayed in the second frame.

19. A method comprising:

displaying a first digital image of a first tissue slice on a graphical user interface such that an area of the first digital image is enclosed by a frame;

displaying a portion of a second digital image of a second tissue slice on the graphical user interface, wherein the first tissue slice and the second tissue slice are taken from a needle biopsy, wherein the displayed portion of the second digital image has a shape of the frame, and wherein the displayed portion of the second digital image corresponds to the area of the first digital image that is enclosed by the frame;

initially coregistering a pixel that is centered in the displayed portion of the second digital image with an associated pixel of the first digital image that is centered in the frame using multiple second landmarks in the second digital image and multiple associated first landmarks in the first digital image; and

shifting the displayed portion of the second digital image in response to a user shifting the first digital image relative to the frame, wherein the second digital image is shifted such that each coregistered pixel of the displayed portion of the second digital image is determined using first landmarks that are weighted based on their distance from the center of the frame.

20. The method of claim 19, wherein the second digital image is shifted such that each coregistered pixel of the displayed portion of the second digital image is determined using first landmarks that are weighted based on their distances to the associated pixel of the first digital image that is centered in the frame.

21. The method of claim 19, wherein as the user shifts the first digital image each pixel that is centered in the displayed portion of the second digital image corresponds to a coregistered pixel of the first digital image that is centered in the frame.

22. The method of claim 19, further comprising:

dynamically coregistering the pixels of the displayed portion of the second digital image as the user shifts the first digital image such that the pixel that is centered in the displayed portion is coregistered with the pixel that is centered in the frame using only those second landmarks that are associated with first landmarks located inside the frame.

23. The method of claim 19, further comprising:

dynamically coregistering the pixels of the displayed portion of the second digital image as the user zooms the first digital image within the frame such that the pixel that is centered in the displayed portion is coregistered with the pixel that is centered in the frame.

24. The method of claim 19, wherein the displayed portion of the second digital image is displayed in a coregistered orientation to how the first digital image is oriented in the frame.

25. The method of claim 19, wherein the first landmarks are located along a middle path of an image object in the first digital image.

26. The method of claim 19, wherein the first and second landmarks include orientation vectors, and wherein the relative orientation of the first digital image to the second digital image is calculated by a weighted mean of the orientation vectors of the first landmarks and a weighted mean of the orientation vectors of the second landmarks.