AUTOMATED SYNCHRONIZED NAVIGATION SYSTEM FOR DIGITAL PATHOLOGY IMAGING

• US 20130243302A1
• Filed: 03/14/2012
• Published: 09/19/2013
• Est. Priority Date: 03/14/2012
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: LIU, Ming-Chang, YU, Liangyin, MIZUTANI, Yoichi, YOSHIOKA, Shigeatsu, KAJIMOTO, Masato
• Granted Patent: US 8,755,633 B2
• Time in Patent Office: Days
• US Class Current: 382/133
• CPC Class Codes: G06T 2207/30024 Cell structures in vitro; Tissue sections in vitro
• CPC Class Codes: G06T 3/147 using affine transformations
• CPC Class Codes: G06T 7/33 using feature-based methods

Abstract

A method for synchronizing navigation in pathology stain images includes (a) downscaling the pathology stain images, (b) estimating rotation of the downscaled images, (c) aligning the downscaled images to generate aligned coordinates, and (d) transforming the aligned coordinates to original image coordinates in the pathology stain images to thereby generate alignment data. Also provided is a system for synchronized navigation in pathology stain images having original resolutions comprising a downscaler, a rotation estimator, an alignment module, and a coordinate transformer. The system may also include an image display system to display corresponding areas of the pathology stain images.

Claims

1. A computer-implemented method for synchronizing navigation in pathology stain images, the method comprising the steps performed by one or more computers of:

(a) downscaling the pathology stain images;

(b) estimating rotation of the downscaled images;

(c) aligning the downscaled images to generate aligned coordinates; and

(d) transforming the aligned coordinates to original image coordinates in the pathology stain images to thereby generate alignment data.

2. The method of claim 1, wherein (a) downscaling the pathology stain images comprises:

detecting tissue areas in the pathology stain images;

determining suitable resolutions for aligning the pathology stain images; and

downscaling the pathology stain images to the determined resolutions.

3. The method of claim 1, wherein (b) estimating rotation comprises:

selecting a candidate rotation angle;

rotating one of the downscaled images by the candidate rotation angle;

aligning, after the rotating, the downscaled images to determine matched point features between the downscaled images; and

selecting a final rotation angle based on the number of matched point features.

4. The method of claim 3, wherein estimating rotation further comprises

filtering out false matched point features.

5. The method of claim 3, further comprising:

filling in background for the rotated image; and

compensating translation for the rotated image.

6. The method of claim 1, wherein (c) aligning the downscaled images comprises:

determining correspondent point features between the downscaled images;

creating triangular meshes for the downscaled images from the correspondent point features;

refining point correspondence between the downscaled images based on affine transformation estimation using the triangular meshes;

creating refined triangular meshes for the downscaled images from the refined point correspondence; and

generating aligned coordinates through affine transformation based on the refined triangular meshes.

7. The method of claim 6, further comprising

re-determining aligned coordinates between the downscaled images to control the number of aligned coordinates.

8. The method of claim 1, wherein (d) transforming the aligned coordinates to original image coordinates comprises:

mapping the aligned coordinates back to original image coordinates in the pathology stain images at original resolutions and in original orientations;

creating triangular meshes for the pathology stain images based on the mapped original image coordinates; and

generating alignment data through affine mapping using the triangular meshes.

9. The method of claim 8, wherein generating alignment data comprises generating forward and backward mapping functions between the pathology stain images.

10. The method of claim 1, wherein the pathology stain images are selected from immunohistochemical (IHC) stain images and hematoxylin and eosin (H&

E) stain images.

11. The method of claim 1, further comprising displaying corresponding areas of the pathology stain images based on the alignment data.

12. The method of claim 11, wherein the displaying comprises:

determining the center position of one of the pathology stain images;

transforming the center position to a corresponding coordinate another pathology stain image based on the alignment data; and

moving the center position of the another pathology stain image to the corresponding coordinate.

13. A computer-implemented system for synchronized navigation in pathology stain images, comprising:

a downscaler to detect tissue areas and downsample the pathology stain images;

a rotation estimator to determine a rotation angle and rotate the downscaled images;

an alignment module to align the downscaled images to generate aligned coordinates; and

a coordinate transformer to transform the aligned coordinates to original image coordinates in the pathology stain images to thereby generate alignment data.

14. The system of claim 13, further comprising a display system to display corresponding areas of the pathology stain images based on the alignment data.

15. The system of claim 13, further comprising a database to store alignment data.

16. The system of claim 13, wherein the downscaler is configured to:

detect tissue areas in the pathology stain images;

determine suitable resolutions for aligning the pathology stain images; and

downscale the pathology stain images to the determined resolutions.

17. The system of claim 13, wherein the rotation estimator is configured to:

estimate the rotation angle between the downscaled images;

rotate at least one of the downscaled images by the rotation angle such that the downscaled images are aligned in orientation;

fill in background for the at least one rotated image; and

compensate translation for the at least one rotated image

18. The system of claim 13, wherein the alignment module is configured to:

determine correspondent point features between the downscaled images;

create triangular meshes for the downscaled images from the correspondent point features; and

generate aligned coordinates through affine transformation based on the triangular meshes.

19. The system of claim 13, wherein the coordinate transformer is configured to:

map the aligned coordinates back to original image coordinates in the pathology stain images at original resolutions and original orientations;

create triangular meshes for the pathology stain images based on the mapped original image coordinates; and

generate alignment data through affine mapping using the triangular meshes.

20. The system of claim 14, wherein the system is configured to:

identify the center position of a first one of the pathology stain images;

transform the center position to a corresponding coordinate in a second one of the pathology stain images based on the alignment data; and

move the center position of the another pathology stain image to the corresponding coordinate.