SYSTEM AND METHOD FOR ANNOTATING IMAGES BY PROPAGATING INFORMATION

• US 20150262372A1
• Filed: 09/27/2013
• Published: 09/17/2015
• Est. Priority Date: 09/28/2012
• Status: Active Grant

• Current Assignee: UCL Business PLC (University of London)
• Original Assignee: UCL Business PLC (University of London)
• Inventors: Ourselin, Sebastien, Cardoso, Manuel Jorge, Wolz, Robin, Ruekert, Daniel
• Granted Patent: US 9,619,890 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06T 2207/10072 Tomographic images
• CPC Class Codes: G06T 2207/10088 Magnetic resonance imaging [MRI]
• CPC Class Codes: G06T 2207/20036 Morphological image processing
• CPC Class Codes: G06T 2207/20072 Graph-based image processing
• CPC Class Codes: G06T 2207/20128 Atlas-based segmentation
• CPC Class Codes: G06T 2207/20216 Image averaging
• CPC Class Codes: G06T 2207/30016 Brain
• CPC Class Codes: G06T 7/162 involving graph-based methods
• CPC Class Codes: G06T 7/174 involving the use of two or more images

Abstract

One embodiment of the invention provides a computer-implemented method of annotating images in a data set comprising a first set of images which initially have annotations and a second set of images which initially do not have annotations. The method comprises the steps of determining spatial correspondences for all pairs of images in the data set, wherein each spatial correspondence represents a mapping of a location in a first image to a corresponding location in a second image. For each mapping, the method further comprises calculating a connection strength between the location in the first image and the corresponding location in the second image. The method further comprises iteratively propagating information across images in the data set using the connection strengths of the mappings as weights for propagating the information, augmented by an estimate of propagation accuracy, wherein the propagated information is used to provide annotations to images which do not initially have annotations, and wherein each iteration performs a propagation across all images in the data set.

Claims

1. A computer-implemented method of annotating images in a data set comprising a first set of images which initially have annotations and a second set of images which initially do not have annotations, the method comprising:

determining spatial correspondences for all pairs of images in the data set, wherein each spatial correspondence represents a mapping of a location in a first image in the first set of images to a corresponding location in a second image in the second set of images;

for each mapping, calculating a connection strength between the location in the first image and the corresponding location in the second image; and

iteratively propagating information across images in the data set using the connection strengths of the mappings as weights for propagating the information, augmented by an estimate of propagation accuracy, wherein the propagated information is used to provide annotations to images which do not initially have annotations, and wherein each iteration performs a propagation across all images in the data set.

2. The method of claim 1, further comprising defining a pairwise measure of similarity between one or more features in a pair of images for use in determining said connection strength.

3. The method of claim 2, wherein the pairwise measure of similarity is based on intensity similarity between the images.

4. The method of claim 2, wherein the pairwise measure of similarity is based on a characteristic of deformation between a pair of images.

5. The method of any of claim 2, wherein the pairwise measure of similarity is based on clinical metadata, including age, gender, pathological status, cognitive or biological testing.

6. The method of any of claim 2, wherein the pairwise measure of similarity is calculated locally on a restricted portion of the images.

7. The method of claim 1, wherein the connection strength is spatially variant across a pair of images.

8. The method of claim 1, wherein a pair of images represents different properties of the same object.

9. The method of claim 1, wherein the propagation step is performed by weighted averaging, classifier fusion or kernel regression.

10. The method of claim 1, wherein the step of propagating the information comprises propagating labels, intensities, probabilities, pathological status, statistics, vectors, displacement fields or a combination of these.

11. The method of claim 1, wherein the connection strength and/or the information propagation step is augmented by an intensity-based refinement operation.

12. The method of claim 1, wherein the estimate of propagation accuracy is based on the number of iterations before an image is annotated.

13. The method of claim 1, wherein the estimate of propagation accuracy is based on a normalised weighted sum of information available at directly neighbouring locations.

14. The method of claim 13, wherein the weighted sum involves weightings that are based on a geodesic distance, such that images in the second set which are closer to images in the first set are regarded as having a higher accuracy.

15. The method of claim 14, wherein the geodesic distance represents the distance between a given position in an image in the second set and the closest source of information for that position in a separate image from the first set.

16. The method of claim 14, wherein the weighting is based on the sum of (i) the smallest value of a neighbour'"'"'s geodesic distance, and (ii) the pairwise distance for all neighbouring nodes.

17. The method of claim 1, wherein the images are of different subjects, different pathologies, different time-points of the same subject, or any combination of these.

18. The method of claim 1, wherein the images are magnetic resonance images.

19. A non-transitory computer-readable storage medium storing instructions that when executed by a computer cause the computer to perform a method for annotating images in a data set comprising a first set of images which initially have annotations and a second set of images which initially do not have annotations, the method comprising:

determining spatial correspondences for all pairs of images in the data set, wherein each spatial correspondence represents a mapping of a location in a first image in the first set of images to a corresponding location in a second image in the second set of images;

for each mapping, calculating a connection strength between the location in the first image and the corresponding location in the second image; and

iteratively propagating information across images in the data set using the connection strengths of the mappings as weights for propagating the information, augmented by an estimate of propagation accuracy, wherein the propagated information is used to provide annotations to images which do not initially have annotations, and wherein each iteration performs a propagation across all images in the data set.

20. (canceled)

21. An imaging apparatus that annotates images in a data set comprising a first set of images which initially have annotations and a second set of images which initially do not have annotations, wherein during operation, the imaging apparatus:

determines spatial correspondences for all pairs of images in the data set, wherein each spatial correspondence represents a mapping of a location in a first image in the first set of images to a corresponding location in a second image in the second set of images;

for each mapping, calculates a connection strength between the location in the first image and the corresponding location in the second image; and

iteratively propagates information across images in the data set using the connection strengths of the mappings as weights for propagating the information, augmented by an estimate of propagation accuracy, wherein the propagated information is used to provide annotations to images which do not initially have annotations, and wherein each iteration performs a propagation across all images in the data set.

22. The imaging apparatus of claim 21, wherein said apparatus comprises a medical scanner.

23. A computer system for annotating images in a data set comprising a first set of images which initially have annotations and a second set of images which initially do not have annotations, wherein during operation, the computer system:

determines spatial correspondences for all pairs of images in the data set, wherein each spatial correspondence represents a mapping of a location in a first image to a corresponding location in a second image;

for each mapping, calculates a connection strength between the location in the first image and the corresponding location in the second image; and

iteratively propagates information across images in the data set using the connection strengths of the mappings as weights for propagating the information, augmented by an estimate of propagation accuracy, wherein the propagated information is used to provide annotations to images which do not initially have annotations, and wherein each iteration performs a propagation across all images in the data set.

24. (canceled)

25. (canceled)