System for interpretation of image patterns in terms of anatomical or curated patterns

• US 10,540,536 B2
• Filed: 08/03/2015
• Issued: 01/21/2020
• Est. Priority Date: 08/01/2014
• Status: Active Grant

• Current Assignee: Katholieke Universiteit Leuven, Vanderbilt University
• Original Assignee: Katholieke Universiteit Leuven KU Leuven R&D
• Inventors: Caprioli, Richard M., De Moor, Bart, Van De Plas, Raf, Verbeeck, Nico, Waelkens, Etienne
• Primary Examiner(s): Lu, Tom Y
• Application Number:
  US15/500,998
• Publication Number:
  US 20170220850A1
• Time in Patent Office: 1,632 Days
• CPC Class Codes: G06F 18/23 Clustering techniques
• CPC Class Codes: G06T 2207/10072 Tomographic images
• CPC Class Codes: G06T 2207/10081 Computed x-ray tomography [CT]
• CPC Class Codes: G06T 2207/10104 Positron emission tomography [PET]
• CPC Class Codes: G06T 2207/10116 X-ray image
• CPC Class Codes: G06T 2207/10128 Scintigraphy
• CPC Class Codes: G06T 2207/10132 Ultrasound image
• CPC Class Codes: G06T 2207/30048 Heart; Cardiac
• CPC Class Codes: G06T 7/0012 Biomedical image inspection
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/143 involving probabilistic approaches, e.g. Markov random field [MRF] modelling
• CPC Class Codes: G06V 20/698 Matching; Classification
• CPC Class Codes: H01J 49/0004 Imaging particle spectrometry

Abstract

Imaging mass spectrometry (IMS) has become a prime tool for studying the distribution of biomolecules in tissue. Although IMS data sets can become very large, computational methods have made it practically feasible to search these experiments for relevant findings. However, these methods lack access to an important source of information that many human interpretations rely upon: anatomical insight. In this work, this need is addressed by (1) integrating a curated anatomical data source with an empirically acquired IMS data source, establishing an algorithm-accessible link between them; and (2) demonstrating the potential of such an IMS-anatomical atlas link by applying it toward automated anatomical interpretation of ion distributions in tissue.

Claims

1. A method for facilitating interpretation of image patterns indicative of spatial distribution of a property using curated image patterns, the method comprising:

receiving an image pattern to interpret (IPTI) and a set of curated image patterns, wherein the IPTI and at least one curated image pattern of the set of curated image patterns have been spatially registered to a common set of spatial locations;

defining a model structure of a relationship between the IPTI and at least one curated image pattern using the common set of spatial locations, wherein one or more models that correspond to the model structure comprise coefficients and parameters and each set of different model coefficients and parameters represents a different interpretation of the IPTI; and

selecting a model of the one or more models that optimizes an objective function.

2. The method of claim 1, wherein the structure of the relationship between the IPTI and the at least one curated image pattern is one of non-linear and multivariate.

3. The method of claim 1, wherein the objective function comprises a definition and/or optimization constraints related to human understanding and analysis.

4. The method of claim 1, wherein the objective function comprises a definition and/or optimization constraints related to computer analysis or computational use.

5. The method of claim 1, further comprising generating a decomposition of the IPTI comprising the at least one curated image pattern.

6. The method of claim 1, further comprising scoring the one or more models based on the objective function.

7. The method of claim 1, wherein the set of curated image patterns comprise information indicative of regions of interest.

8. The method of claim 1, wherein the IPTI is acquired from a scan of an anatomical region and/or anatomical feature.

9. The method of claim 1, wherein the IPTI is acquired using one of mass spectrometry imaging, magnetic resonance imaging, an ion imaging source, an ambient-pressure ion source, a low-pressure ion source, a computed tomography, radiography, nuclear medicine, ultrasound, elastography, tactile imaging, photoacoustic imaging, thermography, tomography, echocardiography, and spectroscopy.

10. A device for facilitating interpretation of image patterns related to a property of interest using curated image patterns, the device comprising:

a non-transitory storage medium having software instructions stored thereon; and

a processor configured to receive an image pattern to interpret (IPTI) from an imaging device and to receive a set of curated image patterns that are spatially registered to a common set of spatial locations, wherein the processor is further configured to execute the software instructions to perform the steps of;

defining a model structure of a relationship between the IPTI and the at least one curated image pattern using the common set of spatial locations, wherein one or more models that correspond to the model structure comprise coefficients and parameters and each set of different model coefficients and parameters represents a different interpretation of the IPTI; and

selecting a model of the one or more models that optimizes an objective function.

11. The device of claim 10, wherein the imaging device is one of a mass spectrometer, an magnetic resonance imaging system, an ion imaging system, an ion imaging system having an ambient-pressure ion source, an ion imaging system having a low-pressure ion source, a computed tomography system, a radiography system, a nuclear medicine system, an ultrasound system, an elastography system, a tactile imaging system, a photoacoustic imaging system, a thermography imaging system, a tomography imaging system, an echocardiography imaging system, and a spectroscopy system.

12. The device of claim 10, wherein the set of curated patterns are one of a set of anatomical patterns, a set of areas defined by an anatomical atlas, a set of areas defined by the Allen Mouse Brain Atlas, and a set of biomedical patterns.

13. The device of claim 10, wherein the at least one curated pattern comprises ones in locations that are part of a region of interest and zeros in other regions than the region of interest.

14. The device of claim 13, wherein a Gaussian filter configured to eliminate crisp borders to soften binary assignments at edges of the region of interest of the at least one curated pattern.

15. The device of claim 10, wherein the common set of spatial locations are stored in an image registration, spatial transformation, or a spatial transformation matrix.

16. The device of claim 10, wherein spatially registering the IPTI and at least one curated image pattern comprises one or more of:

a manual image registration, an automated image registration, a rigid or affine registration, and a non-rigid registration.

17. The device of claim 10, wherein the IPTI is a microscopic image or a micrograph acquired from one or more of optical microscopy, transmission electron microscopy, scanning electron microscopy, ultramicroscopy, scanning probe microscopy, fluorescence microscopy, differential interference contrast microscopy, phase contrast microscopy, and tag-based microscopy.

18. The device of claim 10, wherein the IPTI is one of an ion image, an ion distribution image, a MALDI-based ion image, a SIMS-based ion image, a LA-ICP-based ion image, and a LAESI-based ion image.

19. The device of claim 10, wherein the processor is further configured to execute the software instructions and perform the step of generating a decomposition of the IPTI comprising the at least one curated image pattern.

20. A system for interpreting images, the system comprising:

an imaging device configured to generate an image pattern to interpret (IPTI) indicative of a spatial distribution of a property;

a processor configured to;

receive an image pattern to interpret (IPTI) and a set of curated image patterns, wherein the IPTI and at least one curated image pattern of the set of curated image patterns have been spatially registered to a common set of spatial locations;

define a model structure of a relationship between the IPTI and the at least one curated image pattern using the common set of spatial locations, wherein one or more models that correspond to the model structure comprise coefficients and parameters and each set of different model coefficients and parameters represents a different interpretation of the IPTI; and

select a model of the one or more models that optimizes an objective function.