Image processing device, pathological diagnosis support system, image processing program, and pathological diagnosis support method

• US 9,779,500 B2
• Filed: 06/27/2014
• Issued: 10/03/2017
• Est. Priority Date: 07/03/2013
• Status: Active Grant

• Current Assignee: Konica Minolta Inc.
• Original Assignee: Konica Minolta Inc.
• Inventors: Watanabe, Yasuhiro, Isoda, Takeshi, Gonda, Kohsuke, Ohuchi, Noriaki, Watanabe, Mika
• Primary Examiner(s): Tabatabai, Abolfazl
• Application Number:
  US14/902,410
• Publication Number:
  US 20160371834A1
• Time in Patent Office: 1,194 Days
• Field of Search: 382128-134, 600300, 600411, 600427
• CPC Class Codes: A61B 2576/00 Medical imaging apparatus involving image processing or analysis A61B1/00009, A61B6/52 and A61B8/52 take precedence; image processing per se G06T; details of algorithms for analysing medical images G06T7/0012
• CPC Class Codes: A61B 5/0062 Arrangements for scanning
• CPC Class Codes: A61B 5/0071 by measuring fluorescence emission
• CPC Class Codes: A61B 5/7264 Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems neural networks per se G06N3/00; expert systems per se G06N5/00
• CPC Class Codes: G01N 21/6456 Spatial resolved fluorescence measurements; Imaging
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/10061 from scanning electron microscope
• CPC Class Codes: G06T 2207/10064 Fluorescence image
• CPC Class Codes: G06T 2207/20036 Morphological image processing
• CPC Class Codes: G06T 2207/20221 Image fusion; Image merging
• CPC Class Codes: G06T 2207/30004 Biomedical image processing
• CPC Class Codes: G06T 2207/30024 Cell structures in vitro; Tissue sections in vitro
• CPC Class Codes: G06T 3/14 Transformations for image registration, e.g. adjusting or mapping for alignment of images
• CPC Class Codes: G06T 7/0012 Biomedical image inspection
• CPC Class Codes: G06T 7/33 using feature-based methods
• CPC Class Codes: G06T 7/60 Analysis of geometric attributes
• CPC Class Codes: G06V 20/693 Acquisition
• CPC Class Codes: G06V 20/695 Preprocessing, e.g. image segmentation
• CPC Class Codes: G06V 20/698 : Matching; Classification
• CPC Class Codes: G16H 30/40 : for processing medical images, e.g. editing

Abstract

An image processing device includes an input unit and an alignment unit. The input unit inputs a cell shape image and a fluorescence image. The cell shape image shows a shape of a cell in a tissue section. The fluorescence image shows expression of a specific protein as a fluorescent bright point in a region same as a region in the tissue section. The alignment unit aligns the cell shape image and the fluorescence image based on an information source detected in both the cell shape image and the fluorescence image.

Claims

1. An image processing device comprising:

an interface which inputs a cell shape image showing a shape of a cell in a tissue section and a fluorescence image showing expression of a specific protein as a fluorescent bright point in a region same as a region in the tissue section; and

a central processing unit (CPU) which aligns the cell shape image and the fluorescence image based on an information source detected in both the cell shape image and the fluorescence image,

wherein the information source is staining information with a certain staining reagent, and the CPU calculates image feature amounts in the cell shape image and the fluorescence image from the staining information on the cell shape image and the staining information on the fluorescence image, respectively, and aligns the cell shape image and the fluorescence image based on the image feature amounts.

2. The image processing device according to claim 1, wherein

the information source includes other staining information with a certain staining reagent different from the certain straining reagent, and

the CPU calculates image feature amounts in the cell shape image and the fluorescence image from the staining information on the cell shape image and the other staining information on the fluorescence image, respectively, and aligns the cell shape image and the fluorescence image based on the image feature amounts.

3. The image processing device according to claim 2, wherein the another staining reagent contains a fluorescent particle different from a fluorescent particle as a source of the fluorescent bright point in the fluorescence image at least in one of emission wavelength, particle size and shape.

4. The image processing device according to claim 1,

wherein the CPU calculates from the cell shape image a cell feature amount which includes at least one factor of:

area of a cell nucleus;

average density of a cell nucleus;

brightness variation of a cell nucleus;

circularity of a cell nucleus;

oblateness of a cell nucleus; and

thickness of a nuclear membrane of a cell nucleus to area of the cell nucleus;

the CPU specifies an observation target cell to observe based on the cell feature amount; and

the CPU which determines a status of the expression of the specific protein in the observation target cell based on the number of fluorescent bright points in the aligned fluorescence image on the observation target cell.

5. A pathological diagnosis support system comprising:

the image processing device according to claim 1; and

an image acquisition device which acquires the cell shape image and the fluorescence image used by the image processing device.

6. The image processing device according to claim 1, further comprising a display, wherein the CPU obtains analysis results based on the image feature amounts and the display displays the analysis results.

7. A non-transitory computer readable storage medium having an image processing program for execution on a computer the program causing the computer to perform the following steps:

inputting, using an interface, a cell shape image showing a shape of a cell in a tissue section and a fluorescence image showing expression of a specific protein as a fluorescent bright point in a region same as a region in the tissue section; and

aligning, using a central processing unit (CPU), the cell shape image and the fluorescence image based on an information source detected in both the cell shape image and the fluorescence image,

wherein the information source is staining information with a certain staining reagent, and the step of aligning includes calculating image feature amounts in the cell shape image and the fluorescence image from the staining information on the cell shape image and the staining information on the fluorescence image, respectively, and aligning the cell shaped image and the fluorescence image based on the image feature amounts.

8. A pathological diagnosis support method using a cell shape image showing a shape of a cell in a tissue section and a fluorescence image showing expression of a specific protein as a fluorescent bright point in a region same as a region in the tissue section, comprising:

a step of staining the tissue section with a certain staining reagent to be detected in both the cell shape image and the fluorescence image;

a step of acquiring the cell shape image and the fluorescence image of the stained tissue section; and

a step of aligning, using a central processing unit (CPU), the cell shape image and the fluorescence image based on staining information with the certain staining reagent,

wherein the step of aligning includes calculating image amounts in the cell shape image and the fluorescence image from the staining information on the cell shape image and the staining information on the fluorescence image, respectively, and aligning the cell shape image and the fluorescence image based on the image feature amounts.