MEDICAL IMAGING APPARATUS, IMAGE PROCESSING APPARATUS, AND IMAGE PROCESSING METHOD

• US 20200134825A1
• Filed: 10/07/2019
• Published: 04/30/2020
• Est. Priority Date: 10/29/2018
• Status: Active Grant

• Current Assignee: FUJIFILM Healthcare Corp. (Fujifilm Holdings Corporation)
• Original Assignee: Hitachi, Ltd.
• Inventors: LI, Yun, TOYOMURA, Takashi, INOUE, Kenta, MAEDA, Toshinori
• Granted Patent: US 11,450,003 B2
• Time in Patent Office: Days
• CPC Class Codes: A61B 6/032 Transmission computed tomography [CT]
• CPC Class Codes: A61B 6/5205 involving processing of raw data to produce diagnostic data
• CPC Class Codes: A61B 8/483 involving the acquisition of a 3D volume of data
• CPC Class Codes: A61B 8/488 involving Doppler signals
• CPC Class Codes: A61B 8/5207 involving processing of raw data to produce diagnostic data, e.g. for generating an image reconstruction G06T11/00
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G06T 11/003 Reconstruction from projections, e.g. tomography
• CPC Class Codes: G06T 2200/04 involving 3D image data
• CPC Class Codes: G06T 2200/24 involving graphical user interfaces [GUIs]
• CPC Class Codes: G06T 2207/10 Image acquisition modality
• CPC Class Codes: G06T 2207/10076 4D tomography; Time-sequential 3D tomography
• CPC Class Codes: G06T 2207/10081 Computed x-ray tomography [CT]
• CPC Class Codes: G06T 2207/10088 Magnetic resonance imaging [MRI]
• CPC Class Codes: G06T 2207/10136 3D ultrasound image
• CPC Class Codes: G06T 2207/20016 Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/20084 Artificial neural networks [ANN]
• CPC Class Codes: G06T 2207/30004 Biomedical image processing
• CPC Class Codes: G06T 2207/30044 Fetus; Embryo
• CPC Class Codes: G06T 7/0012 : Biomedical image inspection
• CPC Class Codes: G06T 7/0016 : involving temporal comparison
• CPC Class Codes: G06T 7/11 : Region-based segmentation
• CPC Class Codes: G06T 7/62 : of area, perimeter, diameter or volume

Abstract

Provided is a technology for extracting an image of a target plane from 2D or 3D image data acquired by a medical imaging apparatus with a small amount of computation and at high speed. A plane of a target plane including a predetermined structure is extracted from image data of a subject. A region of the predetermined structure included in the plane is detected by applying a learning model learned using learning data including a target plane for learning including an image of the structure and a region-of-interest plane for learning obtained by cutting out and enlarging a partial region including the structure in the target plane for learning to a plurality of planes obtained from the image data, and the plane of the target plane is extracted based on the detected region of the predetermined structure.

Claims

1. A medical imaging apparatus comprising:

an imaging unit that collects image data of a subject; and

an image processing unit that performs a process of extracting a plane of a target plane including a predetermined structure from the image data collected by the imaging unit,

wherein the image processing unit includes a learning model storage unit that stores a learning model learned using learning data, a structure extraction unit that detects a region of the predetermined structure included in the plane by applying the learning model to a plurality of planes obtained from the image data, and a plane extraction unit that extracts the plane of the target plane from the plurality of planes based on the detected region of the predetermined structure, and

the learning data includes a target plane for learning including an image of the structure imaged in advance for the target plane, and a region-of-interest plane for learning obtained by cutting out and enlarging a region of interest including the structure in the target plane for learning.

2. The medical imaging apparatus according to claim 1, wherein when a region of a structure included in the plane is detected by applying the learning model to the plane obtained from the image data, the structure extraction unit generates an image obtained by cutting out a region including the region of the structure from the plane and further detects a structure included in the cut-out image by applying the learning model again.

3. The medical imaging apparatus according to claim 1,

wherein the target plane includes a first structure and a second structure located inside the first structure as the structure, the target plane for learning of the learning data includes images of the first structure and the second structure, and the region-of-interest plane for learning is an image obtained by cutting out and enlarging a partial region including the first structure of the target plane for learning, and

when a region of the first structure is detected by applying the learning model to the plane obtained from the image data, the structure extraction unit generates an image obtained by cutting out a region including the region of the first structure from the plane, applies the learning model to the cut-out image, and detects a region of the second structure.

4. The medical imaging apparatus according to claim 1, wherein the learning data further includes information specifying a type of the structure and position information of a region in which the structure is located on an image, each of which is associated with the target plane for learning or the region-of-interest plane for learning including the structure.

5. The medical imaging apparatus according to claim 1, wherein the learning model is a reduced model, in which at least one of an input image size and the number of levels of a high-accuracy model learned using the target plane as learning data is reduced based on analysis of a relative size of a detection target region, relearned using the learning data including the target plane for learning and the region-of-interest plane for learning.

6. The medical imaging apparatus according to claim 5, wherein when the learning model is the reduced model in which the input image size is reduced, the structure extraction unit reduces the plane obtained from the image data to the input image size and inputs the plane to the learning model as an input image.

7. The medical imaging apparatus according to claim 1,

wherein the image processing unit further includes a total score computation unit,

the structure extraction unit further outputs a score indicating a reliability of detection for the detected region of the structure,

the total score computation unit computes a geometric score from a geometric positional relationship between regions of the structures detected by the structure extraction unit, and computes a total score using the score indicating the reliability and the geometric score, and

the plane extraction unit selects the plane, the total score of which is high, as a plane of the target plane.

8. The medical imaging apparatus according to claim 1, wherein the image processing unit further includes an automatic measurement unit that computes a measurement value determined in advance using a region of the structure of the plane selected as a plane of the target plane.

9. The medical imaging apparatus according to claim 7, wherein the image processing unit further includes a display control unit that causes a display unit to display a plane of the target plane selected by the plane extraction unit, a region of the structure, and the total score.

10. The medical imaging apparatus according to claim 7,

wherein the structure extraction unit successively extracts regions of the structure for a plurality of planes obtained from the image data,

the total score computation unit computes the total score for each of the planes, and

the plane extraction unit selects a tomographic image of the target plane by analyzing changes in the total score successively computed for the plurality of planes, and causes a display unit to display the selected tomographic image.

11. The medical imaging apparatus according to claim 1, wherein the structure extraction unit selects one of a plurality of detected regions of the structure as a region of interest.

12. The medical imaging apparatus according to claim 11, further comprising

a display control unit that causes a display unit to enlarge and display a region of the plane corresponding to the region of interest selected by the structure extraction unit.

13. The medical imaging apparatus according to claim 11,

wherein the imaging unit collects the image data by transmitting an ultrasonic beam while scanning a subject, and

the imaging unit limits a scanning range of the ultrasonic beam to a range of the region of interest selected by the structure extraction unit.

14. An image processing apparatus for performing a process of receiving image data from a subject and extracting a plane of a target plane including a predetermined structure from the image data, the image processing apparatus comprising:

a learning model storage unit that stores a learning model learned using learning data;

a structure extraction unit that detects a region of the predetermined structure included in the plane by applying the learning model to a plurality of planes obtained from the image data; and

a plane extraction unit that extracts the plane of the target plane based on the detected region of the predetermined structure,

wherein the learning data includes a target plane for learning including an image of the structure captured in advance for the target plane, and a region-of-interest plane for learning obtained by cutting out and enlarging a partial region including the structure in the target plane for learning.

15. An image processing method of extracting a plane of a target plane including a predetermined structure from image data of a subject, the image processing method comprising

detecting a region of the predetermined structure included in the plane by applying a learning model learned using learning data including a target plane for learning including an image of the structure and a region-of-interest plane for learning obtained by cutting out and enlarging a partial region including the structure in the target plane for learning to a plurality of planes obtained from the image data, and extracting the plane of the target plane based on the detected region of the predetermined structure.