Medical scan image analysis system

• US 10,340,044 B2
• Filed: 08/30/2017
• Issued: 07/02/2019
• Est. Priority Date: 05/25/2017
• Status: Active Grant

• Current Assignee: Enlitic, Inc.
• Original Assignee: Enlitic, Inc.
• Inventors: Yao, Li, Bernard, Devon, Lyman, Kevin, Almeida, Diogo, Howard, Jeremy
• Primary Examiner(s): Woldemariam, Aklilu K
• Application Number:
  US15/690,752
• Publication Number:
  US 20180342060A1
• Time in Patent Office: 671 Days
• Field of Search: 382103, 382128, 382129, 382130, 382131, 382132, 706 12, 706 25, 706 47, 706 59
• CPC Class Codes: A61B 5/002 Monitoring the patient using a local or closed circuit, e.g. in a room or building A61B5/0017 takes precedence
• CPC Class Codes: A61B 5/0022 Monitoring a patient using a global network, e.g. telephone networks, internet
• CPC Class Codes: A61B 6/032 Transmission computed tomography [CT]
• CPC Class Codes: A61B 6/4233 using matrix detectors
• CPC Class Codes: A61B 6/463 characterised by displaying multiple images or images and diagnostic data on one display
• CPC Class Codes: A61B 6/468 allowing annotation or message recording
• CPC Class Codes: A61B 6/50 specially adapted for specific body parts; specially adapted for specific clinical applications
• CPC Class Codes: A61B 6/503 for diagnosis of the heart
• CPC Class Codes: A61B 6/505 for diagnosis of bone
• CPC Class Codes: A61B 6/5217 extracting a diagnostic or physiological parameter from medical diagnostic data
• CPC Class Codes: A61B 6/5288 involving retrospective matching to a physiological signal
• CPC Class Codes: A61B 6/5294 involving using additional data, e.g. patient information, image labeling, acquisition parameters
• CPC Class Codes: A61B 6/563 involving image data transmission via a network
• CPC Class Codes: A61B 8/468 allowing annotation or message recording
• CPC Class Codes: A61B 8/5223 for extracting a diagnostic or physiological parameter from medical diagnostic data for algorithms to analyse biomedical images G06T7/0012
• CPC Class Codes: A61B 8/565 involving data transmission via a network
• CPC Class Codes: G01T 1/247 Detector read-out circuitry for processing gain or off-set correction H04N
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06F 3/048 : Interaction techniques based on graphical user interfaces [GUI]
• CPC Class Codes: G06F 3/04842 : Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/0485 : Scrolling or panning
• CPC Class Codes: G06F 3/167 : Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06F 40/169 : Annotation, e.g. comment data or footnotes
• CPC Class Codes: G06F 40/197 : Version control for software G06F8/71
• CPC Class Codes: G06F 40/247 : Thesauruses; Synonyms
• CPC Class Codes: G06F 40/279 : Recognition of textual entities
• CPC Class Codes: G06F 40/30 : Semantic analysis
• CPC Class Codes: G06F 40/56 : Natural language generation
• CPC Class Codes: G06N 20/10 : using kernel methods, e.g. support vector machines [SVM]
• CPC Class Codes: G06N 3/04 : Architecture, e.g. interconnection topology
• CPC Class Codes: G06N 3/045 : Combinations of networks
• CPC Class Codes: G06N 3/084 : Backpropagation, e.g. using gradient descent
• CPC Class Codes: G06N 7/01 : Probabilistic graphical models, e.g. probabilistic networks
• CPC Class Codes: G06Q 10/10 : Office automation; Time management
• CPC Class Codes: G06Q 10/103 : Workflow collaboration or project management
• CPC Class Codes: G06Q 50/22 : Social work or social welfare, e.g. community support activities or counselling services
• CPC Class Codes: G06T 11/003 : Reconstruction from projections, e.g. tomography
• CPC Class Codes: G06T 11/60 : Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 2200/24 : involving graphical user interfaces [GUIs]
• CPC Class Codes: G06T 2207/10081 : Computed x-ray tomography [CT]
• CPC Class Codes: G06T 2207/10088 : Magnetic resonance imaging [MRI]
• CPC Class Codes: G06T 2207/10104 : Positron emission tomography [PET]
• CPC Class Codes: G06T 2207/10116 : X-ray image
• CPC Class Codes: G06T 2207/10132 : Ultrasound image
• CPC Class Codes: G06T 2207/20081 : Training; Learning
• CPC Class Codes: G06T 2207/30004 : Biomedical image processing
• CPC Class Codes: G06T 2207/30061 : Lung
• CPC Class Codes: G06T 2207/30068 : Mammography; Breast
• CPC Class Codes: G06T 7/0012 : Biomedical image inspection
• CPC Class Codes: G06T 7/0016 : involving temporal comparison
• CPC Class Codes: G06T 7/11 : Region-based segmentation
• CPC Class Codes: G06V 10/751 : Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
• CPC Class Codes: G06V 10/761 : Proximity, similarity or dissimilarity measures
• CPC Class Codes: G06V 10/764 : using classification, e.g. of video objects
• CPC Class Codes: G06V 10/98 : Detection or correction of errors, e.g. by rescanning the pattern or by human intervention; Evaluation of the quality of the acquired patterns
• CPC Class Codes: G06V 2201/03 : Recognition of patterns in medical or anatomical images
• CPC Class Codes: G16H 10/60 : for patient-specific data, e.g. for electronic patient records
• CPC Class Codes: G16H 15/00 : ICT specially adapted for medical reports, e.g. generation or transmission thereof
• CPC Class Codes: G16H 30/20 : for handling medical images, e.g. DICOM, HL7 or PACS
• CPC Class Codes: G16H 30/40 : for processing medical images, e.g. editing
• CPC Class Codes: G16H 40/20 : for the management or administration of healthcare resources or facilities, e.g. managing hospital staff or surgery rooms
• CPC Class Codes: G16H 40/63 : for local operation
• CPC Class Codes: G16H 40/67 : for remote operation
• CPC Class Codes: G16H 50/20 : for computer-aided diagnosis, e.g. based on medical expert systems
• CPC Class Codes: G16H 50/30 : for calculating health indices; for individual health risk assessment
• CPC Class Codes: G16H 50/50 : for simulation or modelling of medical disorders
• CPC Class Codes: G16H 50/70 : for mining of medical data, e.g. analysing previous cases of other patients
• CPC Class Codes: H04L 67/01 : Protocols
• CPC Class Codes: H04L 67/12 : specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
• CPC Class Codes: H04N 5/32 : Transforming X-rays cameras or camera modules for generating image signals from X-rays H04N23/30; circuitry of SSIS for transforming X-rays into image signals H04N25/30

Abstract

A medical scan image analysis system is operable to receive a plurality of medical scans that represent a three-dimensional anatomical region and include a plurality of cross-sectional image slices. A plurality of three-dimensional subregions corresponding to each of the plurality of medical scans are generated by selecting a proper subset of the plurality of cross-sectional image slices from each medical scan, and by further selecting a two-dimensional subregion from each proper subset of cross-sectional image slices. A learning algorithm is performed on the plurality of three-dimensional subregions to generate a fully convolutional neural network. Inference data corresponding to a new medical scan received via the network is generated by performing an inference algorithm on the new medical scan by utilizing the fully convolutional neural network. An inferred abnormality is identified in the new medical scan based on the inference data.

Claims

1. A medical scan image analysis system, comprising:

a processing system that includes a processor; and

a memory that stores executable instructions that, when executed by the processing system, facilitate performance of operations comprising;

receiving a plurality of medical scans via a network, wherein each of the plurality of medical scans represents a three-dimensional anatomical region and includes a plurality of cross-sectional image slices;

generating a plurality of three-dimensional subregions corresponding to each of the plurality of medical scans by selecting a proper subset of the plurality of cross-sectional image slices from each of the plurality of medical scans, and by further selecting a two-dimensional subregion from each proper subset of the plurality of cross-sectional image slices of the each of the medical scans;

performing a learning algorithm on the plurality of three-dimensional subregions to generate a fully convolutional neural network;

receiving a new medical scan that is not included in the plurality of medical scans;

generating inference data corresponding to the new medical scan by performing a first inference algorithm on the new medical scan by utilizing the fully convolutional neural network; and

identifying an inferred abnormality in the new medical scan based on the inference data;

wherein performing the learning algorithm includes utilizing a forward propagation algorithm on the plurality of three-dimensional subregions to generate a preliminary set of neural network parameters, and by utilizing a back propagation algorithm to generate an updated set of neural network parameters based on a calculated set of parameter errors and the preliminary set of neural network parameters, and wherein performing the first inference algorithm includes utilizing the forward propagation algorithm on the new medical scan based on the updated set of neural network parameters.

2. The medical scan image analysis system of claim 1, wherein at least one of the plurality of medical scans includes a known abnormality, wherein selecting the proper subset of the plurality of cross-sectional image slices of the at least one of the plurality of medical scans is based on a known location of the known abnormality, and wherein selecting the two-dimensional subregion from each proper subset of the plurality of cross-sectional image slices is based on the known location of the known abnormality.

3. The medical scan image analysis system of claim 2, wherein each of the plurality of three-dimensional subregions of the at least one of the plurality of medical scans include the known abnormality.

4. The medical scan image analysis system of claim 2, wherein the each of the plurality of three-dimensional subregions are one of:

randomly or pseudo-randomly selected, wherein a first subregion option corresponding to a one of the plurality of medical scans has a first selection probability based on a first proximity of the first subregion option to the known location of the known abnormality, wherein a second subregion option corresponding to the one of the plurality of medical scans has a second selection probability based on a second proximity of the first subregion option to the known location of the known abnormality, and wherein the first selection probability is higher than the second selection probability based on the first proximity being nearer than the second proximity.

5. The medical scan image analysis system of claim 1, wherein a first number of pixels in a one three-dimensional subregions of the plurality of three-dimensional subregions is smaller than a second number of pixels in a corresponding one of the plurality of medical scans.

6. The medical scan image analysis system of claim 1, wherein generating the inference data includes generating a plurality of abnormality probabilities, wherein each of the plurality of abnormality probabilities is mapped to a pixel location of each of a plurality of cross-sectional image slices of the new medical scan, and wherein identifying the inferred abnormality is based on the plurality of abnormality probabilities.

7. The medical scan image analysis system of claim 1, wherein the operations further comprise:

generating classification data corresponding to the inferred abnormality based on the inference data.

8. The medical scan image analysis system of claim 7, wherein generating the classification data includes generating a new three-dimensional subregion of the new medical scan that includes the inferred abnormality and by performing a second inference algorithm on the new three-dimensional subregion.

9. The medical scan image analysis system of claim 1, wherein the operations further comprise:

determining a density window for the plurality of medical scans by determining a low end density cut-off value and high end density cut-off value based on Hounsfield units of the each of the plurality of medical scans;

generating a plurality of preprocessed medical scans by utilizing the density window to mask a first subset of pixels of the plurality of medical scans;

wherein the plurality of three-dimensional subregions are generated by utilizing the plurality of preprocessed medical scans.

10. The medical scan image analysis system of claim 9, wherein the operations further comprise:

generating an updated low end density cut-off value and an updated high end density cut-off value in a first iteration of the learning algorithm;

generating a plurality of processed medical scans by utilizing the updated low end density cut-off value and the updated high end density cut-off value to mask a second subset of pixels of the plurality of medical scans; and

performing a subsequent iteration of the learning algorithm on the plurality of processed medical scans to generate the fully convolutional neural network.

11. The medical scan image analysis system of claim 1, wherein the operations further comprise:

generating padded data for each of the plurality of three-dimensional subregions based on a data reflection at a plurality of boundaries of each of the plurality of three-dimensional subregions;

wherein performing the learning algorithm includes convolving each of the plurality of three-dimensional subregions, and wherein the padded data is utilized to convolve each of the plurality of three-dimensional subregions at the plurality of boundaries.

12. A method for execution by a medical scan image analysis system that includes a processor, the method comprising:

receiving a plurality of medical scans via a network, wherein each of the plurality of medical scans represents a three-dimensional anatomical region and includes a plurality of cross-sectional image slices;

determining a density window for the plurality of medical scans by determining a low end density cut-off value and high end density cut-off value based on Hounsfield units of the each of the plurality of medical scans;

generating a plurality of preprocessed medical scans by utilizing the density window to mask a first subset of pixels of the plurality of medical scans;

generating a plurality of three-dimensional subregions corresponding to each of the plurality of medical scans by selecting a proper subset of the plurality of cross-sectional image slices from each of the plurality of medical scans, and by further selecting a two-dimensional subregion from each proper subset of the plurality of cross-sectional image slices of the each of the medical scans, wherein the plurality of three-dimensional subregions are generated by utilizing the plurality of preprocessed medical scans;

performing a learning algorithm on the plurality of three-dimensional subregions to generate a fully convolutional neural network;

receiving a new medical scan that is not included in the plurality of medical scans;

generating inference data corresponding to the new medical scan by performing a first inference algorithm on the new medical scan by utilizing the fully convolutional neural network; and

identifying an inferred abnormality in the new medical scan based on the inference data.

13. The method of claim 12, wherein performing the learning algorithm includes utilizing a forward propagation algorithm on the plurality of three-dimensional subregions to generate a preliminary set of neural network parameters, and by utilizing a back propagation algorithm to generate an updated set of neural network parameters based on a calculated set of parameter errors and the preliminary set of neural network parameters, and wherein performing the first inference algorithm includes utilizing the forward propagation algorithm on the new medical scan based on the updated set of neural network parameters.

14. The method of claim 12, wherein at least one of the plurality of medical scans includes a known abnormality, wherein selecting the proper subset of the plurality of cross-sectional image slices of the at least one of the plurality of medical scans is based on a known location of the known abnormality, and wherein selecting the two-dimensional subregion from each proper subset of the plurality of cross-sectional image slices is based on the known location of the known abnormality.

15. The method of claim 14, wherein each of the plurality of three-dimensional subregions of the at least one of the plurality of medical scans include the known abnormality.

16. The method of claim 14, wherein the each of the plurality of three-dimensional subregions are selected randomly or pseudo-randomly, wherein a first subregion option corresponding to a one of the plurality of medical scans has a first selection probability based on a first proximity of the first subregion option to the known location of the known abnormality, wherein a second subregion option corresponding to the one of the plurality of medical scans has a second selection probability based on a second proximity of the first subregion option to the known location of the known abnormality, and wherein the first selection probability is higher than the second selection probability based on the first proximity being nearer than the second proximity.

17. The method of claim 12, wherein generating the inference data includes generating a plurality of abnormality probabilities, wherein each of the plurality of abnormality probabilities is mapped to a pixel location of each of a plurality of cross-sectional image slices of the new medical scan, and wherein identifying the inferred abnormality is based on the plurality of abnormality probabilities.

18. An article of manufacture that includes a tangible storage medium that stores operational instructions, that when executed by a processor, causes the processor to:

receive a plurality of medical scans via a network, wherein each of the plurality of medical scans represents a three-dimensional anatomical region and includes a plurality of cross-sectional image slices;

generate a plurality of three-dimensional subregions corresponding to each of the plurality of medical scans by selecting a proper subset of the plurality of cross-sectional image slices from each of the plurality of medical scans, and by further selecting a two-dimensional subregion from each proper subset of the plurality of cross-sectional image slices of the each of the medical scans;

perform a learning algorithm on the plurality of three-dimensional subregions to generate a fully convolutional neural network;

receive a new medical scan that is not included in the plurality of medical scans;

generate inference data corresponding to the new medical scan by performing a first inference algorithm on the new medical scan by utilizing the fully convolutional neural network; and

identify an inferred abnormality in the new medical scan based on the inference data;

wherein performing the learning algorithm includes utilizing a forward propagation algorithm on the plurality of three-dimensional subregions to generate a preliminary set of neural network parameters, and by utilizing a back propagation algorithm to generate an updated set of neural network parameters based on a calculated set of parameter errors and the preliminary set of neural network parameters, and wherein performing the first inference algorithm includes utilizing the forward propagation algorithm on the new medical scan based on the updated set of neural network parameters.