Methods and Systems for Performing Image Analytics Using Graphical Reporting Associated with Clinical Images

• US 20160364862A1
• Filed: 06/10/2016
• Published: 12/15/2016
• Est. Priority Date: 06/12/2015
• Status: Active Grant

• Current Assignee: Merative US LP
• Original Assignee: Merge Healthcare Incorporated (International Business Machines Corporation)
• Inventors: Reicher, Murray A., DeVries, Jon T., Ferro, Michael W., Sati, Marwan
• Granted Patent: US 11,301,991 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: A61B 10/02 Instruments for taking cell samples or for biopsy A61B10/0038 and A61B10/0045 take precedence; needle locating or guiding means A61B17/3403; samplers for enzymology or microbiology C12M1/26; sampling or preparing biological specimens G01N33/48
• CPC Class Codes: A61B 2034/108 Computer aided selection or customisation of medical implants or cutting guides
• CPC Class Codes: A61B 2090/373 using light, e.g. by using optical scanners
• CPC Class Codes: A61B 2090/374 NMR or MRI
• CPC Class Codes: A61B 2090/3762 using computed tomography systems [CT]
• CPC Class Codes: A61B 2090/378 using ultrasound
• CPC Class Codes: A61B 34/10 Computer-aided planning, simulation or modelling of surgical operations
• CPC Class Codes: A61B 5/0077 Devices for viewing the surface of the body, e.g. camera, magnifying lens
• CPC Class Codes: A61B 5/015 By temperature mapping of body part
• CPC Class Codes: A61B 5/055 involving electronic [EMR] or nuclear [NMR] magnetic resonance, e.g. magnetic resonance imaging
• CPC Class Codes: A61B 5/318 Heart-related electrical modalities, e.g. electrocardiography [ECG]
• CPC Class Codes: A61B 5/4312 Breast evaluation or disorder diagnosis A61B5/0091 takes precedence
• CPC Class Codes: A61B 5/441 Skin evaluation, e.g. for skin disorder diagnosis
• CPC Class Codes: A61B 5/7267 involving training the classification device
• CPC Class Codes: G06F 18/213 Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06F 18/217 Validation; Performance evaluation; Active pattern learning techniques
• CPC Class Codes: G06F 18/2411 based on the proximity to a decision surface, e.g. support vector machines
• CPC Class Codes: G06F 18/2413 based on distances to training or reference patterns
• CPC Class Codes: G06F 18/285 Selection of pattern recognition techniques, e.g. of classifiers in a multi-classifier system
• CPC Class Codes: G06F 3/0482 : Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/0488 : using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 40/169 : Annotation, e.g. comment data or footnotes
• CPC Class Codes: G06F 40/205 : Parsing
• CPC Class Codes: G06N 20/00 : Machine learning
• CPC Class Codes: G06T 11/008 : Specific post-processing after tomographic reconstruction, e.g. voxelisation, metal artifact correction
• CPC Class Codes: G06T 2200/24 : involving graphical user interfaces [GUIs]
• CPC Class Codes: G06T 2207/10088 : Magnetic resonance imaging [MRI]
• CPC Class Codes: G06T 2207/20081 : Training; Learning
• CPC Class Codes: G06T 2207/30004 : Biomedical image processing
• CPC Class Codes: G06T 2207/30008 : Bone
• CPC Class Codes: G06T 2207/30016 : Brain
• CPC Class Codes: G06T 2207/30052 : Implant; Prosthesis
• CPC Class Codes: G06T 2207/30061 : Lung
• CPC Class Codes: G06T 2207/30068 : Mammography; Breast
• CPC Class Codes: G06T 2207/30088 : Skin; Dermal
• CPC Class Codes: G06T 7/0012 : Biomedical image inspection
• CPC Class Codes: G06T 7/0014 : using an image reference approach
• CPC Class Codes: G06V 10/764 : using classification, e.g. of video objects
• CPC Class Codes: G06V 2201/031 : of internal organs
• CPC Class Codes: G16H 10/20 : for electronic clinical trials or questionnaires
• CPC Class Codes: G16H 15/00 : ICT specially adapted for medical reports, e.g. generation or transmission thereof
• CPC Class Codes: G16H 20/10 : relating to drugs or medications, e.g. for ensuring correct administration to patients
• CPC Class Codes: G16H 30/20 : for handling medical images, e.g. DICOM, HL7 or PACS
• CPC Class Codes: G16H 50/20 : for computer-aided diagnosis, e.g. based on medical expert systems
• CPC Class Codes: G16H 50/50 : for simulation or modelling of medical disorders
• CPC Class Codes: G16Z 99/00 : Subject matter not provided for in other main groups of this subclass

Abstract

Methods and systems for performing image analytics using graphical reporting associated with clinical images. One system includes at least one data source and a server. The server includes an electronic processor and an interface for communicating with the data source. The electronic processor is configured to receive training information from the at least one data source over the interface. The training information includes a plurality of images and graphical reporting associated with each of the plurality of images. Each graphical reporting includes a graphical marker designating a portion of one of the plurality of images and diagnostic information associated with the portion of the one of the plurality of images. The electronic processor is also configured to perform machine learning to develop a model using the training information. The model is used to automatically analyze an image.

Claims

1. A system for performing image analytics using graphical reporting associated with clinical images, the system comprising:

at least one data source; and

a server including an electronic processor and an interface for communicating with the data source the electronic processor configured to

receive training information from the at least one data source over the interface, the training information including a plurality of images and graphical reporting associated with each of the plurality of images, each graphical reporting including a graphical marker designating a portion of one of the plurality of images and diagnostic information associated with the portion of the one of the plurality of images, and

perform machine learning to develop a model using the training information, wherein the model is used to automatically analyze an image.

2. The system of claim 1, wherein the graphical marker includes a one-dimensional graphical marker, a two-dimensional graphical marker, or a three-dimensional graphical marker.

3. The system of claim 1, wherein the graphical marker was manually generated by a diagnosing physician.

4. The system of claim 1, wherein the graphical marker was automatically generated by a computer system.

5. The system of claim 1, wherein the diagnostic information includes a classification of a plurality of categories.

6. The system of claim 1, wherein the diagnostic information includes a normal category, an abnormal category, and an indeterminate category.

7. The system of claim 1, wherein the diagnostic information includes a probability.

8. The system of claim 1, wherein the diagnostic information includes a diagnosis.

9. The system of claim 1, wherein the diagnostic information includes an anatomical structure.

10. The system of claim 1, wherein the diagnostic information includes a measurement or a location.

11. The system of claim 1, wherein the diagnostic information includes morphological characteristics of an anomaly within the portion.

12. The system of claim 1, wherein the training information further includes a Digital Imaging and Communications in Medicine (DICOM) header file associated with at least one of the plurality of images.

13. The system of claim 1, wherein the training information further includes procedure information associated with at least one of the plurality of images.

14. The system of claim 1, wherein the training information further includes patient information associated with at least one of the plurality of images.

15. The system of claim 14, wherein the electronic processor is configured to receive the patient information from a hospital information system (HIS) or an electronic medical record (EMR).

16. The system of claim 1, wherein the training information further includes a report associated with at least one of the plurality of images.

17. The system of claim 16, wherein the report includes an order, a DICOM structured radiology report, a pathology report, or a test result.

18. The system of claim 1, wherein the training information further includes an imaging property of a first image included in the plurality of images, the imaging property including at least one selected from a group consisting of a first indication of whether a contrast agent was used, a second indication of whether a radioactive isotope was used, a time after an agent or isotope was introduced, an orientation, and an image acquisition parameter.

19. The system of claim 1, wherein the at least one data source includes a picture archiving and communications system (PACS), a vendor neutral archive, a radiology information system (RIS), an electronic medical record (EMR), a hospital information system (HIS), an image study ordering system, or a computer assisted detection (CAD) system.

20. The system of claim 1, wherein the electronic processor is configured to perform the machine learning to develop the model using the training information by evaluating a first portion designated by a first graphical marker within a first image included in the plurality of images on a second image included in the plurality of images, the second image acquired under a different imaging condition than the first image.

21. The system of claim 20, wherein the different imaging condition includes at least one selected from a group consisting of a different image within the same exam, a different exam, a different time after injection of an agent or isotope, a different magnetic resonance imaging (MRI) acquisition parameter, and a different radiation parameter.

22. The system of claim 1, wherein the electronic processor is configured to perform the machine learning to develop the model using the training information by comparing an anatomical position associated with a first image included in the plurality of images with an anatomical position in a second plurality of images acquired using different image acquisition properties than the first image.

23. The system of claim 1, wherein the electronic processor is configured to perform the machine learning to develop the model using the training information by obtaining a clinical result associated with a first image included in the plurality of images,

performing a comparison of the clinical result and the diagnostic information associated with the first image,

assigning a weight to the diagnostic information based on the comparison, and

developing the model using the training information based on the weight.

24. The system of claim 23, wherein the clinical result includes at least one image.

25. The system of claim 23, wherein the clinical result includes at least one laboratory result.

26. The system of claim 1, further comprising updating the model based on a clinical result associated with a first image included in the plurality of images.

27. The system of claim 26, wherein the clinical result includes at least one image.

28. The system of claim 26, wherein the clinical result include a laboratory result.

29. The system of claim 1, further comprising updating the model based on feedback designating a correctness of the training information.

30. The system of claim 1, wherein the training information includes a data record specifying coordinates of a first graphical marker of a first image included in the plurality of images.

31. The system of claim 1, wherein the training information includes a first image included in the plurality of images, wherein a first graphical marker is superimposed on the first image and wherein the electronic processor is configured to automatically detect the first graphical marker within the first image.