AUTOMATIC GENERATION OF RADIOLOGY REPORTS FROM IMAGES AND AUTOMATIC RULE OUT OF IMAGES WITHOUT FINDINGS

• US 20170337329A1
• Filed: 05/18/2016
• Published: 11/23/2017
• Est. Priority Date: 05/18/2016
• Status: Abandoned Application

• Current Assignee: Siemens Healthcare GMBH (Siemens AG)
• Original Assignee: Siemens Healthcare GMBH (Siemens AG)
• Inventors: Liu, Wen P., Georgescu, Bogdan, Zhou, Shaohua Kevin, Comaniciu, Dorin
• Application Number:
  US15/158,375
• Publication Number:
  US 20170337329A1
• Time in Patent Office: Days
• CPC Class Codes: A61B 6/032 Transmission computed tomography [CT]
• CPC Class Codes: A61B 6/4441 the rigid structure being a C-arm or U-arm
• CPC Class Codes: A61B 6/463 characterised by displaying multiple images or images and diagnostic data on one display
• CPC Class Codes: A61B 6/545 involving automatic set-up of acquisition parameters
• CPC Class Codes: G06F 40/169 Annotation, e.g. comment data or footnotes
• CPC Class Codes: G06F 40/186 Templates
• CPC Class Codes: G06F 40/40 Processing or translation of natural language natural language analysis G06F40/20; semantic analysis G06F40/30
• CPC Class Codes: G06F 40/56 Natural language generation
• CPC Class Codes: G06Q 10/10 Office automation; Time management
• CPC Class Codes: G16H 10/60 for patient-specific data, e.g. for electronic patient records
• CPC Class Codes: G16H 15/00 ICT specially adapted for medical reports, e.g. generation or transmission thereof
• CPC Class Codes: G16H 30/40 for processing medical images, e.g. editing
• CPC Class Codes: G16H 40/63 for local operation
• CPC Class Codes: G16H 50/20 for computer-aided diagnosis, e.g. based on medical expert systems

Abstract

A computer-implemented method for automatically generating a radiology report includes a computer receiving an input dataset comprising a plurality of multidimensional patient images and patient information and parsing the input dataset using learned models to determine a clinical domain and relevant image annotations. The computer populates an annotation table using the relevant image annotations and applies one or more domain-specific scriptable rules to populate a report template based on the annotation table. The computer may then generate a natural language radiology report based on the report template.

Claims

1. A computer-implemented method for automatically generating a radiology report, the method comprising:

receiving, by a computer, an input dataset comprising a plurality of multidimensional patient images and patient information;

parsing, by the computer, the input dataset using learned models to determine a clinical domain and relevant image annotations;

populating, by the computer, an annotation table using the relevant image annotations;

applying, by the computer, one or more domain-specific scriptable rules to populate a report template based on the annotation table; and

generating, by the computer, a natural language radiology report based on the report template.

2. The method of claim 1, further comprising:

receiving, by the computer, an indication of a clinical study being performed on the input dataset,

wherein the natural language radiology report provides an explanation of a clinical finding relevant to the clinical study and one or more image features corresponding to the clinical finding.

3. The method of claim 2, further comprising:

presenting the natural language radiology report in an interactive graphical user interface which allows a user to retrieve images depicting the one or more image features via activation of one or more links embedded in the natural language radiology report.

4. The method of claim 1, wherein the natural language radiology report comprises one or more recommendations for modifying a scanner acquisition protocol to acquire one or more additional patient images.

5. The method of claim 4, further comprising:

receiving, by the computer, an indication of a clinical study being performed on the plurality of multidimensional patient images;

detecting, by the computer, that target anatomy relevant to the clinical study is partially or completely out of the field of view of all of the plurality of multidimensional patient images,

wherein the one or more recommendations for modifying the scanner acquisition protocol comprise a recommended modification to patient positioning during imaging.

6. The method of claim 1, wherein a rule-out process is applied to the plurality of multidimensional patient images prior parsing the plurality of multidimensional patient images using the learned models, the rule-out process comprising:

receiving, by the computer, an indication of a clinical study being performed on the plurality of multidimensional patient images;

identifying, by the computer, a subset of the plurality of multidimensional patient images which are irrelevant to the clinical study; and

disregarding, by the computer, the subset of the plurality of multidimensional patient images from the input dataset or the input dataset as a whole.

7. The method of claim 1, further comprising:

performing an offline preparation process comprising;

creating a clinical report template based on existing clinical reports and domain knowledge;

identifying one or more clinical report concepts and acceptable data ranges relevant to the clinical report concepts based on the existing clinical reports and the domain knowledge;

using the clinical report template, the one or more clinical report concepts, and the acceptable data ranges relevant to the clinical report concepts to create an annotation specification comprising one or more annotation tables and the one or more domain-specific scriptable rules;

using the annotation specification to create an annotation system;

applying the annotation system to one or more training images to yield one or more training image annotations; and

training one or more image parsing models based on the training images and the training image annotations.

8. The method of claim 7, wherein the domain knowledge comprises one or more of clinical standards, clinical guidelines, or information provided in clinical consults.

9. The method of claim 7, further comprising:

using a basic report template provided in a Radiological Society of North America standardized format to create the clinical report template based on the existing clinical reports and domain knowledge.

10. A computer-implemented method for automatically generating a radiology report, the method comprising:

performing, by a computer, an offline training process comprising;

determining a plurality of possible image annotations associated with a clinical domain;

determining a plurality of scriptable rules for populating a domain-specific clinical report template with information relevant to the plurality of possible image annotations;

training one or more domain specific models to parse image information and output one or more of the plurality of possible image annotations; and

performing, by the computer, an online report generation process comprising;

receiving an input dataset comprising a plurality of multidimensional patient images and patient information;

deriving one or more relevant image annotations associated with the input dataset based on the plurality of possible image annotations associated with the clinical domain and the one or more domain specific models;

populating the domain-specific clinical report template using the one or more relevant image annotations and the plurality of scriptable rules; and

identifying one or more clinically relevant findings based on the populated domain-specific clinical report template.

11. The method of claim 10, wherein the online report generation process further comprises:

generating a natural language radiology report based on the one or more clinically relevant findings.

12. The method of claim 11, wherein the online report generation process further comprises:

presenting the natural language radiology report in an interactive graphical user interface which allows a user to retrieve images depicting image features relevant to the clinically relevant findings via activation of one or more links embedded in the natural language radiology report.

13. The method of claim 10, wherein the online report generation process further comprises:

identifying a change to an existing image acquisition protocol based on the one or more clinically relevant findings.

14. The method of claim 13, wherein the online report generation process further comprises:

automatically implementing the change to the existing image acquisition protocol on an image scanner to acquire one or more new images.

15. The method of claim 13, wherein the online report generation process further comprises:

displaying the change to the existing image acquisition protocol in a graphical user interface as a recommendation to a user.

16. The method of claim 15, wherein the online report generation process further comprises:

detecting, by the computer, that target anatomy relevant to the clinical domain is partially or completely out of the field of view of all of the plurality of multidimensional patient images,

wherein the change to the existing image acquisition protocol comprises a recommended modification to patient positioning during imaging.

17. The method of claim 13, wherein the online report generation process further comprises:

identifying, by the computer, a subset of the plurality of multidimensional patient images which are irrelevant to the clinical domain based on the one or more clinically relevant findings; and

disregarding, by the computer, the subset of the plurality of multidimensional patient images from the input dataset or the input dataset as a whole.

18. A system for automatically generating a radiology report, the system comprising:

a medical information database comprising one or more diagnostic multidimensional (e.g. 2D/3D/4D) image data and non-image patient metadata;

one or more processors configured to;

communicate with the medical information database to retrieve a patient-specific input dataset;

parse the patient-specific input dataset using learned models to determine a clinical domain and relevant image annotations;

populate an annotation table using the relevant image annotations;

apply one or more domain-specific scriptable rules to populate a report template based on the annotation table; and

identify one or more clinically relevant findings based on the populated report template.

19. The system of claim 18, wherein the one or more processors are further configured to:

generate a natural language radiology report based on the report template.

20. The system of claim 19, wherein the one or more processors are further configured to:

present the natural language radiology report in an interactive graphical user interface which allows a user to retrieve images depicting image features relevant to the clinically relevant findings via activation of one or more links embedded in the natural language radiology report.