Medical scan annotator system

• US 10,553,312 B2
• Filed: 10/24/2018
• Issued: 02/04/2020
• Est. Priority Date: 05/25/2017
• Status: Active Grant

• Current Assignee: Enlitic, Inc.
• Original Assignee: Enlitic, Inc.
• Inventors: Bernard, Devon, Lyman, Kevin, Yao, Li, Basham, Brian, Covington, Ben
• Primary Examiner(s): Tsai, Tsung Yin
• Application Number:
  US16/169,231
• Publication Number:
  US 20190057769A1
• Time in Patent Office: 468 Days
• CPC Class Codes: A61B 5/002 Monitoring the patient using a local or closed circuit, e.g. in a room or building A61B5/0017 takes precedence
• CPC Class Codes: A61B 5/0022 Monitoring a patient using a global network, e.g. telephone networks, internet
• CPC Class Codes: A61B 6/032 Transmission computed tomography [CT]
• CPC Class Codes: A61B 6/4233 using matrix detectors
• CPC Class Codes: A61B 6/463 characterised by displaying multiple images or images and diagnostic data on one display
• CPC Class Codes: A61B 6/468 allowing annotation or message recording
• CPC Class Codes: A61B 6/50 specially adapted for specific body parts; specially adapted for specific clinical applications
• CPC Class Codes: A61B 6/503 for diagnosis of the heart
• CPC Class Codes: A61B 6/505 for diagnosis of bone
• CPC Class Codes: A61B 6/5217 extracting a diagnostic or physiological parameter from medical diagnostic data
• CPC Class Codes: A61B 6/5288 involving retrospective matching to a physiological signal
• CPC Class Codes: A61B 6/5294 involving using additional data, e.g. patient information, image labeling, acquisition parameters
• CPC Class Codes: A61B 6/563 involving image data transmission via a network
• CPC Class Codes: A61B 8/468 allowing annotation or message recording
• CPC Class Codes: A61B 8/5223 for extracting a diagnostic or physiological parameter from medical diagnostic data for algorithms to analyse biomedical images G06T7/0012
• CPC Class Codes: A61B 8/565 involving data transmission via a network
• CPC Class Codes: G01T 1/247 Detector read-out circuitry for processing gain or off-set correction H04N
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06F 3/048 : Interaction techniques based on graphical user interfaces [GUI]
• CPC Class Codes: G06F 3/04842 : Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/0485 : Scrolling or panning
• CPC Class Codes: G06F 3/167 : Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06F 40/169 : Annotation, e.g. comment data or footnotes
• CPC Class Codes: G06F 40/197 : Version control for software G06F8/71
• CPC Class Codes: G06F 40/247 : Thesauruses; Synonyms
• CPC Class Codes: G06F 40/279 : Recognition of textual entities
• CPC Class Codes: G06F 40/30 : Semantic analysis
• CPC Class Codes: G06F 40/56 : Natural language generation
• CPC Class Codes: G06N 20/10 : using kernel methods, e.g. support vector machines [SVM]
• CPC Class Codes: G06N 3/04 : Architecture, e.g. interconnection topology
• CPC Class Codes: G06N 3/045 : Combinations of networks
• CPC Class Codes: G06N 3/084 : Backpropagation, e.g. using gradient descent
• CPC Class Codes: G06N 7/01 : Probabilistic graphical models, e.g. probabilistic networks
• CPC Class Codes: G06Q 10/10 : Office automation; Time management
• CPC Class Codes: G06Q 10/103 : Workflow collaboration or project management
• CPC Class Codes: G06Q 50/22 : Social work or social welfare, e.g. community support activities or counselling services
• CPC Class Codes: G06T 11/003 : Reconstruction from projections, e.g. tomography
• CPC Class Codes: G06T 11/60 : Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 2200/24 : involving graphical user interfaces [GUIs]
• CPC Class Codes: G06T 2207/10081 : Computed x-ray tomography [CT]
• CPC Class Codes: G06T 2207/10088 : Magnetic resonance imaging [MRI]
• CPC Class Codes: G06T 2207/10104 : Positron emission tomography [PET]
• CPC Class Codes: G06T 2207/10116 : X-ray image
• CPC Class Codes: G06T 2207/10132 : Ultrasound image
• CPC Class Codes: G06T 2207/20081 : Training; Learning
• CPC Class Codes: G06T 2207/30004 : Biomedical image processing
• CPC Class Codes: G06T 2207/30061 : Lung
• CPC Class Codes: G06T 2207/30068 : Mammography; Breast
• CPC Class Codes: G06T 7/0012 : Biomedical image inspection
• CPC Class Codes: G06T 7/0016 : involving temporal comparison
• CPC Class Codes: G06T 7/11 : Region-based segmentation
• CPC Class Codes: G06V 10/751 : Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
• CPC Class Codes: G06V 10/761 : Proximity, similarity or dissimilarity measures
• CPC Class Codes: G06V 10/764 : using classification, e.g. of video objects
• CPC Class Codes: G06V 10/98 : Detection or correction of errors, e.g. by rescanning the pattern or by human intervention; Evaluation of the quality of the acquired patterns
• CPC Class Codes: G06V 2201/03 : Recognition of patterns in medical or anatomical images
• CPC Class Codes: G16H 10/60 : for patient-specific data, e.g. for electronic patient records
• CPC Class Codes: G16H 15/00 : ICT specially adapted for medical reports, e.g. generation or transmission thereof
• CPC Class Codes: G16H 30/20 : for handling medical images, e.g. DICOM, HL7 or PACS
• CPC Class Codes: G16H 30/40 : for processing medical images, e.g. editing
• CPC Class Codes: G16H 40/20 : for the management or administration of healthcare resources or facilities, e.g. managing hospital staff or surgery rooms
• CPC Class Codes: G16H 40/63 : for local operation
• CPC Class Codes: G16H 40/67 : for remote operation
• CPC Class Codes: G16H 50/20 : for computer-aided diagnosis, e.g. based on medical expert systems
• CPC Class Codes: G16H 50/30 : for calculating health indices; for individual health risk assessment
• CPC Class Codes: G16H 50/50 : for simulation or modelling of medical disorders
• CPC Class Codes: G16H 50/70 : for mining of medical data, e.g. analysing previous cases of other patients
• CPC Class Codes: H04L 67/01 : Protocols
• CPC Class Codes: H04L 67/12 : specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
• CPC Class Codes: H04N 23/30 : for generating image signals from X-rays
• CPC Class Codes: H04N 5/32 : Transforming X-rays cameras or camera modules for generating image signals from X-rays H04N23/30; circuitry of SSIS for transforming X-rays into image signals H04N25/30

Abstract

A medical scan annotator system is operable to select a medical scan for transmission via a network to a first client device and a second client device for display via an interactive interface, and annotation data is received from the first client device and the second client device in response. Annotation similarity data is generated by comparing the first annotation data to the second annotation data, and consensus annotation data is generated based on the first annotation data and the second annotation data in response to the annotation similarity data indicating that the difference between the first annotation data and the second annotation data compares favorably to an annotation discrepancy threshold. The consensus annotation data is mapped to the medical scan in a medical scan database.

Claims

1. A medical scan annotator system, comprising:

a medical scan database that includes a plurality of medical scans;

a user database that includes a plurality of user profiles corresponding to a plurality of users of the medical scan annotator system;

a processing system that includes a processor; and

a memory that stores executable instructions that, when executed by the processing system, facilitate performance of operations comprising;

transmitting, via a network, a first medical scan from the medical scan database to a first client device associated with a first user in the user database, wherein the medical scan is displayed to the first user via a first interactive interface of a first display device corresponding to the first client device;

receiving first annotation data from the first client device via the network, wherein the first annotation data is generated by the first client device via the first interactive interface;

transmitting the first medical scan to a second client device associated with a second user in the user database via the network, wherein the medical scan is displayed to the second user via a second interactive interface of a second display device corresponding to the second client device;

receiving second annotation data from the second client device via the network, wherein the second annotation data is generated by the second client device via the second interactive interface;

generating annotation similarity data by comparing the first annotation data to the second annotation data;

generating consensus annotation data based on the first annotation data and the second annotation data in response to the annotation similarity data indicating that a difference between the first annotation data and the second annotation data compares favorably to an annotation discrepancy threshold; and

mapping the consensus annotation data to the first medical scan in the medical scan database.

2. The medical scan annotator system of claim 1, wherein the operations further comprise:

selecting a first expert user from a set of expert users in the user database in response to the annotation similarity data indicating that the difference between the first annotation data and the second annotation data compares unfavorably to the annotation discrepancy threshold;

transmitting the first medical scan to a third client device associated with the first expert user, wherein the first medical scan is displayed to the first expert user via a third interactive interface displayed by a third display device corresponding to the third client device;

receiving third annotation data from the third client device, wherein the third annotation data is generated by the third client device in response to a first prompt via the third interactive interface displayed by the third display device to provide third annotation data; and

mapping the third annotation data to the first medical scan in the medical scan database.

3. The medical scan annotator system of claim 2, wherein the operations further comprise:

transmitting the first annotation data and the second annotation data to the third client device, wherein the first annotation data and the second annotation data are displayed via the third interactive interface;

wherein the third interactive interface includes a prompt to select one of;

the first annotation data or the second annotation data, and wherein the third annotation data corresponds to the first annotation data in response to a selection of the first annotation data via the third interactive interface.

4. The medical scan annotator system of claim 2, wherein the user database stores user performance data corresponding to the plurality of users, and wherein the operations further comprise:

generating an annotation accuracy score by comparing the first annotation data to the third annotation data; and

updating the user performance data corresponding to the first user in the user database based on the annotation accuracy score.

5. The medical scan annotator system of claim 2, wherein the medical scan database includes a set of triaged medical scans and a set of corresponding priority values received from a triaging entity via the network, wherein the first medical scan is selected from the set of triaged medical scans based on a ranking of the set of corresponding priority values, and wherein the third annotation data is transmitted to the triaging entity via the network.

6. The medical scan annotator system of claim 5, wherein the user database stores user efficiency data corresponding to the plurality of users, wherein the set of triaged medical scans are assigned to a subset of the plurality of users based on the set of corresponding priority values and further based on the user efficiency data.

7. The medical scan annotator system of claim 1, wherein the user database stores user specialization data corresponding to the plurality of users, and wherein the operations further comprise:

determining a medical scan category corresponding to the first medical scan; and

selecting the first user based on determining that the user specialization data compares favorably to the medical scan category.

8. The medical scan annotator system of claim 1, wherein the user database stores user performance data corresponding to the plurality of users, and wherein the operations further comprise:

selecting a second medical scan from the medical scan database for transmission via a network to a fourth client device associated with a fourth user of the medical scan annotator system, wherein the second medical scan is displayed to the fourth user via a fourth interactive interface displayed by a fourth display device corresponding to the fourth client device;

receiving fourth annotation data from the fourth client device, wherein the fourth annotation data is generated by the fourth client device based on user input to the fourth interactive interface in response to a prompt via the fourth interactive interface displayed by the fourth display device to provide the fourth annotation data corresponding to the second medical scan;

retrieving truth annotation data mapped to the second medical scan from a medical scan database;

generating an annotation accuracy score by comparing the fourth annotation data to the truth annotation data;

updating user performance data corresponding to the fourth user in the user database based on the annotation accuracy score.

9. The medical scan annotator system of claim 8, wherein the operations further comprise:

receiving truth annotation data from a fifth client device corresponding to a second expert user; and

mapping the truth annotation data to the second medical scan.

10. The medical scan annotator system of claim 8, wherein the operations further comprise:

generating the truth annotation data by performing an automated annotating function on the second medical scan that utilizes a computer vision model, wherein the computer vision model is trained on a subset of the plurality of medical scans; and

mapping the truth annotation data to the second medical scan.

11. The medical scan annotator system of claim 8, wherein the user performance data includes specialization data that includes a plurality of category specialty scores, and wherein the operations further comprise:

determining at least one image category corresponding to the second medical scan; and

updating at least one category specialty score of the specialization data of the fourth user based on the annotation accuracy score, wherein the at least one category specialty score corresponds to the at least one image category.

12. The medical scan annotator system of claim 8, wherein the operations further comprise:

adding the fourth user to a set of expert users in the user database in response to determining that the updated user performance data compares favorably to an expert status threshold.

13. The medical scan annotator system of claim 1, wherein the first annotation data indicates one of:

that the first medical scan includes no abnormalities or that the first medical scan includes an abnormality.

14. The medical scan annotator system of claim 13, wherein the first interactive interface prompts the first user to provide abnormality classification data in response to the first annotation data indicating that the first medical scan includes an abnormality, and wherein the first annotation data further indicates the abnormality classification data.

15. The medical scan annotator system of claim 13, wherein the first interactive interface prompts the first user to provide abnormality location data in response to the first annotation data indicating that the first medical scan includes an abnormality, and wherein the first annotation data indicates the abnormality location data.

16. The medical scan annotator system of claim 1, wherein the first medical scan is one of:

a chest x-ray or a chest computed tomography (CT) scan.

17. A method for execution by a medical scan annotator system that includes a processor, the method comprising:

transmitting, via a network, a first medical scan from a medical scan database to a first client device associated with a first user in a user database, wherein the first medical scan is displayed to the first user via a first interactive interface of a first display device corresponding to the first client device;

receiving first annotation data from the first client device via the network, wherein the first annotation data is generated by the first client device via the first interactive interface;

transmitting the first medical scan to a second client device associated with a second user in the user database via the network, wherein the first medical scan is displayed to the second user via a second interactive interface of a second display device corresponding to the second client device;

receiving second annotation data from the second client device via the network, wherein the second annotation data is generated by the second client device via the second interactive interface;

generating annotation similarity data by comparing the first annotation data to the second annotation data;

generating consensus annotation data based on the first annotation data and the second annotation data in response to the annotation similarity data indicating that a difference between the first annotation data and the second annotation data compares favorably to an annotation discrepancy threshold; and

mapping the consensus annotation data to the first medical scan in the medical scan database.

18. The method of claim 17, further comprising:

selecting a first expert user from a set of expert users in the user database in response to the annotation similarity data indicating that the difference between the first annotation data and the second annotation data compares unfavorably to the annotation discrepancy threshold;

transmitting the first medical scan to a third client device associated with the first expert user, wherein the first medical scan is displayed to the first expert user via a third interactive interface displayed by a third display device corresponding to the third client device;

receiving third annotation data from the third client device, wherein the third annotation data is generated by the third client device in response to a first prompt via the third interactive interface displayed by the third display device to provide third annotation data; and

mapping the third annotation data to the first medical scan in the medical scan database.

19. The method of claim 18, further comprising:

generating an annotation accuracy score by comparing the first annotation data to the third annotation data; and

updating user performance data corresponding to the first user in the user database based on the annotation accuracy score.

20. An article of manufacture that includes a tangible storage medium that stores operational instructions, that when executed by a processor, causes the processor to:

transmit, via a network, a medical scan from a medical scan database to a first client device associated with a first user in a user database, wherein the medical scan is displayed to the first user via a first interactive interface of a first display device corresponding to the first client device;

receive first annotation data from the first client device via the network, wherein the first annotation data is generated by the first client device via the first interactive interface;

transmit the medical scan to a second client device associated with a second user in the user database via the network, wherein the medical scan is displayed to the second user via a second interactive interface of a second display device corresponding to the second client device;

receive second annotation data from the second client device via the network, wherein the second annotation data is generated by the second client device via the second interactive interface;

generate annotation similarity data by comparing the first annotation data to the second annotation data;

generate consensus annotation data based on the first annotation data and the second annotation data in response to the annotation similarity data indicating that a difference between the first annotation data and the second annotation data compares favorably to an annotation discrepancy threshold; and

map the consensus annotation data to the medical scan in the medical scan database.