Device to enhance and present medical image using corrective mechanism

• US 10,453,570 B1
• Filed: 11/14/2018
• Issued: 10/22/2019
• Est. Priority Date: 06/04/2018
• Status: Active Grant

• Current Assignee: Rutgers University
• Original Assignee: Sonavista, Inc.
• Inventors: Podilchuk, Christine I., Mammone, Richard
• Primary Examiner(s): Sherali, Ishrat I
• Application Number:
  US16/190,598
• Time in Patent Office: 342 Days
• Field of Search: 382103, 382128, 382254, 382275, 382155
• CPC Class Codes: G06T 2207/10004 Still image; Photographic image
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/10081 Computed x-ray tomography [CT]
• CPC Class Codes: G06T 2207/10088 Magnetic resonance imaging [MRI]
• CPC Class Codes: G06T 2207/10104 Positron emission tomography [PET]
• CPC Class Codes: G06T 2207/10116 X-ray image
• CPC Class Codes: G06T 2207/10132 Ultrasound image
• CPC Class Codes: G06T 2207/20076 Probabilistic image processing
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/20084 Artificial neural networks [ANN]
• CPC Class Codes: G06T 2207/20104 Interactive definition of region of interest [ROI]
• CPC Class Codes: G06T 2207/20201 Motion blur correction
• CPC Class Codes: G06T 2207/30168 Image quality inspection
• CPC Class Codes: G06T 5/60 using machine learning, e.g. neural networks
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening
• CPC Class Codes: G06T 5/80 Geometric correction
• CPC Class Codes: G06T 7/0012 Biomedical image inspection
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: G06V 2201/03 Recognition of patterns in medical or anatomical images
• CPC Class Codes: G16H 30/20 : for handling medical images, e.g. DICOM, HL7 or PACS
• CPC Class Codes: G16H 30/40 : for processing medical images, e.g. editing
• CPC Class Codes: G16H 40/63 : for local operation
• CPC Class Codes: G16H 50/70 : for mining of medical data, e.g. analysing previous cases of other patients

Abstract

A device to enhance and present a medical image using a corrective mechanism is described. An image analysis application executed by the device captures a digital copy of the medical image displayed on a display device. A flawed photography effect associated with the digital copy is identified by processing the digital copy. Next, the digital copy is enhanced based on the flawed photography effect. Furthermore, the enhanced digital copy can be processed with an artificial intelligence mechanism to generate an annotation. The annotation is associated with a cancer identification. In addition, the enhanced digital copy and the annotation are displayed.

Claims

1. A device to enhance and present a medical image using a corrective mechanism, wherein the device performs one or more operations comprising:

receiving a digital copy from a camera, wherein the camera captured the digital copy of the medical image displayed on a monitor, and wherein the monitor is external to the camera;

identifying a flawed photography effect associated with the digital copy by processing the digital copy using a deep learning mechanism performed by a computer analysis and correction module (CACM);

enhancing the digital copy using the deep learning mechanism performed by the CACM based on the flawed photography effect;

processing the enhanced digital copy using the deep learning mechanism performed by the CACM to generate an annotation, wherein the annotation is associated with a cancer identification; and

displaying one or more of the enhanced digital copy and the annotation overlaid by an augmented reality mechanism on to the monitor displaying the medical image.

2. The device of claim 1, wherein

the camera includes a polarizer to reduce one or more of a reflection effect and a glare effect associated with capturing the digital copy of the medical image.

3. The device of claim 1, wherein the device performs one or more additional operations comprising:

analyzing one or more non-image information associated with the digital copy and the camera using the deep learning mechanism performed by the CACM, wherein the non-image information includes one or more of a flash information, a focal length, a shutter speed, a camera model information, an aperture setting, and a capture date and time information;

identifying the flawed digital effect from the one or more analyzed non-image information using the deep learning mechanism performed by the CACM; and

enhancing the digital copy to correct the flawed digital effect using the deep learning mechanism performed by the CACM.

4. The device of claim 1, wherein the device performs one or more additional operations comprising:

analyzing one or more image characteristic information associated with the digital copy using the deep learning mechanism performed by the CACM, wherein the one or more characteristic information includes an orientation, a red-eye detection, a blur, a color balance, an exposure, and a noise information;

identifying the flawed digital effect from the one or more analyzed characteristic information using the deep learning mechanism performed by the CACM; and

enhancing the digital copy to correct the flawed digital effect using the deep learning mechanism performed by the CACM.

5. The device of claim 1, wherein the mobile device performs one or more additional operations comprising:

identifying a region of interest (ROI) within the digital copy automatically using the deep learning mechanism performed by the CACM; and

processing the ROI to identify the flawed photography effect using the deep learning mechanism performed by the CACM.

6. The device of claim 1, wherein the mobile device performs one or more additional operations comprising:

receiving a selection of a region of interest (ROI) within the digital copy from a user using the deep learning mechanism performed by the CACM; and

processing the ROI to identify the flawed photography effect using the deep learning mechanism performed by the CACM.

7. The device of claim 1, wherein the mobile device performs one or more additional operations comprising:

identifying a glare effect within the digital copy as the flawed photography effect using the deep learning mechanism performed by the CACM; and

enhancing the digital copy to correct the flawed photography effect using the deep learning mechanism performed by the CACM.

8. The device of claim 1, wherein the mobile device performs one or more additional operations comprising:

identifying a reflection effect within the digital copy as the flawed photography effect using the deep learning mechanism performed by the CACM; and

enhancing the digital copy to correct the flawed photography effect using the deep learning mechanism performed by the CACM.

9. The device of claim 1, wherein the mobile device performs one or more additional operations comprising:

identifying a moiré

pattern within the digital copy as the flawed photography effect using the deep learning mechanism performed by the CACM; and

enhancing the digital copy to correct the flawed photography effect using the deep learning mechanism performed by the CACM.

10. The device of claim 1, wherein the mobile device performs one or more additional operations comprising:

identifying a geometric distortion within the digital copy as the flawed photography effect using the deep learning mechanism performed by the CACM; and

enhancing the digital copy to correct the flawed photography effect using the deep learning mechanism performed by the CACM.

11. The device of claim 1, wherein the mobile device performs one or more additional operations comprising:

identifying a noise within the digital copy as the flawed photography effect using the deep learning mechanism performed by the CACM; and

enhancing the digital copy to correct the flawed photography effect using the deep learning mechanism performed by the CACM.

12. The device of claim 1, wherein the mobile device performs one or more additional operations comprising:

identifying a light exposure issue as the flawed photography effect using the deep learning mechanism performed by the CACM; and

enhancing the digital copy to correct the flawed photography effect using the deep learning mechanism performed by the CACM.

13. The device of claim 1, wherein the mobile device performs one or more additional operations comprising:

providing a color map of a region of interest (ROI) within the digital copy as the annotation using the deep learning mechanism performed by the CACM, wherein the color map describes a probability of a malignancy associated with the ROI.

14. The device of claim 1, wherein the mobile device performs one or more additional operations comprising:

identifying a motion blur as the flawed photography effect using the deep learning mechanism performed by the CACM; and

enhancing the digital copy to correct the flawed photography effect using the deep learning mechanism performed by the CACM.

15. A mobile device for enhancing and presenting a medical image using a corrective mechanism, the mobile device comprising:

a display component configured to accept an input and display an output associated with an image analysis application,

a camera component configured to capture a first digital copy of the medical image displayed on a monitor in relation to the image analysis application, wherein the camera component is external to the monitor,

a memory configured to store instructions associated with the image analysis application,

a processor coupled to the display component, the camera component, and the memory, the processor executing the instructions associated with the image analysis application, wherein the image analysis application includes;

a computer analysis and correction module (CACM) performing one or more operations comprising;

receiving a first digital copy of the medical image from the camera component;

identifying a flawed photography effect associated with the first digital copy by processing the first digital copy using a deep learning mechanism;

enhancing the first digital copy based on the flawed photography effect using the deep learning mechanism;

processing the first digital copy using the deep learning mechanism to generate an annotation, wherein the annotation includes one of a suspicious label, a not suspicious label, and a follow-up label associated with a cancer identification; and

displaying the first digital copy and the annotation overlaid by an augmented reality mechanism on to the monitor displaying the medical image.

16. The mobile device of claim 15, wherein the medical image includes one of an ultrasound scan, a x-ray scan, a magnetic resonance imaging (MRI) scan, a computed tomography (CT) scan, and a positron emission tomography (PET) scan.

17. The mobile device of claim 15, wherein the flawed photography effect is identified by processing a second digital copy of the medical image using the deep learning mechanism, wherein the second digital copy is received from the camera component.

18. A method of enhancing and presenting a medical image using a corrective mechanism, the method comprising:

receiving a digital copy of the medical image displayed on a monitor, wherein the digital copy is captured by an image source, wherein the monitor is external to the image source;

identifying a flawed photography effect associated with the digital copy by processing the digital copy using a deep learning mechanism;

enhancing the digital copy based on the flawed photography effect using the deep learning mechanism;

processing the digital copy using the deep learning mechanism to generate an annotation, wherein the annotation includes one of a suspicious label, a not suspicious label, and a follow-up label associated with a cancer identification; and

providing the digital copy and the annotation for a presentation overlaid by an augmented reality mechanism on to the monitor displaying the medical image.