Medical image based distortion correction mechanism

• US 10,311,570 B1
• Filed: 12/07/2018
• Issued: 06/04/2019
• Est. Priority Date: 12/07/2018
• Status: Active Grant

• Current Assignee: Rutgers University
• Original Assignee: Sonavista, Inc.
• Inventors: Podilchuk, Christine I., Mammone, Richard
• Primary Examiner(s): Abdi, Amara
• Application Number:
  US16/213,201
• Time in Patent Office: 179 Days
• CPC Class Codes: G06F 18/24143 Distances to neighbourhood prototypes, e.g. restricted Coulomb energy networks [RCEN]
• CPC Class Codes: G06F 18/40 Software arrangements specially adapted for pattern recognition, e.g. user interfaces or toolboxes therefor
• CPC Class Codes: G06T 2207/20084 Artificial neural networks [ANN]
• CPC Class Codes: G06T 2207/20092 Interactive image processing based on input by user
• CPC Class Codes: G06T 2207/30168 Image quality inspection
• CPC Class Codes: G06T 7/0002 Inspection of images, e.g. flaw detection
• CPC Class Codes: G06T 7/0014 using an image reference approach
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 10/82 using neural networks
• CPC Class Codes: G06V 10/945 User interactive design; Environments; Toolboxes
• CPC Class Codes: G06V 2201/03 Recognition of patterns in medical or anatomical images

Abstract

A mobile device to provide a medical image based distortion correction mechanism is described. An image analysis application, executed by the mobile device, captures a digital copy of the medical image with a camera component in response to a user action. Distortion(s) associated with the digital copy are identified by processing the digital copy with deep neural network (DNN) model(s). Next, a manual correction description is determined to correct the distortion(s) in relation to the camera component and the medical image. Furthermore, a notification to recapture the digital copy is provided to the user. The notification includes the manual correction description. Additionally, in response to another user action to correct the distortion(s) or a failure to detect the user execute manual correction(s) associated with the distortion(s) within a time period, the distortion(s) within the digital copy are corrected and the corrected digital copy is provided to the user.

Claims

1. A mobile device to provide a medical image based distortion correction mechanism, wherein the device is configured to:

capture a digital copy of the medical image with a camera component in response to a first user action, wherein the medical image is displayed by a display device;

identify one or more distortions associated with the digital copy by processing the digital copy with one or more deep neural network (DNN) models;

determine a manual correction description to correct the one or more distortions in relation to the camera component and the medical image;

provide a notification to recapture the digital copy to a user, wherein the notification includes the manual correction description; and

in response to a second user action to correct the one or more distortions or a failure to detect the user execute one or more manual corrections associated with the one or more distortions within a time period,

correct the one or more distortions within the digital copy, and

provide the corrected digital copy to the user.

2. The mobile device of claim 1, wherein the device is further configured to:

determine an input image set of the one or more DNN models comparable to the digital copy, wherein the one or more DNN models include an output image set correlated to the input image set;

determine one or more variances between the output image set and the digital copy; and

designate the one or more variances as the one or more distortions.

3. The mobile device of claim 2, wherein the output image set includes one or more distortion corrections applied to the input image set.

4. The mobile device of claim 1, wherein the device is further configured to:

identify a glare effect within the digital copy as one of the one or more distortions; and

determine the manual correction description associated with the glare effect, wherein the manual correction description includes a suggestion to move the camera component in relation to the display device to remove or minimize the glare effect.

5. The mobile device of claim 1, wherein the device is further configured to:

identify a reflection effect within the digital copy as one of the one or more distortions; and

determine the manual correction description associated with the reflection effect, wherein the manual correction description includes a suggestion to move the camera component closer or away from the display device to remove or minimize the reflection effect.

6. The mobile device of claim 1, wherein the device is further configured to:

identify a moiré

pattern within the digital copy as one of the one or more distortions; and

determine the manual correction description associated with the moiré

pattern, wherein the manual correction description includes a suggestion to adjust a focus associated with the camera component.

7. The mobile device of claim 1, wherein the device is further configured to:

identify a geometric distortion within the digital copy as one of the one or more distortions; and

determine the manual correction description associated with the geometric distortion, wherein the manual correction description includes a suggestion to adjust a focus associated with the camera component.

8. The mobile device of claim 1, wherein the device is further configured to:

identify a noise within the digital copy as one of the one or more distortions; and

determine the manual correction description associated with the noise, wherein the manual correction description includes a suggestion to adjust a lighting associated with the display device.

9. The mobile device of claim 1, wherein the device is further configured to:

identify a light exposure issue within the digital copy as one of the one or more distortions; and

determine the manual correction description associated with the light exposure issue, wherein the manual correction description includes a suggestion to adjust a lighting associated with the display device.

10. The mobile device of claim 1, wherein the device is further configured to:

identify a motion blur within the digital copy as one of the one or more distortions; and

determine the manual correction description associated with the motion blur, wherein the manual correction description includes a suggestion to steady a grip on the camera component while recapturing the digital copy.

11. A mobile device for providing a mechanism to correct a distortion in a medical image, the mobile device comprising:

a camera component configured to capture a digital copy of the medical image in relation to the image analysis application,

a memory configured to store instructions associated with the image analysis application,

a processor coupled to the camera component and the memory, the processor executing the instructions associated with the image analysis application, wherein the image analysis application includes;

an image analysis module configured to;

capture the digital copy of the medical image with the camera component in response to a first user action, wherein the medical image is displayed by a display device;

determine an input image set of one or more deep neural network (DNN) models comparable to the digital copy, wherein the one or more DNN models include an output image set correlated to the input image set;

determine one or more variances between the output image set and the digital copy; and

designate the one or more variances as one or more distortions;

determine a manual correction description to correct the one or more distortions in relation to the camera component and the medical image;

provide a notification to recapture the digital copy to a user, wherein the notification includes the manual correction description; and

in response to a second user action to correct the one or more distortions or a failure to detect the user execute one or more manual corrections associated with the one or more distortions within a time period,

correct the one or more distortions within the digital copy, and

provide the corrected digital copy to the user.

12. A method of providing a mechanism to correct a distortion in a medical image, the method comprising:

capturing a digital copy of the medical image with a camera component, wherein the medical image is displayed by a display device;

determining an input image set of one or more deep neural network (DNN) models comparable to the digital copy, wherein the one or more DNN models include an output image set correlated to the input image set;

determining one or more variances between the output image set and the digital copy; and

designating the one or more variances as one or more distortions;

determining a manual correction description to correct the one or more distortions in relation to the camera component and the medical image;

providing a notification to recapture the digital copy to the user, wherein the notification includes the manual correction description;

in response to a user action to correct the one or more distortions or a failure to detect the user execute one or more manual corrections associated with the one or more distortions within a time period,

correcting the one or more distortions within the digital copy, and

providing the corrected digital copy to the user.