Computer aided diagnosis using video from endoscopes

• US 20080058593A1
• Filed: 08/21/2007
• Published: 03/06/2008
• Est. Priority Date: 08/21/2006
• Status: Abandoned Application

• Current Assignee: STI Medical Systems LLC
• Original Assignee: STI Medical Systems LLC
• Inventors: Wolters, Rolf, Gu, Jia
• Application Number:
  US11/895,150
• Publication Number:
  US 20080058593A1
• Time in Patent Office: Days
• US Class Current: 600/109
• CPC Class Codes: A61B 1/000094 extracting biological structures
• CPC Class Codes: G06T 2200/08 involving all processing steps from image acquisition to 3D model generation
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/20036 Morphological image processing
• CPC Class Codes: G06T 2207/20152 Watershed segmentation
• CPC Class Codes: G06T 2207/30028 Colon; Small intestine
• CPC Class Codes: G06T 5/005 Retouching; Inpainting; Scratch removal detecting, correction, reducing or removing defects, e.g. non-responsive pixels of solid state image sensors H04N25/68, scratch removal for cinematographic films scanned by electronic image sensor H04N5/253
• CPC Class Codes: G06T 5/006 Geometric correction correction of chromatic aberrations adapted to be used in an image pickup device containing an electronic image sensor H04N23/10; detecting, correcting, reducing or removing artefacts resulting only from the lens unit, e.g. flare, shading, vignetting or "cos4" H04N25/61
• CPC Class Codes: G06T 5/008 Local, e.g. shadow enhancement
• CPC Class Codes: G06T 5/40 using histogram techniques
• CPC Class Codes: G06T 5/77 Retouching; Inpainting; Scratch removal
• CPC Class Codes: G06T 5/80 Geometric correction
• CPC Class Codes: G06T 5/94 based on local image properties, e.g. for local contrast enhancement
• CPC Class Codes: G06T 7/0012 Biomedical image inspection
• CPC Class Codes: G06T 7/593 from stereo images

Abstract

A process for providing computer aided diagnosis from video data of an organ during an examination with an endoscope, comprising analyzing and enhancing image frames from the video and detecting and diagnosing any lesions in the image frames in real time during the examination. Optionally, the image data can be used to create a 3 dimensional reconstruction of the organ.

Claims

1. A process for providing computer aided diagnosis from video data of an organ during an examination with an endoscope, comprising:

analyzing said video data to discard poor quality image frames to provide satisfying image frames from said video data;

enhancing said satisfying image frames;

detecting and diagnosing any lesions in said satisfying image frames;

wherein said analyzing, enhancing, and detecting and diagnosing steps are performed in real time during said examination.

2. A process according to claim 1, wherein said enhancing step comprises:

removing glint from said satisfying image frames;

detecting blur and discarding poor quality image frames that are not sufficiently clear;

enhancement of contrast in said satisfying image frames;

applying super resolution to said satisfying image frames; and

providing video stabilization to said satisfying image frames.

3. A process according to claim 1, wherein said detecting and diagnosing step comprises color calibration, color analysis, texture analysis and feature detection.

4. A process according to claim 1, further comprising:

reconstructing a still image of said organ in three dimensions from said satisfying image frames, wherein said reconstructing is performed in real time during said examination.

5. A process according to claim 4, wherein said reconstructing step comprises fisheye distortion correction, geometry calibration, image based modeling and three dimensional data stitching to form a reconstructed model;

wherein said detecting and diagnosing step further comprises recovering 3 dimensional shape and size information of any lesions in said organ from said reconstructed model.

6. A process according to claim 1, wherein said detecting and diagnosing includes analysis of blood vessels.

7. A process according to claim 1, wherein said detecting and diagnosing includes analysis of pit patterns.

8. A process according to claim 1, wherein said analyzing, enhancing and detecting and diagnosis steps are performed using an automated image processing system on a computer operably connected to said endoscope.

9. A process for providing computer aided diagnosis from video data of an endoscope during a colonoscopy of a large intestine, comprising:

analyzing said video data to discard poor quality image frames to provide satisfying image frames from said video data;

enhancing said satisfying image frames;

detecting and diagnosing any polyps in said satisfying image frames;

wherein said analyzing, enhancing, and detecting and diagnosing steps are all performed in real time during said colonoscopy by software operating on a computer that is operably connected to receive video data from said endoscope.

10. A process according to claim 9, wherein said analyzing step comprises glint detection and elimination and focus analysis.

11. A process according to claim 9, wherein said enhancing step comprises contrast enhancement, super resolution and video stabilization.

12. A process according to claim 9, wherein said detecting and diagnosing step comprises color calibration, color analysis, texture analysis and feature detection.

13. A process according to claim 12, wherein said texture analysis includes analyzing blood vessels and pit patterns.

14. A process according to claim 9, further comprising:

reconstructing a still image of said large intestine in three dimensions from said satisfying image frames to form a reconstructed model, wherein said reconstructing is performed in real time during said colonoscopy;

wherein said detecting and diagnosing step further comprises recovering 3 dimensional shape and size information of any polyps in said colon from said reconstructed model.

15. A process according to claim 14, wherein said reconstructing step comprises fisheye distortion correction, geometry calibration, image based modeling and three dimensional data stitching.

16. A process for providing computer aided diagnosis from video data of an organ during an examination with an endoscope, comprising:

analyzing said video data to discard poor quality image frames to provide satisfying image frames from said video data;

removing glint from said satisfying image frames;

detecting blur and discarding poor quality image frames that are not sufficiently clear;

enhancement of contrast in said satisfying image frames;

applying super resolution to said satisfying image frames;

providing video stabilization to said satisfying image frames. detecting and diagnosing any lesions in said satisfying image frames;

wherein said detecting and diagnosing step is performed in real time during said examination using an automated image processing system on a computer operably connected to said endoscope.

17. A process according to claim 16, further comprising:

reconstructing a still image of said organ in three dimensions from said satisfying image frames to form a reconstructed model; and

recovering 3 dimensional shape and size information of any lesions in said organ from said reconstructed model;

wherein said reconstructing step is performed in real time during said examination.