Endoscope with images optimized based on depth map derived from structured light images

• US 10,624,533 B2
• Filed: 08/19/2019
• Issued: 04/21/2020
• Est. Priority Date: 10/16/2015
• Status: Active Grant

• Current Assignee: CapsoVision, Inc.
• Original Assignee: CapsoVision, Inc.
• Inventors: Wang, Kang-Huai, Wilson, Gordon C.
• Primary Examiner(s): Elahi, Shan E
• Application Number:
  US16/543,922
• Publication Number:
  US 20190365212A1
• Time in Patent Office: 246 Days
• Field of Search: 348 45
• CPC Class Codes: A61B 1/000095 for image enhancement
• CPC Class Codes: A61B 1/00057 provided with means for testing or calibration
• CPC Class Codes: A61B 1/00096 Optical elements
• CPC Class Codes: A61B 1/041 Capsule endoscopes for imaging
• CPC Class Codes: A61B 1/042 characterised by a proximal camera, e.g. a CCD camera
• CPC Class Codes: A61B 1/0661 Endoscope light sources
• CPC Class Codes: A61B 1/2736 Gastroscopes
• CPC Class Codes: G06T 2207/10068 Endoscopic image
• CPC Class Codes: G06T 2207/30028 Colon; Small intestine
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening
• CPC Class Codes: G06T 7/521 from laser ranging, e.g. using interferometry; from the projection of structured light

Abstract

A method and apparatus for processing gastrointestinal (GI) images are disclosed. According to this method, a regular image is received, where the regular image is captured using an imaging apparatus by projecting non-structured light onto a body lumen when the imaging apparatus is in the body lumen. One or more structured-light images captured using the imaging apparatus by projecting the body lumen with structured light are received. A target distance for a target region in the regular image is derived based on said one or more structured-light images. A filter is determined based on the target distance and camera parameters associated with the imaging apparatus. A first processed target region is generated by applying the filter to the target region to improve sharpness of the target region. A first processed regular image comprising the first processed target region is provided.

Claims

1. A method for processing gastrointestinal (GI) images, comprising:

receiving one or more structured-light images, wherein said one or more structured-light images are captured using an imaging apparatus by projecting structured light onto a body lumen when the imaging apparatus is in the body lumen;

receiving a regular image, wherein the regular image is captured using the imaging apparatus by projecting non-structured light onto the body lumen when the imaging apparatus is in the body lumen;

deriving a target distance for a target region in the regular image based on said one or more structured-light images;

determining a filter based on the target distance and camera parameters associated with the imaging apparatus;

generating a first processed target region by applying the filter to the target region to improve sharpness of the target region; and

providing a first processed regular image comprising the first processed target region.

2. The method of claim 1, wherein the filter corresponds to a deconvolution kernel, and wherein parameters of the filter are designed based on the target distance and the camera parameters.

3. The method of claim 1, wherein the imaging apparatus corresponds to a capsule endoscope.

4. The method of claim 1, wherein the imaging apparatus corresponds to a traditional endoscope with a flexible tube.

5. The method of claim 1, wherein the imaging apparatus corresponds to a capsule endoscope with panoramic cameras and the regular image corresponds to a panoramic image comprising multiple individual images captured by the panoramic cameras covering different fields of view.

6. The method of claim 1, wherein the regular image is partitioned into multiple regions, and wherein two different filters are derived for two of the multiple regions having different distances and said two of the multiple regions are processed by the two different filters respectively.

7. The method of claim 1, wherein the filter determined corresponds to a Wiener filter by taking into account of a point spread function (PSF) associated with camera system and noise in the camera system.

8. The method of claim 1, further comprising:

applying a second filter to the target region of the regular image to generate a second processed regular image, wherein the second filter is determined based on a second target distance and the camera parameters associated with the imaging apparatus, and the second target distance corresponds to a small deviation from the target distance.

9. The method of claim 8, further comprising displaying the second processed regular image on a display device for a user to assess visual quality of the second processed regular image.

10. The method of claim 9, further comprising:

applying a third filter to the target region of the regular image to generate a third processed regular image, wherein the third filter is determined based on a third target distance and the camera parameters associated with the imaging apparatus, and the third target distance is modified from the second target distance in a direction depending on a user indication of visual quality comparison between the first processed regular image and the second processed regular image.

11. The method of claim 1, wherein said one or more structured-light images correspond to one structured-light image captured before the regular image.

12. An apparatus for processing gastrointestinal (GI) images, comprising a processor configured to:

receive one or more structured-light images, wherein said one or more structured-light images are captured using an imaging apparatus by projecting structured light onto a body lumen when the imaging apparatus is in the body lumen;

receive a regular image, wherein the regular image is captured using the imaging apparatus by projecting non-structured light onto the body lumen when the imaging apparatus is in the body lumen;

derive a target distance for a target region in the regular image based on said one or more structured-light images;

determine a filter based on the target distance and camera parameters associated with the imaging apparatus;

generate a first processed target region by applying the filter to the target region to improve sharpness of the target region; and

provide a first processed regular image comprising the first processed target region.