Method and apparatus for reducing the inherently dark grid pattern from the video display of images from fiber optic bundles

• US 5,751,340 A
• Filed: 08/21/1996
• Issued: 05/12/1998
• Est. Priority Date: 08/21/1996
• Status: Expired due to Term

• Current Assignee: Karl Storz Se & Co. Kg
• Original Assignee: Karl Storz Se & Co. Kg
• Inventors: Kennedy, Bryan D., Chatenever, David, Irion, Klaus, Strobl, Karlheinz
• Primary Examiner(s): Britton, Howard W.
• Application Number:
  US08/701,131
• Time in Patent Office: 629 Days
• Field of Search: 348/65, 348/73, 348/607, 348/615
• US Class Current: 348/65
• CPC Class Codes: A61B 1/00165 with light-conductive means, e.g. fibre optics A61B1/07 takes precedence
• CPC Class Codes: A61B 1/042 characterised by a proximal camera, e.g. a CCD camera
• CPC Class Codes: H04N 23/55 Optical parts specially adapted for electronic image sensors; Mounting thereof
• CPC Class Codes: H04N 23/555 for picking-up images in sites, inaccessible due to their dimensions or hazardous conditions, e.g. endoscopes or borescopes
• CPC Class Codes: H04N 23/81 for suppressing or minimising disturbance in the image signal generation
• CPC Class Codes: H04N 25/61 the noise originating only from the lens unit, e.g. flare, shading, vignetting or "cos4"
• CPC Class Codes: H04N 5/21 Circuitry for suppressing or minimising disturbance, e.g. moiré or halo

Abstract

A method and apparatus for improving an image received from a endoscope having a flexible fiber optic bundle that produces an interstitial grid overlay due to the dead space between each fiber pixel. The interstitial grid pattern in the image, is substantially eliminated by processing the image through a linear digital filter to attenuate the first-order spatial frequency components that contribute most to the interstitial pattern. The filter employs relatively small convolution kernels that are optimized for specific fiber spacing. The image is then processed through the linear digital filter by convolving the respective pixels in the image by the derived coefficients. The image is improved and the grid pattern may also be reduced by a dilation process which involves processing the image through a dilation algorithm to "grow" the bright center of each fiber pixel in the image so that the effect of the grid pattern is reduced. The process involves filling in the dark area of the grid space with the color and intensity from the nearest fiber center to obtain an equivalent image representation. The dilation process uses a non-linear image processing algorithm to replace each pixel with the maximum pixel brightness in its immediate neighborhood depending on the selection of a kernel. The dilation kernel is selected by determining the number of detector (CCD) pixels that fit in a space between the fiber cores and then selecting a matrix size and kernel pattern.

Claims

1. In combination with an endoscope and a video display, said video display having an array of pixels for displaying purposes, said endoscope including an objective lens at its distal end and a fiber optic bundle for transmitting an image to a CCD camera at it proximal end;

said fiber optic bundle comprising a substantial number of optical fibers each with distal and proximal ends, the ends of said fibers forming an array which is identical at both of said ends of said bundle, whereby to forward an image from said lens to said camera, each said fiber consisting of a central light-transmitting core and a peripheral cladding, said fibers being clustered so as to form voids between groups of contacting fibers, said claddings and said voids being non-transmissive of light so as to form a visible dark grid encompassing said light-transmissive cores;

said camera having arrays of pixels at photo sites on said camera respective to pixels on said display, the improvement comprising;

grid reduction means receiving pixel signals from said camera and generating respective enhanced signals for said display, said grid reduction means comprising circuitry which adjusts the color and intensity of the signal generated by the CCD respective to pixels in said grid, thereby to reduce the prominence of said grid.

2. Apparatus according to claim 1 in which said circuit means adjusts the output signal of each camera pixel as a function of the color and intensity of at least some near neighboring camera pixels.

3. Apparatus according to claim 2 in which the intensity of the signal received from the camera is adjusted to the color and intensity of the most intense one of a group of nearby camera pixels.

4. Apparatus according to claim 3 in which said group of nearby covered pixels comprises a matrix array of surrounding camera pixels of a rectangular array including said respective pixel.

5. Apparatus according to claim 3 in which said matrix array is a symmetric matrix array which includes said respective pixel.

6. Apparatus according to claim 1 in which said circuit means comprises filter means which alternates the frequency components of the spatial frequencies which comprise the grid pattern.

7. Apparatus according to claim 6 in which said filter means comprises a convolver which multiplies the respective pixels in a sequence by respective stored values and sums the multiplied outputs to uniformly increase the intensity of respective pixels in said sequence thereby attenuating said grid pattern.

8. Apparatus according to claim 7 in which said respective stored values are coefficients of values in a kernel.

9. Apparatus according to claim 8 in which said kernel is a 3×

7 stage array.

10. Apparatus according to claim 9 in which each pixel is multiplied by a coefficient derived from each stage in said array and added to the output of a previous stage and summed at an output.

11. Apparatus according to claim 10 in which said kernel is shafted a predetermined amount often processing each pixel sequence.

12. Apparatus according to claim 11 in which said filter means filters the luminance signal.

13. Apparatus according to claim 11 in which said filter means comprises a filter in the luminance channel and in both chroma channels or in red, green, blue channels.

14. Apparatus according to claim 13 including field and frame memory control means for controlling the field mode or frame mode input to said filter means.

15. A method of improving an endoscope image received through a fiber optic bundle to remove said interstitial grid pattern comprising;

converting said endoscope image to a digital image signal;

processing said digital image signal to generate respective enhanced pixel signals;

converting said processed digital signal image to a switchable color output signal;

whereby said interstitial grid pattern is substantially removed.

16. The method according to claim 15 in which said processing comprises linear digital filtering of said digital image to remove the spatial frequencies composing said interstitial grid pattern.

17. The method according to claim 16 in which said linear digital filtering comprises;

selecting a kernel having predetermined elements from a ratio of the pixel to fiber spacing of said fiber optic bundle;

calculating the coefficient of the elements in said kernel;

sequentially multiplying the pixels in said digital image by each of said coefficients in said kernel and summing said multiplied values whereby relative pixel brightness is enhanced and said interstitial pattern is substantially valued.

18. The method according to claim 17 including selecting 3×

7 element kernel.

19. The method according to claim 18 in which each pixel is sequentially multiplied by each coefficient, added to the value of the previous multiplied pixel and summed at the output.

20. The method according to claim 16 in which said linear digital filtering comprises processing said digital image in a convolver.

21. The method according to claim 20 comprising processing only the luminance signal.

22. The method according to claim 15 comprising dilating respective pixels in said digital image signal to improve the overall brightness and substantially eliminate the interstitial grid pattern.

23. The method according to claim 22 comprising dilating respective pixels by replacing each pixel with the maximum pixel brightness of a pixel in the immediate neighborhood.

24. The method according to claim 23 comprises selecting a kernel of pixels;

processing each pixel of said kernel in an image processing algorithm.

25. The method according to claim 24 comprising selecting an optimum kernel by determining the ratio of detector pixels to the space between fiber cores of said fiber optic bundle;

select a matrix size relative to said ratio;

set all coefficients of said kernel used for dilation to one;

whereby said image processing algorithm replaces the brightness of each pixel with the brightness of a nearby brightest pixel in said kernel domain.

26. The method according to claim 25 in which said selected kernel domain is a symmetric matrix.

27. The method according to claim 26 in which said symmetric matrix is between 3×

3 and 6×

6 symmetric matrix.