Video imaging system with external area processing optimized for small-diameter endoscopes

• US 5,589,874 A
• Filed: 05/04/1995
• Issued: 12/31/1996
• Est. Priority Date: 06/09/1993
• Status: Expired due to Term

• Current Assignee: Covidien AG (Medtronic Plc)
• Original Assignee: Origin Medsystems Inc. (Boston Scientific Corporation)
• Inventors: Buchin, Michael P.
• Primary Examiner(s): Chin, Tommy P.
• Assistant Examiner(s): Rao, A.
• Application Number:
  US08/434,207
• Time in Patent Office: 607 Days
• Field of Search: 348/65, 348/72, 348/70, 348/588, 348/597, 348/586, 128/10, 128/6
• US Class Current: 348/72
• CPC Class Codes: A61B 1/0005 combining images e.g. side-by-side, superimposed or tiled
• CPC Class Codes: A61B 1/00059 provided with identification means for the endoscope
• CPC Class Codes: A61B 1/00126 optical, e.g. for light supply cables
• CPC Class Codes: A61B 1/042 characterised by a proximal camera, e.g. a CCD camera
• CPC Class Codes: H04N 23/555 for picking-up images in sites, inaccessible due to their dimensions or hazardous conditions, e.g. endoscopes or borescopes
• CPC Class Codes: H04N 23/72 Combination of two or more compensation controls

Abstract

A method for deriving an output signal from an input video signal generated by an image sensor having an image formed on a part of it. The input video signal includes an image portion generated by the part of the image sensor on which the image is formed, and an external portion generated by the part of the image sensor on which the image is not formed. In the method according to the invention, the input video signal is received, the external portion of the input video signal is identified, and the output signal is generated from the input signal by replacing the external portion with a synthesized portion. The synthesized portion may set the external area to black level, to a predetermined luminance and/or hue, or to a luminance and/or hue adaptively determined in response to the image portion of the input signal. A position-dependent variation may also be imposed on the external area. The external area may also be rearranged to change the displayed location of the image. The method also includes various ways of identifying the external portion of the input video signal.

Claims

1. A method of deriving an output video signal by processing an input video signal generated by an image sensor on which an image of a fibre-optic imaging bundle is formed by an image-forming apparatus of which the fibre-optic imaging bundle is a part, the image of the fibre-optic imaging bundle being formed on only a part of the image sensor to mitigate pixellation artifacts resulting from the optical fibres of the fibre-optic imaging bundle, the method comprising steps of:

receiving the input video signal, the input video signal including plural frames, each of the frames having a frame structure, each of the frames including an image portion generated by the part of the image sensor on which the image of the fibre-optic imaging bundle is formed, and an external portion generated by the part of the image sensor on which the image is not formed;

automatically identifying the external portion of the input video signal;

synthesizing, in response to the identifying step, a synthesized external signal portion corresponding to the external portion of the input video signal; and

generating frames of the output video signal by replacing the external portion of each of the frames of the input video signal with the synthesized external signal portion synthesized in the synthesizing step to provide a respective one of the frames of the output video signal with the same frame structure as the frames of the input video signal, wherein, in the step of synthesizing a synthesized external signal portion, the synthesized external signal portion is synthesized to have a predetermined level.

2. The method of claim 1, wherein, in the step of synthesizing a synthesized external signal portion, the synthesized external signal portion is synthesized to have a level corresponding to black level.

3. The method of claim 1, wherein, in the step of synthesizing a synthesized external signal portion, the synthesized external signal portion is synthesized to have a level corresponding to a predetermined luminance and, when the output video signal is a color video signal, to a predetermined hue.

4. The method of claim 1, wherein:

the step of synthesizing a synthesized external signal portion includes steps of;

determining at least one of a luminance and a hue of the image portion of the input video signal;

adaptively determining at least one of an adaptively-determined luminance and an adaptively-determined hue in response to the at least one of the luminance and the hue of the image portion of the input video signal determined in the determining step; and

generating the synthesized external signal portion with a level corresponding to the at least one of the adaptively-determined luminance and the adaptively-determined hue.

5. The method of claim 3, wherein the step of synthesizing a synthesized external signal portion includes a step of spatially varying at least one of the predetermined luminance and the predetermined hue.

6. The method of claim 5, wherein:

the image has a center;

in the step of spatially varying at least one of the predetermined luminance and the predetermined hue, the at least one of the predetermined luminance and the predetermined hue is varied in a radially-varying pattern centered on the image; and

the step of spatially varying at least one of the predetermined luminance and the predetermined hue includes steps of;

determining, for each one of plural points in the external portion of the input video signal, a distance of the one of the points from the center of the image, and, and, for each one of the plural points;

determining a correction factor from the distance of the one of the points and the radially-varying pattern; and

modifying the level corresponding to the at least one of the predetermined luminance and the predetermined hue according to the correction factor.

7. The method of claim 4, wherein the step of synthesizing a synthesized external signal portion additionally includes a step of spatially varying at least one of the adaptively-determined luminance and the adaptively-determined hue prior to the generating step.

8. The method of claim 7, wherein:

the image has a center;

in the step of spatially varying at least one of the predetermined luminance and the predetermined hue, the at least one of the predetermined luminance and the predetermined hue is varied in a radially-varying pattern centered on the image,

the step of spatially varying at least one of the predetermined luminance and the predetermined hue includes steps of;

determining, for each one of plural points in the external portion of the input video signal, a distance of the one of the points from the center of the image, and, for each one of the plural points;

determining a correction factor from the distance of the one of the points and the radially-varying pattern, and

modifying the level corresponding to the at least one of the adaptively-determined luminance and the adaptively-determined hue according to the correction factor.

9. A method of deriving an output video signal by processing an input video signal generated by an image sensor on which an image of a fibre-optic imaging bundle is formed by an image-forming apparatus of which the fibre-optic imaging bundle is a part, the image of the fibre-optic imaging bundle being formed on only a part of the image sensor to mitigate pixellation artifacts resulting from the optical fibres of the fibre-optic imaging bundle, the method comprising steps of:

receiving the input video signal, the input video signal including plural frames, each of the frames having a frame structure, each of the frames including an image portion generated by the part of the image sensor on which the image of the fibre-optic imaging bundle is formed, and an external portion generated by the part of the image sensor on which the image is not formed;

automatically identifying the external portion of the input video signal;

synthesizing, in response to the identifying step, a synthesized external signal portion corresponding to the external portion of the input video signal; and

generating frames of the output video signal by replacing the external portion of each of the frames of the input video signal with the synthesized external signal portion synthesized in the synthesizing step to provide a respective one of the frames of the output video signal with the same frame structure as the frames of the input video signal, wherein;

the image of the fibre-optic imaging bundle is formed in a position on the image sensor,

the method is for deriving an output video signal wherein, when the output video signal is displayed to provide a picture, the image portion is displayed located in a selected position in the picture, the selected position being different from a position corresponding to the position of the image on the image sensor, and

in the input video signal, the image portion has a positional relationship to the frames of the input video signal, and wherein;

the step of generating frames of the output video signal includes a step of transferring the image portion from the frames of the input video signal to the respective frames of the output video signal in a positional relationship to the frames of the output video signal, the transferring step selectively changing the positional relationship of the image portion to the frames of the output video signal relative to the positional relationship of the image portion to the frames of the input video signal.

10. The method of claim 9, wherein the step of transferring the image portion from the frames of the input video signal to the frames of the output video signal includes steps of:

providing a memory including plural storage locations;

storing the image portion of the frames of the input video signal and the synthesized external signal portion in the storage locations in the memory corresponding to the positional relationship of the image portion to the frames of the output video signal; and

sequentially reading the storage locations in the memory to generate the each of the frames of the output video signal.

11. The method of claim 9, wherein the step of transferring the image portion from the frames of the input video signal to the frames of the output video signal includes steps of:

providing a memory including plural storage locations;

sequentially storing each of the frames of the input video signal into the storage locations in the memory; and

reading only the storage locations in the memory wherein the image portion is stored into the respective one of the frames of the output video signal in a sequence corresponding to the positional relationship of the image portion to the frames of the output video signal.

12. The method of claim 9, wherein the method is for deriving the output video signal for display using a display apparatus susceptible to screen burn, and method is for gradually and progressively changing the position in which the image portion of the output video signal is displayed on the display apparatus about the selected position to reduce screen burn on the display apparatus, and the method additionally comprises a step of gradually and progressively changing the positional relationship between the image portion and the frames of the output video signal.

13. The method of claim 10, wherein:

the input video signal additionally includes plural lines each including an image part generated by the part of the image sensor on which the image of the fibre-optic imaging bundle is formed, and an external part generated by the part of the image sensor on which the image is not formed;

in the step of identifying the external portion of the input video signal, the lines including an image part and an external part are identified, and, for each of such lines, a position of a boundary between the image part and the external part is identified; and

the step of storing the image portion of the frames of the input video signal and the synthesized external signal portion in storage locations in the memory includes steps of;

reading an image offset indicating the selected position of the image portion in the picture,

in response to the image offset and the lines and the boundary positions identified in the identifying step, reserving reserved storage locations corresponding to the positional relationship of the image portion to the frames of the output video signal, and

storing each one of the frames of the input video signal in the memory by;

storing the image portion of the one of the frames of the input video signal in the reserved storage locations; and

sequentially storing in the storage locations in the memory the Synthesized external image portion synthesized in the synthesizing step in lieu of the external portion of the one of the frames of the input video signal, but, when sequentially storing the synthesized external signal portion would store the synthesized external signal portion in one of the reserved storage locations, storing the synthesized external signal portion in one of the storage locations in which the image portion would be stored if the positional relationship of the image portion to the frames the output video signal were the same as the positional relationship of the image portion to the frames of the input video signal.

14. The method of claim 11, wherein:

the input video signal includes plural lines each including an image part generated by the pan of the image sensor on which the image of the fibre-optic imaging bundle is formed, and an external part generated by the part of the image sensor on which the image is not formed;

in the step of identifying the external portion of the video signal, the lines including an image part and an external part are identified, and, for each of such lines, a position of a boundary between the image part and the external part is identified; and

the step of transferring the image portion from the frames of the input video signal to the frames of the output video signal additionally includes steps of;

reading an image offset indicating the selected position of the displayed image,

in response to the image offset, and the lines and the boundary positions identified in the identifying step, determining positions of the image portion and the synthesized external signal portion in the output video signal, and

reading from the storage locations in the memory into the output video signal by;

sequentially reading the storage locations in the memory wherein the image portion is stored into the image portion of the output video signal, and

sequentially reading the synthesized external signal portion into the output video signal, but, when sequentially reading the synthesized external signal portion into the output video signal would read the synthesized external signal portion into the image portion of the output video signal, reading the synthesized external signal portion into a part of the output signal into which the image portion would have been read if the positional relationship of image portion to the frames of the output video signal were the same as the positional relationship of the image portion to the frames of the input video signal.

15. The method of claim 12, wherein the step of gradually and progressively changing the positional relationship between the image portion and the frames of the output signal includes steps of:

initializing a current positional relationship to a value indicating a positional relationship between the image portion and a one of the frames of the output video signal corresponding to the selected position of the displayed image portion; and

repeating steps of;

determining the current positional relationship,

generating a random variation in the current positional relationship,

adding the random variation to the current positional relationship to determine a new positional relationship,

calculating a vector between the current positional relationship and the new positional relationship,

progressively changing the positional relationship between the image portion and the frames of the output video signal along the vector until the positional relationship reaches the new positional relationship, and

setting the current positional relationship to the new positional relationship.