Wide dynamic range using monochromatic sensor

• US 9,509,917 B2
• Filed: 07/26/2013
• Issued: 11/29/2016
• Est. Priority Date: 07/26/2012
• Status: Active Grant

• Current Assignee: Depuy Synthes Products Incorporated (Johnson & Johnson)
• Original Assignee: Depuy Synthes Products Incorporated (Johnson & Johnson)
• Inventors: Blanquart, Laurent, Richardson, John
• Primary Examiner(s): Perungavoor, Sath V
• Assistant Examiner(s): AYNALEM, NATHNAEL B
• Application Number:
  US13/952,564
• Publication Number:
  US 20140160260A1
• Time in Patent Office: 1,222 Days
• Field of Search: 348/241, 348/294, 348/208.99, 348/362, 348/65, 348/68
• US Class Current: 1/1
• CPC Class Codes: A61B 1/045 Control thereof
• CPC Class Codes: A61B 1/0605 for spatially modulated illumination
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 2209/042 having a single pick-up sensor
• CPC Class Codes: H04N 23/10 for generating image signals from different wavelengths
• CPC Class Codes: H04N 23/12 with one sensor only
• CPC Class Codes: H04N 23/56 provided with illuminating means
• CPC Class Codes: H04N 23/74 by influencing the scene brightness using illuminating means
• CPC Class Codes: H04N 23/741 by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors
• CPC Class Codes: H04N 23/843 Demosaicing, e.g. interpolating colour pixel values
• CPC Class Codes: H04N 25/10 for transforming different wavelengths into image signals
• CPC Class Codes: H04N 25/58 involving two or more exposures
• CPC Class Codes: H04N 25/583 with different integration times
• CPC Class Codes: H04N 25/585 with pixels having different sensitivities within the sensor, e.g. fast or slow pixels or pixels having different sizes
• CPC Class Codes: H04N 25/70 SSIS architectures; Circuits associated therewith
• CPC Class Codes: H04N 25/778 comprising amplifiers shared between a plurality of pixels, i.e. at least one part of the amplifier must be on the sensor array itself

Abstract

A digital imaging method for use with an endoscope in ambient light deficient environments is disclosed. In an implementation, the method may include illuminating the environment with pulsed electromagnetic radiation and sensing differing exposures of reflected electromagnetic radiation with a pixel array of an image sensor. The method may further include receiving image data from the pixel array that corresponds to the differing exposures. The method may include creating exposed frames from the image data, where each frame corresponds with a different exposure. The method may further include emitting pulses of electromagnetic radiation during a blanking period. The method may further include creating a display frame from the exposed frames. The method may further include creating a stream of images by sequentially combining display frames taken using the differing exposures to provide increased dynamic range.

Claims

1. A digital imaging method for use with an endoscope in ambient light deficient environments comprising:

illuminating the environment with pulsed electromagnetic radiation;

sensing a plurality of differing exposures of reflected electromagnetic radiation with a pixel array of an image sensor;

receiving image data from said pixel array corresponding to the plurality of differing exposures of reflected electromagnetic radiation;

creating a plurality of exposed frames from said image data, wherein each exposed frame corresponds to a different exposure;

emitting pulses of electromagnetic radiation during a predefined portion of a frame readout period of the image sensor during which pulse sensitive pixels are not being read;

creating a single display frame from the plurality of exposed frames; and

creating a stream of images by sequentially combining a plurality of display frames taken using the plurality of differing exposures to provide increased dynamic range.

2. The method of claim 1, wherein the pulsed electromagnetic radiation is pulsed in a pattern of red, green, blue, and green.

3. The method of claim 1, wherein the pulsed electromagnetic radiation is pulsed in a pattern of luminance, chrominance blue, luminance, and chrominance red.

4. The method of claim 1, wherein the pulsed electromagnetic radiation is pulsed in a four cycle pattern.

5. The method of claim 1, wherein the pulsed electromagnetic radiation is pulsed in a three cycle pattern.

6. The method of claim 1, wherein the pulsed electromagnetic radiation comprises pulses in red, green, and blue color space and luminance, chromared, and chromablue color space.

7. The method of claim 1, comprising horizontal binning during a charge period of a pixel array operation.

8. The method of claim 1, further comprising varying the pattern of a frame exposure period.

9. The method of claim 1, further comprising creating a three dimensional image stream by combining the image frames of a plurality of pixel arrays disposed on a plurality of substrates that are offset relative a scene.

10. The method of claim 1, wherein the pixel array is disposed on a first substrate and supporting circuitry is disposed remotely on a second substrate.

11. The method of claim 1, further comprising pulsing an emitter to produce a pattern of green long exposed combined with green short exposed then blue then green long exposed combined with green short exposed then red in a cycle.

12. The method of claim 1, further comprising pulsing an emitter to produce a pattern of green long exposed combined with green short exposed then blue long exposed combined with blue short exposed then green long exposed combined with green short exposed then red long exposed combined with red short exposed in a cycle.

13. The method of claim 1, further comprising pulsing an emitter to produce a pattern of luminance long exposed combined with luminance short exposed then chrominance blue then luminance long exposed combined with luminance short exposed then chrominance red in a cycle.

14. The method of claim 1, further comprising pulsing an emitter to produce a pattern of luminance long exposed combined with luminance short exposed then chrominance blue long exposed combined with chrominance blue short exposed then luminance long exposed combined with luminance short exposed then chrominance red long exposed combined with chrominance red short exposed in a cycle.

15. The method of claim 1, further comprising pulsing an emitter to produce a pattern having of chrominance blue and chrominance red within the same frame.

16. The method of claim 1, further comprising pulsing an emitter to produce a pattern having of blue and red within the same frame.

17. A digital imaging method for use with an endoscope in ambient light deficient environments comprising:

illuminating the environment with pulsed electromagnetic radiation;

sensing a plurality of differing exposures of reflected electromagnetic radiation with a pixel array of an image sensor;

receiving image data from said pixel array corresponding to the plurality of differing exposures of reflected electromagnetic radiation;

creating a plurality of exposed frames from said image data, wherein each exposed frame corresponds to a different exposure;

emitting pulses of electromagnetic radiation during a predefined portion of a frame readout period of the image sensor during which pulse sensitive pixels are not being read;

creating a stream of images by sequentially combining a plurality of display frames taken using the plurality of differing exposures to provide increased dynamic range; and

pulsing an emitter to produce a pattern of luminance long exposed combined with luminance short exposed then chrominance blue then luminance long exposed combined with luminance short exposed then chrominance red in a cycle.

18. A digital imaging method for use with an endoscope in ambient light deficient environments comprising:

illuminating the environment with pulsed electromagnetic radiation;

sensing a plurality of differing exposures of reflected electromagnetic radiation with a pixel array of an image sensor;

receiving image data from said pixel array corresponding to the plurality of differing exposures of reflected electromagnetic radiation;

creating a plurality of exposed frames from said image data, wherein each exposed frame corresponds to a different exposure;

emitting pulses of electromagnetic radiation during a predefined portion of a frame readout period of the image sensor during which pulse sensitive pixels are not being read;

creating a stream of images by sequentially combining a plurality of display frames taken using the plurality of differing exposures to provide increased dynamic range; and

pulsing an emitter to produce a pattern of luminance long exposed combined with luminance short exposed then chrominance blue long exposed combined with chrominance blue short exposed then luminance long exposed combined with luminance short exposed then chrominance red long exposed combined with chrominance red short exposed in a cycle.