Continuous video in a light deficient environment

• US 10,568,496 B2
• Filed: 07/26/2013
• Issued: 02/25/2020
• Est. Priority Date: 07/26/2012
• Status: Active Grant

• Current Assignee: Depuy Synthes Products Incorporated (Johnson & Johnson)
• Original Assignee: Depuy Synthes Products Incorporated (Johnson & Johnson)
• Inventors: Blanquart, Laurent, Talbert, Joshua D., Wichern, Donald M., Richardson, John, Henley, Jeremiah D.
• Primary Examiner(s): Neal, Timothy J
• Assistant Examiner(s): Woo, Jae
• Application Number:
  US13/952,518
• Publication Number:
  US 20140163319A1
• Time in Patent Office: 2,405 Days
• Field of Search: 600178, 600180, 600118, 600160
• CPC Class Codes: A61B 1/00006 of control signals
• CPC Class Codes: A61B 1/000095 for image enhancement
• CPC Class Codes: A61B 1/045 Control thereof
• CPC Class Codes: A61B 1/05 characterised by the image sensor, e.g. camera, being in the distal end portion
• CPC Class Codes: A61B 1/051 Details of CCD assembly
• CPC Class Codes: A61B 1/0638 providing two or more wavelengths
• CPC Class Codes: A61B 1/0646 with illumination filters
• CPC Class Codes: A61B 1/0684 using light emitting diodes [LED]
• CPC Class Codes: A61B 1/07 using light-conductive means, e.g. optical fibres
• CPC Class Codes: G02B 23/2484 Arrangements in relation to a camera or imaging device processing or control of video signals generated by an endoscope H04N23/60, H04N23/70
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 23/74 by influencing the scene brightness using illuminating means
• CPC Class Codes: H04N 25/531 by controlling rolling shutters in CMOS SSIS
• CPC Class Codes: H04N 25/63 applied to dark current

Abstract

The present disclosure extends to methods, systems, and computer program products for producing an image in light deficient environments and associated structures, methods and features is disclosed and described. The features of the system may include controlling a light source through duration, intensity or both; pulsing a component controlled light source during the blanking time; maximizing the blanking time to allow optimum light; and maintaining color balance.

Claims

1. A system for digital imaging in an ambient light deficient environment comprising:

an imaging sensor comprising an array of pixels for sensing electromagnetic radiation;

an emitter configured to emit pulses of electromagnetic radiation; and

a controller comprising a processor in electrical communication with the imaging sensor and the emitter, the controller configured to coordinate and synchronize the timing of the pulse of electromagnetic radiation from the emitter during a blanking time of the imaging sensor, the blanking time of the imaging sensor corresponding to the time between a readout of pixels in a last row of the imaging sensor and a beginning of a next frame readout phase of an operational cycle of the imaging sensor,

wherein the controller captures a plurality of frames during a plurality of operational cycles of the imaging sensor, each one of the plurality of frames being captured during each one of the operational cycles, wherein each of the frames in the plurality of frames correspond to a single pulse of electromagnetic radiation emitted by the emitter and produce imaging sensor data that corresponds in time with each pulse of electromagnetic radiation to produce a plurality of data sets that correspond to the plurality of frames;

wherein the controller causes the imaging sensor to read each data set corresponding to each of the plurality of frames from the array of pixels during a frame readout phase of the operational cycle of the imaging sensor, the frame readout phase corresponding to the amount of time required to read each pixel in the array of pixels and wherein at least a first frame in the plurality of frames is captured at a point during the operational cycle of the imaging sensor that overlaps the frame readout phase of the operational cycle of the imaging sensor;

wherein the plurality of frames and the plurality of data sets are combined to form a single image frame; and

wherein the single image frame is combined with a plurality of image frames to produce a video stream.

2. The system of claim 1, wherein a frame readout phase for the first frame is equal in duration to the frame readout phase for a second frame.

3. The system of claim 1, wherein the blanking time is variable based on the emitted pulses of electromagnetic radiation.

4. The system of claim 1, wherein the system further comprises a connection cable electrically connecting a hand piece and the controller.

5. The system of claim 1, wherein the imaging sensor is a monochrome sensor.

6. The system of claim 1, wherein the emitter is configured to emit the electromagnetic radiation pulses at a plurality of electromagnetic wavelengths.

7. The system of claim 6, wherein the emitter is configured to emit the electromagnetic radiation pulses at three or more electromagnetic wavelengths.

8. The system of claim 7, wherein the three or more electromagnetic wavelengths of electromagnetic radiation emitted by the emitter comprise:

a green wavelength of electromagnetic radiation;

a red wavelength of electromagnetic radiation; and

a blue wavelength of electromagnetic radiation.

9. The system of claim 8, wherein the blue, red and green wavelengths of electromagnetic radiation are emitted in a repeating pattern of pulses.

10. The system of claim 9, wherein the green wavelength is represented in the repeating pattern of pulses twice as often as the red and blue wavelengths.

11. The system of claim 1, wherein the pulses of electromagnetic radiation emitted by the emitter comprise a first wavelength for the first frame and a second wavelength for a second frame.

12. The system of claim 1, wherein the pulses of electromagnetic radiation emitted by the emitter comprise one or more wavelengths that are visible to human eyes.

13. The system of claim 1, wherein the pulses of electromagnetic radiation emitted by the emitter comprise one or more wavelengths that are not visible to human eyes.

14. The system of claim 1, wherein the pulses of electromagnetic radiation emitted by the emitter comprise one or more wavelengths that are visible to human eyes and one or more wavelengths that are not visible to human eyes.

15. The system of claim 1, wherein the pulses of electromagnetic radiation emitted by the emitter comprise differing magnitudes for differing wavelengths.

16. The system of claim 15, wherein one or more pixels in the array of pixels in the imaging sensor include a light sensitivity to the differing wavelengths.

17. The system of claim 1, wherein the imaging sensor is disposed at a distal portion of an endoscope.

18. The system of claim 1, wherein the imaging sensor is disposed within a hand piece.