YCbCr pulsed illumination scheme in a light deficient environment

• US 10,785,461 B2
• Filed: 04/29/2019
• Issued: 09/22/2020
• Est. Priority Date: 07/26/2012
• Status: Active Grant

• Current Assignee: Depuy Synthes Products Incorporated (Johnson & Johnson)
• Original Assignee: Depuy Synthes Products Incorporated (Johnson & Johnson)
• Inventors: Blanquart, Laurent, Richardson, John
• Primary Examiner(s): Aynalem, Nathnael
• Application Number:
  US16/398,114
• Publication Number:
  US 20190253685A1
• Time in Patent Office: 512 Days
• Field of Search: 348 68, 348234, 398172
• CPC Class Codes: A61B 1/000095 for image enhancement
• CPC Class Codes: A61B 1/045 Control thereof
• CPC Class Codes: A61B 1/051 Details of CCD assembly
• CPC Class Codes: A61B 1/063 for monochromatic or narrow-band illumination
• CPC Class Codes: A61B 1/0638 providing two or more wavelengths
• CPC Class Codes: A61B 1/0655 Control therefor
• CPC Class Codes: F04C 2270/041 Controlled or regulated
• CPC Class Codes: H04N 1/6016 Conversion to subtractive colour signals
• CPC Class Codes: H04N 23/10 for generating image signals from different wavelengths
• CPC Class Codes: H04N 23/12 with one sensor only
• CPC Class Codes: H04N 23/555 for picking-up images in sites, inaccessible due to their dimensions or hazardous conditions, e.g. endoscopes or borescopes
• CPC Class Codes: H04N 23/56 provided with illuminating means
• CPC Class Codes: H04N 23/74 by influencing the scene brightness using illuminating means
• CPC Class Codes: H04N 23/84 for processing colour signals
• CPC Class Codes: H04N 25/11 Arrangement of colour filter arrays [CFA]; Filter mosaics
• CPC Class Codes: H04N 25/70 SSIS architectures; Circuits associated therewith
• CPC Class Codes: H04N 25/75 Circuitry for providing, modifying or processing image signals from the pixel array
• CPC Class Codes: H04N 25/76 Addressed sensors, e.g. MOS or CMOS sensors
• CPC Class Codes: H04N 5/04 Synchronising for television systems using pulse code modulation H04N7/56
• CPC Class Codes: H04N 9/646 for image enhancement, e.g. vertical detail restoration, cross-colour elimination, contour correction, chrominance trapping filters
• CPC Class Codes: H04N 9/77 : Circuits for processing the brightness signal and the chrominance signal relative to each other, e.g. adjusting the phase of the brightness signal relative to the colour signal, correcting differential gain or differential phase circuits for matrixing H04N9/67

Abstract

Methods, systems, and computer program products for digital imaging in an ambient light deficient environment are described. The methods, systems, and computer program products described include an imaging sensor with an array of pixels for sensing electromagnetic radiation, an emitter that is configured to emit a pulse of electromagnetic radiation, and a control unit that includes a processor. The control unit is in electrical communication with the imaging sensor and the emitter. The control unit is configured to synchronize the emitter and the imaging sensor so as to produce a plurality of image reference frames. The plurality of image reference frames include a luminance frame with luminance image data and a chrominance frame with chrominance data, where the plurality of image reference frames are combined to form a color image.

Claims

1. A system for digital imaging in an ambient light deficient environment comprising:

an imaging sensor comprising an array of pixels for sensing electromagnetic radiation;

an emitter configured to emit a plurality of pulses of electromagnetic radiation; and

a control unit comprising a processor and wherein the control unit is in electrical communication with the imaging sensor and the emitter;

wherein the control unit is configured to synchronize the emitter and the imaging sensor so as to produce a plurality of image frames; and

wherein the plurality of image frames comprise a luminance frame comprising luminance image data and a chrominance frame comprising chrominance data that are combined to form a color image;

wherein the image sensor comprises pixels having a plurality of pixel sensitivities;

wherein the pixel sensitivities comprise a long exposure and a short exposure;

wherein the control unit is configured to synchronize the emitter and the imaging sensor such that, for a single chrominance frame of the plurality of image frames, a first pulse of electromagnetic radiation in a first portion of an electromagnetic spectrum is pulsed at a start of the long exposure and is switched to a second pulse of electromagnetic radiation in a second portion of the electromagnetic spectrum is at a start of the short exposure is stopped at the start of the short exposure.

2. The system of claim 1, wherein the emitter comprises a plurality of sources that each emits a pulse of a portion of electromagnetic spectrum.

3. The system of claim 2, wherein the plurality of sources are configured to be actuated simultaneously.

4. The system of claim 2, wherein the plurality of sources are configured to produce a pulse of a predetermined interval.

5. The system of claim 1, wherein the pulse of electromagnetic radiation is modulated to provide luminance information according to color transformation coefficients that convert light energy from red, green, blue light energy space to luminance, chrominance blue, and chrominance red light energy space.

6. The system of claim 1, wherein the pulse of electromagnetic radiation is modulated to provide chrominance information according to color transformation coefficients that convert light energy from red, green, blue light energy space to luminance, chrominance blue, and chrominance red light energy space.

7. The system of claim 6, wherein the chrominance information is blue.

8. The system of claim 6, wherein the chrominance information is red.

9. The system of claim 1, wherein the emitter produces a pulsing pattern of luminance, chrominance blue, luminance, chrominance red.

10. The system of claim 1, wherein the emitter produces a pulsing pattern of luminance;

chrominance blue combined with chrominance red;

luminance; and

chrominance blue combined with chrominance red.

11. The system of claim 1, wherein the control unit is configured to use chrominance frames more than once to reconstruct resultant frames.

12. The system of claim 1, wherein a luminance coefficient is added to chrominance frames by an image signal processor and wherein the luminance coefficient is an integer that is a multiple of(½

)ⁿ, where n is a final system frame rate.

13. The system of claim 1, wherein the image sensor comprises uniform, monochromatic pixels configured to be read individually.

14. The system of claim 13, wherein the uniform, monochromatic pixels can be read after exposure to the plurality of pulses of electromagnetic radiation, wherein the plurality of pulses have a long exposure and a short exposure.

15. The system of claim 13, wherein the imaging sensor is a monochrome sensor.

16. The system of claim 1, wherein the image sensor is configured to produce a sequence of frames comprising:

a luminance frame of long exposure pixel data and short exposure pixel data,

a red chrominance frame of long exposure pixel data and short exposure pixel data, and

a blue chrominance frame of long exposure pixel data and short exposure pixel data.

17. The system of claim 16, wherein the emitter emits the plurality of pulses of electromagnetic radiation that comprise luminance, chrominance red, and chrominance blue, such that luminance is represented in the pattern twice as often as red chrominance and blue chrominance.

18. The system of claim 1, wherein a pulse of electromagnetic radiation emitted by the emitter is of a wavelength that is not visible to humans.