White balance and fixed pattern noise frame calibration using distal cap

• US 9,492,060 B2
• Filed: 03/15/2014
• Issued: 11/15/2016
• Est. Priority Date: 03/15/2013
• Status: Active Grant

• Current Assignee: Depuy Synthes Products Incorporated (Johnson & Johnson)
• Original Assignee: Depuy Synthes Products Incorporated (Johnson & Johnson)
• Inventors: Blanquart, Laurent
• Primary Examiner(s): Vaughn, Jr., William C
• Assistant Examiner(s): TOWE, JOSEPH DANIE A
• Application Number:
  US14/214,791
• Publication Number:
  US 20140267656A1
• Time in Patent Office: 976 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: A61B 1/00006 of control signals
• CPC Class Codes: A61B 1/000095 for image enhancement
• CPC Class Codes: A61B 1/00057 provided with means for testing or calibration
• CPC Class Codes: A61B 1/00137 End pieces at either end of the endoscope, e.g. caps, seals or forceps plugs
• CPC Class Codes: A61B 1/06 with illuminating arrangements
• CPC Class Codes: A61B 1/0655 Control therefor
• CPC Class Codes: A61B 5/24 Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 2209/044 using sequential colour illumination
• CPC Class Codes: H04N 23/10 for generating image signals from different wavelengths
• CPC Class Codes: H04N 23/555 for picking-up images in sites, inaccessible due to their dimensions or hazardous conditions, e.g. endoscopes or borescopes
• CPC Class Codes: H04N 23/56 provided with illuminating means
• CPC Class Codes: H04N 23/88 for colour balance, e.g. white-balance circuits or colour temperature control
• CPC Class Codes: H04N 25/673 by using reference sources

Abstract

The disclosure extends to methods, systems, and computer program products for producing an image in light deficient environments and correction of white balance and/or fixed pattern noise at startup or at any other time during a procedure.

Claims

1. A method for adjusting fixed pattern noise in a digital imaging process for use with an endoscope in ambient light deficient environments comprising:

controlling emissions of electromagnetic radiation using a controlled source of electromagnetic radiation, wherein the controlled source of electromagnetic radiation is an emitter that pulses a plurality of wavelengths of electromagnetic radiation;

continuously focusing a scene onto a pixel array;

sensing reflected electromagnetic radiation with said pixel array to create an image frame;

providing at startup a cap that fits over a lumen of the endoscope such that the cap is blocking external illumination to each pixel in said pixel array;

automatically initiating a calibration procedure by a camera control unit and performing correction steps at startup when the endoscope is connected to the camera control unit;

sensing the pixel array while the cap is in place with no applied electromagnetic radiation;

creating a dark frame reference for use in removing fixed pattern noise;

comparing the dark frame reference to the image frame; and

correcting the image frame by removing fixed pattern noise from an image using the dark frame reference.

2. The method of claim 1, wherein the method further comprises actuating an emitter to emit a pulse of a wavelength of electromagnetic radiation to cause illumination within the light deficient environment.

3. The method of claim 2, wherein said pulse is within a wavelength range that comprises a portion of the electromagnetic spectrum.

4. The method of claim 3, wherein said emitter is a laser emitter and wherein the method further comprises pulsing said laser emitter at a predetermined interval.

5. The method of claim 4, wherein the method further comprises actuating a pixel array at a sensing interval that corresponds to the pulse interval of said laser emitter.

6. The method of claim 1, wherein the dark frame is created from a single sensing of the pixel array while the cap is in place.

7. The method of claim 1, wherein a plurality of dark frames are created from a plurality of sensing the pixel array while the cap is in place.

8. The method of claim 7, wherein said dark frames are created upon startup of a system and stored within memory associated with the system.

9. The method of claim 1, wherein said dark frame is created after the pixel array is at operating temperature.

10. The method of claim 1, wherein the dark frame is created after a surgical procedure has begun.

11. The method of claim 10, wherein the dark frame comprises a plurality of dark frames that are created as part of the image frame stream by not applying electromagnetic radiation at given times and stored within memory associated with the system.

12. The method of claim 1, further comprising a plurality of caps that correspond and are opaque to the emitted electromagnetic radiation.

13. The method of claim 1, wherein a response of the pixel array corresponds to a photo-signal generated under controlled monochromatic radiation.

14. The method of claim 1, wherein a response of the pixel array corresponds to the photo-signal generated under a plurality of wavelengths of radiation.

15. The method of claim 1, wherein a response of the pixel array corresponds to the photo-signal generated under a continuous spectrum of wavelengths of radiation.

16. A system for digital imaging for use in ambient light deficient environments comprising:

a camera control unit;

a controlled source of electromagnetic radiation, wherein the controlled source is an emitter that emits a plurality of wavelengths of electromagnetic radiation;

an endoscope comprising a lumen and a CMOS image sensor, wherein the image sensor comprises a pixel array and that senses reflected electromagnetic radiation with said pixel array;

optics for continuously focusing a scene onto said pixel array;

a cap configured to cover a distal end of the lumen such the camera control unit initiates a calibration procedure for adjusting fixed pattern noise and white balance at startup when the endoscope is connected to the camera control unit.

17. The system of claim 16, wherein a dark frame is created from a single sensing of the pixel array while the cap is covering the distal end.

18. The system of claim 16, wherein the cap is sized and shaped to fit snuggly onto the lumen.

19. The system of claim 16, wherein the cap is made of a compliant material.

20. The system of claim 16, wherein the cap is opaque to the electromagnetic radiation emitted by an emitter.

21. The system of claim 16, wherein the endoscope is a reusable endoscopic device.

22. The system of claim 16, wherein the endoscope is a limited use endoscopic device.

23. The system of claim 16, wherein the endoscope is a re-posable use endoscopic device.

24. The system of claim 16, wherein the endoscope is a single-use endoscopic device.

25. A method for adjusting white balance in a digital imaging process for use with an endoscope in ambient light deficient environments comprising:

illuminating a scene using a controlled source of electromagnetic radiation, wherein the controlled source of electromagnetic radiation is an emitter that pulses a plurality of wavelengths of electromagnetic radiation;

continuously focusing the scene onto a pixel array;

sensing reflected electromagnetic radiation with said pixel array;

providing at startup a cap that fits over a lumen of the endoscope, such that the cap is blocking external illumination to each pixel in said pixel array;

automatically initiating a calibration procedure by a camera control unit and performing correction steps at startup when the endoscope is connected to the camera control unit;

sensing the pixel array while the cap is in place with applied electromagnetic radiation;

measuring a response from the pixel array with applied electromagnetic radiation and outputting a result in memory;

using the result to compute white balance coefficients; and

white balancing the system using the white balance coefficients.

26. The method of claim 25, further comprising a plurality of caps that correspond and are opaque to the emitted electromagnetic radiation.

27. The method of claim 25, wherein a response of the pixel array corresponds to a photo-signal generated under controlled monochromatic radiation.

28. The method of claim 25, wherein a response of the pixel array corresponds to the photo-signal generated under a plurality of wavelengths of radiation.

29. The method of claim 25, wherein a response of the pixel array corresponds to the photo-signal generated under a continuous spectrum of wavelengths of radiation.

30. The method of claim 25, wherein the endoscope is a reusable endoscopic device.

31. The method of claim 25, wherein the endoscope is a limited use endoscopic device.

32. The method of claim 25, wherein the endoscope is a re-posable use endoscopic device.

33. The method of claim 25, wherein the endoscope is a single-use endoscopic device.

34. The method of claim 25, wherein the method further comprises actuating a laser emitter to emit a pulse of a wavelength of electromagnetic radiation to cause illumination within the light deficient environment.

35. The method of claim 34, wherein said pulse is within a wavelength range that comprises a portion of the electromagnetic spectrum.

36. The method of claim 35, wherein the method further comprises pulsing said laser emitter at a predetermined interval.

37. The method of claim 36, wherein the method further comprises actuating a pixel array at a sensing interval that corresponds to the pulse interval of said laser emitter.