Camera system with minimal area monolithic CMOS image sensor

• US 10,075,626 B2
• Filed: 10/03/2016
• Issued: 09/11/2018
• Est. Priority Date: 07/26/2012
• Status: Active Grant

• Current Assignee: Depuy Synthes Products Incorporated (Johnson & Johnson)
• Original Assignee: Depuy Synthes Products Incorporated (Johnson & Johnson)
• Inventors: Blanquart, Laurent, Richardson, John
• Primary Examiner(s): Aynalem, Nathnael B
• Application Number:
  US15/284,085
• Publication Number:
  US 20170094139A1
• Time in Patent Office: 708 Days
• CPC Class Codes: A61B 1/000095 for image enhancement
• CPC Class Codes: A61B 1/051 Details of CCD assembly
• CPC Class Codes: A61B 1/3132 for laparoscopy
• CPC Class Codes: A61B 1/317 for bones or joints, e.g. osteoscopes, arthroscopes
• CPC Class Codes: A61B 5/0077 Devices for viewing the surface of the body, e.g. camera, magnifying lens
• CPC Class Codes: H04N 23/555 for picking-up images in sites, inaccessible due to their dimensions or hazardous conditions, e.g. endoscopes or borescopes
• CPC Class Codes: H04N 23/56 provided with illuminating means
• CPC Class Codes: H04N 23/72 Combination of two or more compensation controls
• CPC Class Codes: H04N 25/75 Circuitry for providing, modifying or processing image signals from the pixel array
• CPC Class Codes: H04N 25/76 Addressed sensors, e.g. MOS or CMOS sensors
• CPC Class Codes: H04N 25/78 Readout circuits for addressed sensors, e.g. output amplifiers or A/D converters
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

The disclosure extends to methods, systems, and computer program products for digitally imaging with area limited image sensors, such as within a lumen of an endoscope. The system includes an image sensor comprising a pixel array for sensing electromagnetic radiation. The pixel array comprises a plurality of pixel groups comprising only active pixels, a plurality of pixel groups comprising only optical black pixels, and a plurality of line readouts, wherein each line readout is configured to read out a single group of the plurality of pixel groups in the pixel array. The imaging system samples one or more of the plurality of pixel groups comprising only optical black pixels a plurality of times to reduce a size of the image sensor by reducing a number of optical black columns.

Claims

1. An imaging system comprising:

an image sensor comprising a pixel array for sensing electromagnetic radiation, the pixel array comprising;

a plurality of pixel groups comprising only active pixels, and

a plurality of pixel groups comprising only optical black pixels;

and a plurality of line readouts, wherein each line readout is configured to read out a single group of the plurality of pixel groups in the pixel array, wherein the imaging system samples one or more of the plurality of pixel groups comprising only optical black pixels a plurality of times to reduce a size of the image sensor by reducing a number of optical black columns.

2. The imaging system of claim 1, further comprising an endoscopic device for use in a closed light environment.

3. The imaging system of claim 2, further comprising an endoscope body comprising:

a holding structure;

a lumen attached by a lumen base at a first end of the body; and

a tip portion of the lumen opposite of the lumen base of the body.

4. The imaging system of claim 3, further comprising:

a lens disposed at the distal most portion of the tip portion; and

an image system disposed near the tip portion of the lumen, wherein the image system comprises the pixel array and the plurality of line readouts.

5. The imaging system of claim 1, further comprising a digitizer to convert analog pixel data into digital data before transmission along a length of the lumen.

6. The imaging system of claim 1, wherein the imaging system is configured to calibrate output from the pixel array based on pixel values gathered by the active pixels and the optical black pixels.

7. The imaging system of claim 1, wherein the plurality of pixel groups comprise a plurality of pixel columns, wherein the plurality of pixel columns comprising the optical black pixels are organized in columns adjacent to the plurality of pixel columns comprising the active pixels.

8. The imaging system of claim 1, wherein the imaging system samples the groups of optical black pixels a plurality of times to calculate line offsets.

9. The imaging system of claim 1, wherein the pixel array comprises no operational optical black rows.

10. The imaging system of claim 1, further comprising a black clamp circuit for providing offset control for the data generated by the pixel array.

11. The imaging system of claim 10, wherein the black clamp circuit is located remotely from the imaging system.

12. The imaging system of claim 11, further comprising a command interface to control the black clamp circuit.

13. The imaging system of claim 10, wherein the black clamp circuitry senses a voltage generated by a digital to analog converter.

14. The imaging system of claim 10, wherein the black clamp circuitry senses a voltage generated by a charge pump.

15. The imaging system of claim 10, wherein the plurality of columns are used to calculate a baseline black level for the black clamp circuit and black clamp calculations performed by the image signal processor.

16. The imaging system of claim 15, wherein the black clamp calculations are used to control the voltage offset prior to the digitizer and use a plurality of line offsets to determine the overall offset within a whole frame of data using simple exponential smoothing (SES).

17. The imaging system of claim 1, further comprising storage storing two-dimensional frame data for compensating for fixed pattern noise generated by the imaging system.

18. The imaging system of claim 17, wherein the two-dimensional frame data is derived from dark frame capture.

19. The imaging system of claim 18, wherein the dark frame capture is obtained during a break in pulsing by an emitter.

20. The imaging system of claim 18, wherein the pixel offset is computed by simple exponential smoothing for the dark frame capture.

21. The imaging system of claim 1, further comprising a long registry, wherein the long registry comprises one or more of:

control parameter entries for controlling exposure times of the pixel array and gain of the pixel array;

control parameter entries for controlling incremental offset adjustment of the pixel array or exposure times of the pixel array; and

control parameter entries for controlling incremental offset adjustment of the pixel array or gain of the pixel array.

22. The imaging system of claim 1, further comprising a plurality of control registers, wherein the control registers are digital latches that are loaded via shift registers, wherein one or more of:

the shift registers are arbitrary in length;

a majority of the plurality of control registers are loaded using shift registers that include many tens of bits; and

a majority of the plurality of control registers are loaded using shift registers that include hundreds of bits or more.

23. The imaging system of claim 1, wherein the imaging system comprises a plurality of pixel arrays, wherein the imaging system generates a three-dimensional image based on images from the plurality of pixel arrays.

24. The imaging system of claim 1, wherein the imaging system comprises a first substrate comprising the pixel array and a second substrate comprising supporting circuitry for the pixel array, wherein the second substrate comprising the supporting circuitry is located remotely from the first substrate comprising the pixel array.

25. A method comprising:

gathering pixel data from a first plurality of pixel groups comprising only active pixels, and

gathering pixel data from a second plurality of pixel groups comprising only optical black pixels;

wherein gathering the pixel data from the first plurality of pixel groups and the second plurality of pixel groups comprises gathering pixel data from a plurality of pixel groups of an image sensor, the image sensor comprising a pixel array comprising;

the first plurality of pixel groups;

the second plurality of pixel groups;

a plurality of line readouts, wherein each line readout is configured to read out a single group of the plurality of pixel groups in the pixel array, and sampling one or more of the second plurality of pixel groups comprising only optical black pixels a plurality of times to reduce a size of the image sensor by reducing a number of optical black columns.

26. The method of claim 25, wherein the image sensor receives instructions from a control unit comprising a processor, wherein the control unit is in electrical communication with the image sensor.

27. The method of claim 25, wherein the image sensor is located on a tip of a lumen in an endoscopic system for accessing the ambient light deficient environment.

28. The method of claim 25, further comprising converting analog pixel data into digital data before transmission along a length of a lumen in an endoscopic system.

29. The method of claim 25, further comprising calibrating output from the pixel array based on pixel values gathered by the active pixels and the optical black pixels.

30. The method of claim 25, wherein the plurality of pixel groups comprise a plurality of pixel columns, wherein the plurality of pixel columns comprising the optical black pixels are organized in columns adjacent to the plurality of pixel columns comprising the active pixels.

31. The method of claim 25, further comprising sampling groups of optical black pixels a plurality of times, whereby a number of optical black columns in the image sensor can be reduced to reduce horizontal space for the imaging system.

32. The method of claim 25, further comprising providing offset control for the data generated by the pixel array using a black clamp circuit.

33. The method of claim 32, further comprising one or more of:

sensing a voltage generated by a digital to analog converter using the black clamp circuitry;

sensing a voltage generated by a charge pump using the black clamp circuitry; and

calculating a baseline black level for the black clamp circuit and black clamp calculations performed by an image signal processor.

34. The method of claim 33, wherein the black clamp calculations are used to control the voltage offset prior to the digitizer and use a plurality of line offsets to determine the overall offset within a whole frame of data using simple exponential smoothing (SES).

35. The method of claim 25, further comprising storage storing two-dimensional frame data for compensating for fixed pattern noise generated by the imaging system.

36. The method of claim 35, further comprising capturing a dark frame, wherein the two-dimensional frame data is derived from dark frame.

37. The method of claim 36, wherein capturing the dark frame comprises capturing during a break in pulsing by an emitter.

38. The method of claim 37, further comprising computing the pixel offset by simple exponential smoothing based on the dark frame capture.

39. The method of claim 25, wherein the image sensor comprises a plurality of pixel arrays, the method further comprising generating a three-dimensional image based on images from the plurality of pixel arrays.