Camera system with minimal area monolithic CMOS image sensor

• US 10,701,254 B2
• Filed: 09/10/2018
• Issued: 06/30/2020
• Est. Priority Date: 07/26/2012
• Status: Active Grant

• Current Assignee: Depuy Synthes Products Incorporated (Johnson & Johnson)
• Original Assignee: Depuy Synthes Products Incorporated (Johnson & Johnson)
• Inventors: Blanquart, Laurent, Richardson, John
• Primary Examiner(s): Pontius, James M
• Application Number:
  US16/127,015
• Publication Number:
  US 20190007588A1
• Time in Patent Office: 659 Days
• CPC Class Codes: A61B 1/000095 for image enhancement
• CPC Class Codes: A61B 1/051 Details of CCD assembly
• CPC Class Codes: A61B 1/3132 for laparoscopy
• CPC Class Codes: A61B 1/317 for bones or joints, e.g. osteoscopes, arthroscopes
• CPC Class Codes: A61B 5/0077 Devices for viewing the surface of the body, e.g. camera, magnifying lens
• CPC Class Codes: H04N 23/555 for picking-up images in sites, inaccessible due to their dimensions or hazardous conditions, e.g. endoscopes or borescopes
• CPC Class Codes: H04N 23/56 provided with illuminating means
• CPC Class Codes: H04N 23/72 Combination of two or more compensation controls
• CPC Class Codes: H04N 25/75 Circuitry for providing, modifying or processing image signals from the pixel array
• CPC Class Codes: H04N 25/76 Addressed sensors, e.g. MOS or CMOS sensors
• CPC Class Codes: H04N 25/78 Readout circuits for addressed sensors, e.g. output amplifiers or A/D converters
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

The disclosure extends to methods, systems, and computer program products for digitally imaging with area limited image sensors, such as within a lumen of an endoscope.

Claims

1. A method of calibrating an image sensor, comprising:

providing an image sensor having a first optical black portion and a second optical black portion separated from each other by an active pixel portion;

sampling one or more pixels in the first optical black portion and adding the results to an accumulator;

determining, in response to sampling one or more pixels in the first optical black portion, a black level for the first optical black portion;

comparing the determined black level to a target black level by subtracting the target black level from the determined black level to determine the overall black level for the first optical black portion;

determining whether the overall black level is greater than a threshold black level, and when the overall black level is greater than the threshold black level, applying a proportional adjustment to a blackclamp voltage in the image sensor, and when the overall black level is less than the threshold black level, applying an incremental adjustment to the blackclamp voltage in the image sensor.

2. The method of calibrating the image sensor of claim 1, wherein the image sensor is a CMOS image sensor.

3. The method of calibrating the image sensor of claim 1, wherein calibrating the image sensor includes calibrating the black clamp voltage.

4. The method of calibrating the image sensor of claim 1, wherein calibrating the image sensor includes calibrating the line noise for each column of pixels in the first optical black portion.

5. The method of calibrating the image sensor of claim 1, wherein sampling the one or more pixels in the first optical black portion includes determining a median black offset measurement for each one of the one or more pixels in the first optical black portion and a plurality of other pixels proximate to the each one of the one or more pixels in the first optical black portion.

6. The method of calibrating the image sensor of claim 5, wherein the median black offset measurement for each one of the one or more pixels in the first optical black portion is averaged to provide a final line offset estimate.

7. The method of calibrating the image sensor of claim 1, further comprising:

sampling one or more pixels in the second optical black portion and adding the results to an accumulator;

determining, in response to sampling one or more pixels in the second optical black portion, a black level for the second optical black portion;

comparing the determined black level to a target black level by subtracting the target black level from the determined black level to determine the overall black level for the second optical black portion;

determining whether the overall black level for the first optical black portion and the second optical black portion is greater than a threshold black level, and when the overall black level is greater than the threshold black level, applying a proportional adjustment to a blackclamp voltage in the image sensor, and when the overall black level is less than the threshold black level, applying an incremental adjustment to the blackclamp voltage in the image sensor.

8. The method of calibrating the image sensor of claim 7, wherein sampling the one or more pixels in the second optical black portion includes determining a median black offset measurement for each one of the one or more pixels in the second optical black portion and a plurality of other pixels proximate to the each one of the one or more pixels in the first optical black portion.

9. The method of calibrating the image sensor of claim 8, wherein sampling the one or more pixels in the second optical black portion includes wrapping around a plurality of pixels at an end of the second optical black portion to a beginning of the first optical black portion.

10. The method of calibrating the image sensor of claim 9, further comprising:

averaging the median black offset measurement for each one of the one or more pixels in the second optical black portion to provide a final line offset estimate.

11. The method of calibrating the image sensor of claim 9, wherein the wrap around causes a delay equal to at least the total number of optical black pixels per analog-to-digital converter channel, per row of pixels.

12. The method of calibrating the image sensor of claim 9, wherein the total number of optical black pixels per analog-to-digital converter channel affected by the delay are ignored.

13. The method of calibrating the image sensor of claim 7, further comprising:

determining line offsets for the first optical black portion and the second optical black portion.

14. The method of calibrating the image sensor of claim 13, further comprising:

determining an overall black level by incrementally adjusting a running estimate.

15. The method of calibrating the image sensor of claim 14, further comprising updating the running estimate when the sample is divisible by a binary number (2^q).

16. The method of calibrating the image sensor of claim 15, wherein the updating the running estimate includes multiplying a previous estimate by (2^q−

1)/2^q where q is a tunable parameter.

17. The method of calibrating the image sensor of claim 7, wherein the proportional adjustment corresponds to more than one digital-to-analog converter count.

18. The method of calibrating the image sensor of claim 7, wherein the incremental adjustment corresponds to one digital-to-analog converter count.

19. The method of calibrating the image sensor of claim 7, wherein the blackclamp voltage is applied when optical black pixels from the first optical black portion or the second optical black portion are in a digital readout path for the image sensor.

20. The method of calibrating the image sensor of claim 19, wherein the blackclamp voltage applied to the optical black pixels in the digital readout path reduces line noise.