IMAGING SYSTEMS WITH VISIBLE LIGHT SENSITIVE PIXELS AND INFRARED LIGHT SENSITIVE PIXELS

• US 20150138366A1
• Filed: 11/21/2013
• Published: 05/21/2015
• Est. Priority Date: 11/21/2013
• Status: Active Grant

• Current Assignee: Semiconductor Components Industries LLC (ON Semiconductor Corporation)
• Original Assignee: Aptina Imaging Corporation (ON Semiconductor Corporation)
• Inventors: Keelan, Brian, Mlinar, Marko
• Granted Patent: US 10,136,107 B2
• Time in Patent Office: Days
• US Class Current: 348/164
• CPC Class Codes: H04N 23/11 for generating image signals from visible and infrared light wavelengths
• CPC Class Codes: H04N 23/843 Demosaicing, e.g. interpolating colour pixel values
• CPC Class Codes: H04N 25/131 including elements passing infrared wavelengths
• CPC Class Codes: H04N 25/133 including elements passing panchromatic light, e.g. filters passing white light
• CPC Class Codes: H04N 25/135 based on four or more different wavelength filter elements
• CPC Class Codes: H04N 9/64 Circuits for processing colour signals H04N9/77 takes precedence; camera processing pipelines for processing colour signals H04N23/84

Abstract

An image sensor may have an array of image pixels arranged in color filter unit cells that each have at least one red image pixel that generates red image signals, at least one blue image pixel that generate blue image signals, at least one clear image pixels that generate clear image signals, at least one infrared image pixel that generates infrared image signals, and optionally at least one green image pixel that generates green image signals. The image sensor may be coupled to processing circuitry that performs chroma demosaicking operations on the image signals. The processing circuitry may generate an infrared image using the infrared image signals and a luminance value using the clear, red, blue, and infrared image signals. The processing circuitry may perform point filter operations on the image signals based on the generated luminance value to produce corrected visible light image signals having improved image quality.

Claims

1. An imaging system, comprising:

an array of image sensor pixels, wherein the array of image sensor pixels comprises;

a first group of image sensor pixels configured to generate first image signals in response to light of a first color;

a second group of image sensor pixels configured to generate second image signals in response to light of a second color that is different from the first color;

a third group of image sensor pixels configured to generate third image signals in response to light of at least two colors; and

a group of infrared image sensor pixels configured to generate infrared image signals in response to infrared light.

2. The imaging system defined in claim 1, wherein the at least two colors are selected from a group consisting of:

red, green, and blue.

3. The imaging system defined in claim 2, wherein the array of image sensor pixels comprises a plurality of repeating unit cells of image sensor pixels, wherein each unit cell in the plurality of unit cells comprises a first image sensor pixel from the first group of image sensor pixels, a second image sensor pixel from the second group of image sensor pixels, a third image sensor pixel from the third group of image sensor pixels, and an infrared image sensor pixel from the group of infrared image sensor pixels.

4. The imaging system defined in claim 3, wherein each unit cell in the plurality of unit cells comprises an additional infrared image sensor pixel from the group of infrared image sensor pixels.

5. The imaging system defined in claim 4, wherein each unit cell in the plurality of unit cells comprises eight image sensor pixels arranged in first and second rows, wherein the first row comprises a first pair of image sensor pixels from the third group, and wherein the second row comprises a second pair of image sensor pixels from the third group.

6. The imaging system defined in claim 4, wherein each unit cell in the plurality of unit cells comprises eight image sensor pixels arranged in first and second columns, wherein the first column comprises a first pair of image sensor pixels from the third group, and wherein the second column comprises a second pair of image sensor pixels from the third group.

7. The imaging system defined in claim 3, wherein the array of image sensor pixels further comprises:

a fourth group of image sensor pixels configured to generate fourth image signals in response to light of a third color that is different from the first and second colors, and wherein each unit cell in the plurality of unit cells further comprises;

a fourth image sensor pixel from the fourth group of image sensor pixels.

8. The imaging system defined in claim 7, wherein the first image sensor pixel comprises a red image sensor pixel, wherein the second image sensor pixel comprises a blue image sensor pixel, wherein the fourth image sensor pixel comprises a green image sensor pixel, and wherein each unit cell in the plurality of unit cells comprises three additional image sensor pixels from the third group of image sensor pixels.

9. The imaging system defined in claim 7, wherein the first image sensor pixel comprises a red image sensor pixel, wherein the second image sensor pixel comprises a blue image sensor pixel, wherein the fourth image sensor pixel comprises a green image sensor pixel, wherein each unit cell in the plurality of unit cells further comprises:

an additional red image sensor pixel;

an additional blue image sensor pixel;

an additional green image sensor pixel;

an additional infrared image sensor pixel; and

seven additional image sensor pixels from the third group of image sensor pixels.

10. The imaging system defined in claim 9, wherein each unit cell in the plurality of unit cells comprises sixteen image sensor pixels arranged in first, second, third, and fourth rows, wherein each of the first, second, third, and fourth rows of image sensor pixels comprises a respective pair of image sensor pixels from the third group of image sensor pixels.

11. The imaging system defined in claim 7, further comprising:

processing circuitry, wherein the processing circuitry is configured to perform filtering operations on the first, second, third, and fourth image signals that increase noise correlations associated with first, second, third, and fourth image signals.

12. The imaging system defined in claim 3, wherein the first image sensor pixel comprises a red image sensor pixel, wherein the second image sensor pixel comprises a blue image sensor pixel, and wherein the third image sensor pixel comprises a clear image sensor pixel.

13. A method of generating images with an imaging system, wherein the imaging system comprises an array of image sensor pixels arranged in a plurality of repeating unit cells of image sensor pixels, wherein each unit cell in the plurality of unit cells comprises a first, second, third, and fourth image sensor pixel, the method comprising:

with the first image sensor pixel, capturing first image signals in response to light of a first color;

with the second image sensor pixel, capturing second image signals in response to light of a second color that is different from the first color;

with the third image sensor pixel, capturing third image signals in response to light of at least two of the first color, the second color, and a third color that is different from the first and second colors; and

with the fourth image sensor pixel, capturing infrared image signals in response to infrared light.

14. The method defined in claim 13, wherein the imaging system further comprises processing circuitry, the method further comprising:

with the processing circuitry, performing chroma demosaicking operations on the captured first, second, and third image signals to form first, second, and third sets of image data.

15. The method defined in claim 14, further comprising:

with the processing circuitry, performing infrared light contamination subtraction operations on the first, second, and third sets of image data.

16. The method defined in claim 14, further comprising:

with the processing circuitry, applying a color correction matrix to the first, second, and third sets of image data to generate standard red, standard green, and standard blue image data; and

with the processing circuitry, performing point filter operations on the standard red, standard green, and standard blue image data.

17. The method defined in claim 16, further comprising:

with the processing circuitry, computing a weighted sum of the captured first, second, third and infrared image signals to generate an original luminance value; and

with the processing circuitry, performing the point filter operations on the standard red, standard green, and standard blue image data based on the generated original luminance value.

18. The method defined in claim 13, further comprising:

with the processing circuitry, performing demosaicking operations on the infrared image signals based on at least the first, second, and third image signals.

19. A system, comprising:

a central processing unit;

memory;

input-output circuitry; and

an imaging device, wherein the imaging device comprises;

an array of image pixels arranged in rows and columns, wherein the array of image pixels comprises a first group of image pixels configured to generate first pixel values in response to light of a first color, a second group of image pixels configured to generate second pixel values in response to light of a second color, a group of infrared image pixel configured to generate infrared pixel values in response to infrared light, and a group of clear image pixels configured to generate clear pixel values in response to light of at least two colors.

20. The system defined in claim 19, wherein the group of clear image pixels is arranged in a checkerboard pattern on the array of image pixels and wherein each row and each column in the array comprises at least one clear image pixel from the group of clear image pixels.

21. The system defined in claim 20, wherein the at least two colors are selected from a group consisting of:

the first color, the second color, and a third color that is different from the first and second colors.