Imaging systems with visible light sensitive pixels and infrared light sensitive pixels

• US 10,136,107 B2
• Filed: 11/21/2013
• Issued: 11/20/2018
• Est. Priority Date: 11/21/2013
• Status: Active Grant

• Current Assignee: Semiconductor Components Industries LLC (ON Semiconductor Corporation)
• Original Assignee: Semiconductor Components Industries LLC (ON Semiconductor Corporation)
• Inventors: Keelan, Brian, Mlinar, Marko
• Primary Examiner(s): Beasley, Deirdre L
• Application Number:
  US14/086,705
• Publication Number:
  US 20150138366A1
• Time in Patent Office: 1,825 Days
• Field of Search: 348164
• CPC Class Codes: H04N 23/11 for generating image signals from visible and infrared light wavelengths
• CPC Class Codes: H04N 23/843 Demosaicing, e.g. interpolating colour pixel values
• CPC Class Codes: H04N 25/131 including elements passing infrared wavelengths
• CPC Class Codes: H04N 25/133 including elements passing panchromatic light, e.g. filters passing white light
• CPC Class Codes: H04N 25/135 based on four or more different wavelength filter elements
• CPC Class Codes: H04N 9/64 Circuits for processing colour signals H04N9/77 takes precedence; camera processing pipelines for processing colour signals H04N23/84

Abstract

An image sensor may have an array of image pixels arranged in color filter unit cells that each have at least one red image pixel that generates red image signals, at least one blue image pixel that generate blue image signals, at least one clear image pixels that generate clear image signals, at least one infrared image pixel that generates infrared image signals, and optionally at least one green image pixel that generates green image signals. The image sensor may be coupled to processing circuitry that performs chroma demosaicking operations on the image signals. The processing circuitry may generate an infrared image using the infrared image signals and a luminance value using the clear, red, blue, and infrared image signals. The processing circuitry may perform point filter operations on the image signals based on the generated luminance value to produce corrected visible light image signals having improved image quality.

Claims

1. An imaging system, comprising:

processing circuitry; and

an array of physical image sensor pixels, wherein the array of physical image sensor pixels comprises;

a first group of physical image sensor pixels configured to generate first image signals in response to light of a first color, wherein the first color is selected from a group consisting of;

red, green, and blue;

a second group of physical image sensor pixels configured to generate second image signals in response to light of a second color that is different from the first color, wherein the second color is selected from a group consisting of;

red, green, and blue;

a third group of physical image sensor pixels configured to generate third image signals in response to light of at least two colors, wherein the at least two colors are selected from a group consisting of;

red, green, and blue; and

a group of infrared physical image sensor pixels configured to generate infrared image signals in response to only infrared light, wherein the array of physical image sensor pixels comprises a plurality of repeating unit cells of physical image sensor pixels, wherein each unit cell in the plurality of unit cells comprises eight physical image sensor pixels arranged in first and second rows, the first row comprises a first pair of physical image sensor pixels from the third group and first and second infrared physical image sensor pixels from the group of infrared physical image sensor pixels, and the second row comprises a second pair of physical image sensor pixels from the third group, a physical image sensor pixel from the first group, and a physical image sensor pixel from the second group, and wherein the processing circuitry is configured to use the infrared image signals to correct the first, second, and third image signals.

2. The imaging system defined in claim 1, wherein each unit cell in the plurality of unit cells comprises four consecutive columns in the array, the first infrared physical image sensor pixel is formed in the second column, and the second infrared physical image sensor pixel is formed in the fourth column.

3. The imaging system defined in claim 1, wherein the first group of physical image sensor pixels comprises red physical image sensor pixels, and wherein the second group of physical image sensor pixels comprises blue physical image sensor pixels.

4. The imaging system defined in claim 1, wherein the processing circuitry is configured to perform filtering operations on the first, second, and third image signals that increase noise correlations associated with first, second, and third image signals.

5. The imaging system defined in claim 1, wherein the first group of physical image sensor pixels comprises red physical image sensor pixels, wherein the second group of physical image sensor pixels comprises blue physical image sensor pixels, and wherein the third group of physical image sensor pixels comprises clear physical image sensor pixels having clear color filter elements.

12. The imaging system defined in claim 2, wherein the physical image sensor pixel from the first group is formed in the first column and the image sensor pixel from the second group is formed in the third column.

13. The imaging system defined in claim 12, wherein the first pair of physical image sensor pixels from the third group comprises a first physical image sensor pixel formed in the first column and a second physical image sensor pixel formed in the third column and wherein the second pair of physical image sensor pixels from the third group comprises a third physical image sensor pixel formed in the second column and a fourth image sensor pixel formed in the fourth column.

6. A method of generating images with an imaging system, wherein the imaging system comprises an array of physical image sensor pixels arranged in a plurality of repeating unit cells of physical image sensor pixels and processing circuitry, wherein each unit cell in the plurality of unit cells comprises first, second, third, and fourth physical image sensor pixels in a first row and fifth, sixth, seventh, and eighth physical image sensor pixels in a second row that is adjacent to the first row, the method comprising:

with the second physical image sensor pixel, capturing first image signals in response to light of a first color;

with the fifth physical image sensor pixel, capturing second image signals in response to light of a second color that is different from the first color;

with the seventh physical image sensor pixel, capturing third image signals in response to light of a third color that is different from the first and second colors;

with the first, third, sixth, and eighth physical image sensor pixels, capturing fourth image signals in response to light of at least two colors selected from the group of colors consisting of the first color, the second color, and the third color;

with the fourth physical image sensor pixel, capturing infrared image signals in response to only infrared light;

with the processing circuitry, performing chroma demosaicking operations on the captured first, second, third, and fourth image signals that increase noise correlations associated with the first, second, third, and fourth image signals such that noise fluctuations in the first, second, third, and fourth image signals increase and decrease together in a correlated manner to form first, second, third, and fourth sets of image data;

with the processing circuitry, generating fourth image signals of a fourth color using the infrared image signals;

with the processing circuitry, combining the first image signals, the second image signals, the third image signals, and the fourth image signals to form a first luminance value;

with the processing circuitry, computing a second luminance value based on the first image signals, the second image signals, the third image signals, and the fourth image signals; and

with the processing circuitry, modifying the first image signals, the second image signals, the third image signals, and the fourth image signals using the first and second luminance values.

7. The method defined in claim 6, further comprising:

with the processing circuitry, performing infrared light contamination subtraction operations on the first, second, third, and fourth sets of image data.

8. The method defined in claim 6, further comprising:

with the processing circuitry, applying a color correction matrix to the first, second, third, and fourth sets of image data to generate standard red, standard green, and standard blue image data; and

with the processing circuitry, performing point filter operations on the standard red, standard green, and standard blue image data.

9. The method defined in claim 8, further comprising:

with the processing circuitry, performing the point filter operations on the standard red, standard green, and standard blue image data based on the generated first and second luminance values.

10. A system, comprising:

a central processing unit;

memory;

input-output circuitry; and

an imaging device, wherein the imaging device comprises;

an array of image pixels arranged in rows and columns, wherein the array of image pixels comprises a first group of image pixels configured to generate first pixel values in response to light of a first color, a second group of image pixels configured to generate second pixel values in response to light of a second color, a group of infrared image pixels configured to generate infrared pixel values in response to only infrared light, and a group of broadband image pixels configured to generate broadband pixel values in response to light of at least two colors, wherein the group of broadband image pixels is arranged in a checkerboard pattern on the array of image pixels such that each row and each column in the array comprises at least one broadband image pixel from the group of broadband image pixels.

11. The system defined in claim 10, wherein the at least two colors are selected from a group consisting of:

the first color, the second color, and a third color that is different from the first and second colors.

14. The imaging system defined in claim 10, wherein the array of image pixels comprises a plurality of repeating four-pixel by four-pixel unit cells of image sensor pixels, and wherein each unit cell in the plurality of unit cells comprises eight broadband image pixels, two infrared image pixels, two image pixels from the first group, two image pixels from the second group, and two image pixels from a third group of image pixels configured to generate third pixel values in response to light of a third color that is different from the first and second colors.

15. The system defined in claim 14, wherein each unit cell includes consecutive first, second, third, and fourth rows and consecutive first, second, third, and fourth columns in the array, wherein a first of the two infrared image pixels is formed in the third row and second column, a second of the two infrared image pixels is formed in the first row and fourth column, a first of the two image pixels from the first group is formed in the second row and first column, a second of the two image pixels from the first group is formed in the fourth row and third column, a first of the two image pixels from the second group is formed in the second row and third column, a second of the two image pixels from the second group is formed in the fourth row and first column, a first of the two image pixels from the third group is formed in the first row and second column, and a second of the two image pixels from the third group is formed in the third row and fourth column.

16. The system defined in claim 15, wherein the first group of image pixels comprises red image sensor pixels, the second group of image pixels comprises blue image pixels, and the third group of image pixels comprises green image pixels.

17. The system defined in claim 10, wherein the first color is red, the second color is blue, and the at least two colors are selected from a group consisting of:

red, green, and blue.

18. The system defined in claim 10, wherein the first group of image pixels, the second group of image pixels, the group of infrared image pixels, and the group of broadband image pixels are physical image pixels.

19. The system defined in claim 10, wherein the central processing unit is configured to use the infrared pixel values to correct the first, second, and broadband pixel values.