Wide dynamic range operations for imaging

• US 7,446,812 B2
• Filed: 01/13/2004
• Issued: 11/04/2008
• Est. Priority Date: 01/13/2004
• Status: Active Grant

• Current Assignee: Micron Technology, Inc.
• Original Assignee: Micron Technology, Inc.
• Inventors: Nakamura, Junichi, Ando, Haruhisa
• Primary Examiner(s): Ye; Lin
• Assistant Examiner(s): Hernández; Nelson D
• Application Number:
  US10/755,411
• Publication Number:
  US 20050151866A1
• Time in Patent Office: 1,757 Days
• Field of Search: 348/297, 3482221-2301, 348234-239, 348362-368, 348272-273, 348/367
• US Class Current: 348/362
• CPC Class Codes: H04N 23/741 by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors
• CPC Class Codes: H04N 23/843 Demosaicing, e.g. interpolating colour pixel values
• CPC Class Codes: H04N 25/134 based on three different wavelength filter elements
• CPC Class Codes: H04N 25/583 with different integration times
• CPC Class Codes: H04N 25/70 SSIS architectures; Circuits associated therewith

Abstract

Embodiments provide a method and apparatus that achieve wide dynamic range operation of an image sensor. In an array of pixel cells, first charge is accumulated in a first subset of pixel cells during a first integration period and second charge is accumulated in a second subset of pixel cells during a second integration period. A length of the first integration period is different than a length of the second integration period, and the first and second charge are accumulated during a same frame and are read out.

Claims

1. A method of operating an image sensor comprising an array of pixel cells, the method comprising:

arranging the pixel cells in rows and subgroups, each subgroup including two adjacent rows of pixel cells,

arranging the pixel cells into first and second subsets such that the first subset includes every other subgroup beginning with an uppermost row, the second subset includes every other subgroup beginning with a third row, such that the first and second subsets do not include same subgroups;

accumulating first charge in a first subset of pixel cells during a first integration period;

accumulating second charge in a second subset of pixel cells during a second integration period, a length of the first integration period being different than a length of the second integration period, the pixel cells accumulating only first or second charge per frame;

reading out the accumulated first and second charge; and

further comprising the act of producing respective synthetic signals corresponding to each pixel cell of the array using results of the readout accumulated first and second charge, wherein producing respective synthetic signals comprises applying the equation R(i,j)=S(i,j)+S(i,j+2) to produce synthetic signals corresponding to respective pixel cells of the first subset and applying the equation R(i,j)=S(i,j)+S(i,j−

2) to produce synthetic signals corresponding to respective pixel cells of the second subset, where i is a column number of a pixel cell, j is a row number of the pixel cell, S is an original signal for a pixel cell, and R is the synthesized signal corresponding to a pixel cell.

2. The method of claim 1, further comprising the act of varying the lengths of the first and second integration periods depending upon an intensity of light incident on the array.

3. The method of claim 1, further comprising the act of using results of the readout accumulated first and second charge to determine an aperture condition.

4. The method of claim 1, wherein color filters arranged in a Bayer pattern are provided, wherein each pixel cell is associated with a respective color filter, and wherein R(i,j) is the sum of original signals for pixel cells associated with color filters corresponding to a same color.

5. The method of claim 1, further comprising simultaneously resetting respective photo-conversion devices of each pixel cell to a predetermined voltage prior to the acts of accumulating first charge and accumulating second charge.

6. A method of operating an image sensor comprising an array of pixel cells, the method comprising:

providing color filters corresponding to a plurality of colors, there being at least twice the number of color filters corresponding to a first color than to a second color, each color filter being associated with a respective pixel cell;

arranging the pixel cells into first and second subsets, wherein the act of arranging the pixel cells into first and second subsets comprises arranging the pixel cells such that the first subset includes pixel cells associated with color filters corresponding to the second color and a first half of the pixel cells associated with color filters corresponding to the first color and the second subset includes a second half of the pixel cells associated with color filters corresponding to the first color;

accumulating first charge in a first subset of pixel cells during a first integration period;

accumulating second charge in a second subset of pixel cells during a second integration period, a length of the first integration period being different than a length of the second integration period, the pixel cells accumulating only first or second charge per frame;

reading out the accumulated first and second charge; and

further comprising the act of producing respective synthetic signals corresponding to each pixel cell of the array using results of the readout accumulated first and second charge, wherein producing respective synthetic signals comprises using chrominance information from at least a portion of pixel cells of the first subset to obtain chrominance information for at least a portion of pixel cells of the second subset and using luminance information from at least a portion of pixel cells of the second subset to obtain luminance information for at least a portion of pixel cells of the first subset.

7. The method of claim 6, further comprising the act of arranging the color filters in a Bayer pattern.

8. A method of operating an image sensor comprising an array of pixel cells, the method comprising:

arranging the pixel cells in columns, rows, and first and second subsets, the first and second subsets including subgroups of pixel cells, the subgroups each including two adjacent rows of pixel cells, the first subset including every other subgroup and the second subset including subgroups not in the first subset;

accumulating first charge in a first subset of pixel cells during a first integration period;

accumulating second charge in a second subset of pixel cells during a second integration period, a length of the first integration period being longer than a length of the second integration period, the pixel cells accumulating only first or second charge per frame;

reading out the accumulated first and second charge; and

producing respective synthetic signals corresponding to each pixel cell of the array using results of the read out accumulated first and second charge, wherein producing respective synthetic signals comprises applying the equation R(i,j)=A*S(i,j)+B*(S(i,j−

2)+S(i,j+2)) to produce respective synthetic signals corresponding to pixel cells of the first subset and applying the equation R(i,j)=C*S(i,j)+D*(S(i,j−

2)+S(i,j+2)) to produce synthetic signals corresponding to respective pixel cells of the second subset, where A, B, C and D are numerical parameters between 1 and 0, i is a column number of a pixel cell, j is a row number of a pixel cell, S is an original signal for a pixel cell, and R is the synthesized signal corresponding to a pixel cell.

9. A method of operating an image sensor comprising an array of pixel cells, the method comprising:

arranging the pixel cells in columns, rows, and first and second subsets, the subsets including subgroups;

providing color filters arranged in a Bayer pattern, each pixel cell being associated with a respective color filter, a subgroup including pixel cells of a Bayer pattern unit, the first subset including every other subgroup, the second subset including subgroups not in the first subset, such that the subsets form a checker board-like pattern;

accumulating first charge in a first subset of pixel cells during a first integration period;

accumulating second charge in a second subset of pixel cells during a second integration period, a length of the first integration period being longer than a length of the second integration period, the pixel cells accumulating only first or second charge per frame;

reading out the accumulated first and second charge; and

producing respective synthetic signals corresponding to each pixel cell of the array using results of the read out accumulated first and second charge, wherein producing respective synthetic signals comprises applying the equation R(i,j)=S(i,j)+S(i,j+2) for a sequence j=0, 1, 4, 5, 8, 9. . . n, n+1, and the equation R(i,j)=S(i,j)+S(i,j−

2) for a sequence j=2, 3, 6, 7, 10, 11 . . . n′

, n′

+1 to produce synthetic signals corresponding respective pixel cells, where i is a column number of a pixel cell, j is a row number of a pixel cell, n and n′

are row numbers consistent with the respective sequences, S is an original signal for a pixel cell, and R is the synthesized signal corresponding to a pixel cell.

10. A method of operating an image sensor comprising an array of pixel cells, the method comprising:

accumulating first charge in a first subset of pixel cells during a first integration period;

accumulating second charge in a second subset of pixel cells during a second integration period, a length of the first integration period being different than a length of the second integration period, the pixel cells accumulating only first or second charge per frame;

reading out the accumulated first and second charge;

applying a first equation R(i,j)=S(i,j)+S(i,j+1) for a sequence j=0, 2, 4, 6, . . . n, n+2 to produce synthetic signals corresponding to respective pixel cells;

and applying a second equation R(i,j)=S(i,j)+S(i,j−

1) for a sequence j=1, 3, 5, 7, . . . n′

, n′

+2 to produce synthetic signals corresponding to respective pixel cells, where i is a column number of a pixel cell, j is a row number of a pixel cell, n and n′

are row numbers consistent with the respective sequences, S is an original signal for a pixel cell, and R is the synthesized signal corresponding to a pixel cell.

11. A method of operating an image sensor comprising an array of pixel cells, the method comprising:

accumulating first charge in a first subset of pixel cells during a first integration period;

accumulating second charge in a second subset of pixel cells during a second integration period, a length of the first integration period being longer than a length of the second integration period, the pixel cells accumulating only first or second charge per frame;

reading out the accumulated first and second charge;

applying a first equation R(i,j)=A*S(i,j)+B*(S(i,j−

1)+S(i,j+1)) for a sequence j=0,2,4,6, . . . n, n+2 to produce synthetic signals corresponding respective pixel cells; and

applying a second equation R(i,j)=C*S(i,j)+D*(S(i,j−

1)+S(i,j+1)) for a sequence j=1,3,5,7, . . . n′

, n′

+2 to produce synthetic signals corresponding respective pixel cells, where A, B, C and D are numerical parameters between 1 and 0, i is a column number of a pixel cell, j is a row number of a pixel cell, n and n′

are row numbers consistent with the respective sequences, S is an original signal for a pixel cell, and R is the synthesized signal corresponding to a pixel cell.

12. A method of operating an image sensor comprising an array of pixel cells, the method comprising:

arranging the pixel cells in columns, rows, and first and second subsets, the first subgroup including every other pixel cell in a row and every other pixel cell in a column, the second subset including the pixel cells not in the first subset, the first and second subsets forming a checker board-like pattern;

accumulating first charge in the first subset of pixel cells during a first integration period;

accumulating second charge in the second subset of pixel cells during a second integration period, a length of the first integration period being longer than a length of the second integration period, the pixel cells accumulating only first or second charge per frame;

reading out the accumulated first and second charge; and

producing respective synthetic signals corresponding to each pixel cell of the array using results of the read out accumulated first and second charge, wherein producing respective synthetic signals comprises either;

applying the equation R(ij)=S(i,j)+S(i,j+1) for a sequence j=0, 2, 4, 6, . . . n, n+2 and the equation R(i,j)=S(i,j)+S(i,j−

1) for a sequence j=1,3,5,7, . . . n′

, n′

+2 to produce synthetic signals corresponding respective pixel cells, where i is a colunm number of a pixel cell, j is a row number of a pixel cell, n and n′

are row numbers consistent with the respective sequences, S is an original signal for a pixel cell, and R is the synthesized signal corresponding to a pixel cell;

or

applying the equations R(i,j)=A*S(i,j)+B*(S(i,j−

1)+S(i,j+1)) to produce synthetic signals corresponding respective pixel cells of the first subset and R(i,j)=C*S(i,j)+D*(S(i,j−

1) +S(i,j+1)) to produce synthetic signals corresponding respective pixel cells of the second subset, where A, B, C and D are numerical parameters between 1 and 0, i is a column number of a pixel cell, j is a row number of a pixel cell, S is an original signal for a pixel cell, and R is the synthesized signal corresponding to a pixel cell.

13. A method of operating an image sensor comprising an array of pixel cells, the method comprising:

providing color filters arranged in a Bayer pattern, each pixel cell being associated with a respective color filter;

accumulating first charge in a first subset of pixel cells during a first integration period;

accumulating second charge in a second subset of pixel cells during a second integration period, a length of the first integration period being longer than a length of the second integration period, the pixel cells accumulating only first or second charge per frame, wherein the first subset includes the pixel cells associated with the non-redundant color filters of each Bayer pattern unit, and wherein the second subset includes the pixel cells associated with the redundant color filters of each Bayer pattern unit;

reading out the accumulated first and second charge; and

producing respective synthetic signals corresponding to at least a portion of the pixel cells using results of the read out accumulated first and second charge, wherein the synthetic signals are calculated using chrominance information from pixel cells of the first subset to obtain chrominance information for pixel cells of the second subset and using luminance information from pixel cells of the second subset to obtain luminance information for pixel cells of the first subset.

14. An image sensor comprising:

an array of pixel cells;

a color filter array comprising color filters corresponding to a plurality of colors, the color filter array having at least twice the number of color filters corresponding to a first color than to a second color, wherein each pixel cell is associated with a respective color filter;

timing and control circuitry in communication with the array, the timing and control circuitry configured to cause first charge to be accumulated in a first subset of pixel cells during a first integration period and second charge to be accumulated in a second subset of pixel cells during a second integration period, a length of the first integration period being different than a length of the second integration period, the pixel cells accumulating only first or second charge per frame, wherein the first subset includes pixel cells associated with color filters corresponding to the second color and a first half of the pixel cells associated with color filters corresponding to the first color, and wherein the second subset includes a second half of the pixel cells associated with color filters corresponding to the first color;

readout circuitry for reading out the accumulated first and second charge; and

processing circuitry for producing respective synthetic signals corresponding to each pixel cell of the array using results of the read out accumulated first and second charge, wherein the processing circuitry is configured to produce respective synthetic signals corresponding to at least a portion of the pixel cells using chrominance information from pixel cells of the first subset to obtain chrominance information for pixel cells of the second subset and using luminance information from pixel cells of the second subset to obtain luminance information for pixel cells of the first subset.

15. The image sensor of claim 14, wherein pixel cells of the array are arranged in rows, wherein the first and second subsets include subgroups of pixel cells, the subgroups each including two adjacent rows of pixel cells, the first subset including every other subgroup and the second subset including every other subgroup, such that the first and second subsets do not include same subgroups.

16. The image sensor of claim 15, wherein the processing circuitry is configured to apply the equation R(i,j)=S(i,j)+S(i,j+2) to produce synthetic signals corresponding to respective pixel cells of the first subset and applying the equation R(i,j)=S(i,j)+S(i,j−

2) to produce synthetic signals corresponding to respective pixel cells of the second subset, where i is a colunm number of a pixel cell, j is a row number of a pixel cell, S is an original signal for a pixel cell, and R is the synthesized signal corresponding to a pixel cell.

17. The image sensor of claim 15, wherein the processing circuitry is configured to apply the equation R(i,j)=A*S(i,j)+B*(S(i,j−

2)+S(i,j+2)) to produce respective synthetic signals corresponding to pixel cells of the first subset and applying the equation R(i,j)=C *S(i,j)+D*(S(i,j−

2)+S(i,j+2)) to produce synthetic signals corresponding to respective pixel cells of the second subset, where A, B, C and D are numerical parameters between 1 and 0, i is a column number of a pixel cell, j is a row number of a pixel cell, S is an original signal for a pixel cell, and R is the synthesized signal corresponding to a pixel cell.

18. The image sensor of claim 14, further comprising:

a first row select line for each row for selecting pixel cells of the first subset; and

a second row select line for each row for selecting pixel cells of the second subset, wherein the pixel cells are arranged in columns and rows, wherein the first and second subsets include subgroups of pixel cells, each subgroup including adjacent pixel cells associated with a color filter of at least one of each color, wherein the first subset includes every other subgroup, and wherein the second subset includes pixel cells not in the first subset.

19. The image sensor of claim 18, wherein each subgroup includes a first pixel cell associated with a color filter corresponding to a first color, a second pixel cell associated with a color filter corresponding to a second color, a third pixel cell associated with a color filter corresponding to a third color, and a fourth pixel cell associated with a color filter corresponding to the first color, the first and second pixel cells being located adjacent to one another and in a same row, the third and fourth pixel cells being adjacent to one another and in a same row, the third and first pixel cells being adjacent to one another in a same column, and the second and fourth pixel cells being adjacent to one another in a same column.

20. The image sensor of claim 19, wherein the processing circuitry is configured to apply the equation R(i,j)=S(i,j)+S(i,j+2) for a sequence j=0, 1, 4, 5, 8, 9 . . . n, n+1, and the equation R(i,j)=S(i,j)+S(i,j−

2) for a sequence j=2, 3, 6, 7, 10, 11 . . . n′

, n′

+1 to produce synthetic signals corresponding respective pixel cells of the array, where i is a column number of a pixel cell, j is a row number of a pixel cell, n and n′

are row numbers consistent with the respective sequences, S is an original signal for a pixel cell, and R is the synthesized signal corresponding to a pixel cell.

21. The image sensor of claim 19, wherein the processing circuitry is configured to apply the equation R(i,j)=A*S(i,j)+B*(S(i−

2,j)+S(i+2,j)+S(i,j−

2)+S(i,j+2)) to produce synthetic signals corresponding to respective pixel cells of the first subset and R(i,j)=C*S(i,j)+D*(S(i−

2,j)+S(i+2,j)+S(i,j−

2)+S(i,j+2)) to produce synthetic signals corresponding respective pixel cells of the second subset, where A, B, C and D are numerical parameters between 1 and 0, i is a column number of a pixel cell, j is a row number of a pixel cell, S is an original signal for a pixel cell, and R is the synthesized signal corresponding to a pixel cell.

22. The image sensor of claim 14, wherein the processing circuitry is configured to apply the equation R(i,j)=S(i,j)+S(i,j+1) for a sequence j=0, 2, 4, 6, . . . n, n+2 and the equation R(i,j)=S(i,j)+S(i,j−

1) for a sequence j=1, 3, 5, 7, . . . n′

, n′

+2 to produce synthetic signals corresponding respective pixel cells of the array, where i is a column number of a pixel cell, j is a row number of a pixel cell, n and n′

are row numbers consistent with the respective sequences, S is an original signal for a pixel cell, and R is the synthesized signal corresponding to a pixel cell.

23. The image sensor of claim 14, wherein the processing circuitry is configured to apply the equations R(i,j)=A*S(i,j)+B*(S(i,j−

1)+S(i,j+1)) for a sequence j=0,2,4,6, . . . n, n+2 and R(i,j)=C*S(i,j)+D*(S(i,j−

1)+S (i,j+1)) for a sequence j=1,3,5,7, . . . n′

, n′

+2 to produce synthetic signals corresponding respective pixel cells of the array, where A, B, C and D are numerical parameters between 1 and 0, i is a column number of a pixel cell, j is a row number of a pixel cell, n and n′

are row numbers consistent with the respective sequences, S is an original signal for a pixel cell, and R is the synthesized signal corresponding to a pixel cell.

24. The image sensor of claim 14, wherein the pixel cells are arranged in columns and rows, wherein the first subset includes every other pixel cell in a row and every other pixel cell in a column, and wherein the second subset includes the pixel cells not in the first subset, such that the first and second subsets form a checker board-like pattern.

25. The image sensor of claim 24, wherein the processing circuitry is configured to apply the equation R(i,j)=S(i,j)+S(i,j+1) for a sequence j=0, 2, 4, 6, . . . n, n+2 and the equation R(i,j)=S(i,j)+S(i,j−

1) for a sequence j=1, 3, 5, 7, . . . n′

, n′

+2 to produce synthetic signals corresponding respective pixel cells, where i is a column number of a pixel cell, j is a row number of a pixel cell, n and n′

are row numbers consistent with the respective sequences, S is an original signal for a pixel cell, and R is the synthesized signal corresponding to a pixel cell.

26. The image sensor of claim 24, wherein the processing circuitry is configured to apply the equations R(i,j)=A*S(i,j)+B*(S(i,j−

1)+S(i,j+1)) to produce synthetic signals corresponding respective pixel cells of the first subset and R(i,j)=C*S(i,j)+D*(S(i,j−

1)+S(i,j+1)) to produce synthetic signals corresponding respective pixel cells of the second subset, where A, B, C and D are numerical parameters between 1 and 0, i is a column number of a pixel cell, j is a row number of a pixel cell, S is an original signal for a pixel cell, and R is the synthesized signal corresponding to a pixel cell.

27. The image sensor of claim 14, wherein the first subset includes pixel cells within a central portion of the array and the second subset includes pixel cells in a perimeter portion of the array, and wherein the processing circuitry is configured to produce respective synthetic signals corresponding to at least a portion of the pixel cells using chrominance information from pixel cells of the second subset to obtain chrominance information for pixel cells of the first subset.

28. The image sensor of claim 14, wherein at least two pixel cells share common readout circuitry.

29. A processor system comprising:

a processor; and

an image sensor coupled to the processor, the image sensor comprising;

a color filter array comprising color filters of a plurality of colors;

an array of pixel cells, the pixel cells arranged in rows and subgroups, the subgroups each comprising a first pixel cell associated with a color filter corresponding to a first color, a second pixel cell associated with a color filter corresponding to a second color, a third pixel cell associated with a color filter corresponding to a third color, and a fourth pixel cell associated with a color filter corresponding to the first color, the first, second, third and fourth pixel cells being adjacent to one another;

the pixel cells further arranged into first and second subsets, the first subset comprising every other subgroup and the second subset comprising subgroups not in the first subset, the first and second subsets arranged in a checker board-like pattern;

timing and control circuitry in communication with the array, the timing and control circuitry configured to cause first charge to be accumulated in a first subset of pixel cells during a first integration period and second charge to be accumulated in a second subset of pixel cells during a second integration period, a length of the first integration period being longer than a length of the second integration period, the pixel cells accumulating only first or second charge per frame;

readout circuitry for reading out the accumulated first and second charge; and

processing circuitry for producing respective synthetic signals corresponding to each pixel cell of the array using results of the read out accumulated first and second charge, the processing circuitry configured to apply the equation R(i,j)=A*S(i,j)+B*(S(i−

2,j)+S(i+2,j)+S(i,j−

2)+S(i,j+2)) to produce synthetic signals corresponding to respective pixel cells of the first subset and R(i,j)=C*S(i,j)+D*(S(i−

2,j)+S(i+2,j)+S(i,j−

2)+S(i,j+2)) to produce synthetic signals corresponding respective pixel cells of the second subset, where A, B, C and D are numerical parameters between 1 and 0, i is a column number of a pixel cell, j is a row number of a pixel cell, S is an original signal for a pixel cell, and R is the synthesized signal corresponding to a pixel cell.

30. The system of claim 29, wherein R(i,j) is the sum of original signals for pixel cells associated with color filters corresponding to a same color.

31. The system of claim 29, wherein A and C equal 0.5, and B and D equal 0.25.

32. The system of claim 29, wherein A equals 1, B and C equal 0, and D equals 0.5.

33. The method of claim 29, wherein producing respective synthetic signals comprises varying A, B, C, and D depending upon an intensity of light incident on the array.