HIGH DYNAMIC RANGE SENSOR WITH REDUCED LINE MEMORY FOR COLOR INTERPOLATION

• US 20090109306A1
• Filed: 10/26/2007
• Published: 04/30/2009
• Est. Priority Date: 10/26/2007
• Status: Active Grant

• Current Assignee: Omnivision Technologies Incorporated (Will Semiconductor Co., Ltd. Shanghai)
• Original Assignee: Omnivision Technologies Incorporated (Will Semiconductor Co., Ltd. Shanghai)
• Inventors: Shan, Jizhang, Kuang, Jiangtao
• Granted Patent: US 7,777,804 B2
• Time in Patent Office: Days
• US Class Current: 348/273
• CPC Class Codes: H04N 23/70 Circuitry for compensating brightness variation in the scene
• CPC Class Codes: H04N 23/843 Demosaicing, e.g. interpolating colour pixel values
• CPC Class Codes: H04N 25/134 based on three different wavelength filter elements
• CPC Class Codes: H04N 25/46 by combining or binning pixels
• CPC Class Codes: H04N 25/533 by using differing integration times for different sensor regions
• CPC Class Codes: H04N 25/571 involving a non-linear response
• CPC Class Codes: H04N 25/581 acquired simultaneously
• CPC Class Codes: H04N 25/583 with different integration times
• CPC Class Codes: H04N 25/587 acquired sequentially, e.g. using the combination of odd and even image fields
• CPC Class Codes: H04N 25/70 SSIS architectures; Circuits associated therewith

Abstract

An image sensor has an array of pixels organized into a row and column format. Pixels are read out in a line-by-line sequence and buffered in a line image buffer. An extended dynamic range is supported by varying a column exposure time according to a periodic sequence. As a result, the pixel exposure times vary within each row. A high dynamic range is generated by combining pixel data of adjacent pixels within the same row that are of the same filter type but having different exposure times. Color interpolation is performed on the combined line data.

Claims

1. A method of achieving a high dynamic range in an image sensor having photo-sensitive pixels arranged in a row and column format with each pixel further assigned a color filter array filter (CFA) filter according to a CFA pattern, comprising:

for each individual column of pixels, selecting a common exposure time for all of the pixels within the individual column as a column exposure time;

selecting the column exposure times to vary along a horizontal dimension according to a periodic sequence selected such that within each individual row of pixels there is a corresponding periodic sequence of pixel exposure times for pixels of each individual color filter type;

reading out rows in a line-by-line sequence;

buffering a selected number of lines in a line memory;

for each line, combining data from sets of adjacent pixels of the same color filter type having at least two different exposure times to generate combined line data having a reduced horizontal resolution and an extended dynamic range; and

performing de-mosaicing on the combined line data.

2. The method of claim 1, wherein the line memory is sized to support 3-line de-mosaicing.

3. The method of claim 1, wherein the color filter array pattern repeats the same color filter type after every two pixels within an individual row and the periodic sequence is selected such that there is a periodic sequence of long exposure time pixels and short exposure time pixels for each color filter type.

4. The method of claim 1, wherein said combining data is performed using a weighted ratio of the contribution of pixels in each set, the weighting selected based on a combination of resolution and signal to noise ratio criteria.

5. The method of claim 1, wherein the line memory is sized to support 3-line demosaicing and a bilinear interpolation is performed on the combined line data.

6. The method of claim 1, further comprising supporting a full resolution mode in which the pixels have the same exposure time and line data is not combined.

7. A method of achieving a high dynamic range in an image sensor having photo-sensitive pixels arranged in a row and column format supporting line-by-line read out of rows with each pixel further being assigned a color filter array (CFA) filter according to a CFA pattern, the method comprising:

for each individual column of pixels, selecting a common exposure time for all of the pixels within the individual column as a column exposure time;

selecting the column exposure times to vary along a horizontal dimension according to a sequence, the sequence selected such that within each individual row the pixels of a particular filter type have a periodic sequence of pixel exposure times;

reading out rows in a line-by-line sequence;

buffering lines in a line memory sized to support 3-line de-mosaicing;

for each line, combining data from sets of adjacent pixels having the same filter type but at least two different exposure times to generate combined line data having no more than half the effective line width and an extended dynamic range; and

performing color interpolation on the combined line data to determine color values at each pixel location.

8. The method of claim 7, wherein the CFA pattern repeats the same color filter type every two pixels within an individual row and the periodic sequence is selected such that there is a periodic sequence of long exposure time pixels and short exposure time pixels for each color filter type.

9. The method of claim 7, wherein said combining data is performed using a weighted ratio of the contribution of pixels in each set, the weighting selected based on a combination of resolution and signal to noise ratio criteria.

10. The method of claim 7, wherein the color interpolation comprises bilinear interpolation is performed on the combined line data.

11. The method of claim 7 further comprising supporting a full resolution mode in which the pixels have the same exposure time and line data is not combined.

12. A method of supporting an extended dynamic range using a 3-line image buffer in an image sensor having photo-sensitive pixels arranged in a row and column format supporting line-by-line read out of rows with each pixel further being assigned a color filter array (CFA) filter according to a CFA pattern, the method comprising:

in a full resolution mode of operation, setting a common default exposure time for each pixel and performing color interpolation using the 3-line image buffer;

in a high dynamic range mode of operation;

selecting a common exposure time for all of the pixels within an individual column as a column exposure time and varying the column exposure times to vary along a horizontal dimension according to a sequence, the sequence selected such that within each individual row the pixels of a particular filter type have a periodic sequence of pixel exposure times;

reading out rows in a line-by-line sequence;

buffering lines in the 3-line image buffer;

for each line, combining data from sets of adjacent pixels having the same filter type but two different exposure times to generate combined line data having half the effective line width and an extended dynamic range; and

performing color interpolation on the combined line data to determine color values at each pixel location.

13. The method of claim 12, wherein the color filter array pattern repeats the same color filter type every two pixels within an individual row and in the high dynamic range mode and the periodic sequence is selected such that there is a periodic sequence of long exposure time pixels and short exposure time pixels for each color filter type.

14. The method of claim 12, wherein in the high dynamic range mode said combining data is performed using a weighted ratio of the contribution of pixels in each set, the weighting selected based on a combination of resolution and signal to noise ratio criteria.

15. An image sensing system, comprising:

an array of photo-sensitive pixels organized into rows and columns with each pixel assigned to a color filter array filter; and

a control block and an address decoder configured to support a high dynamic range mode of operation by selecting a common exposure time for all of the pixels within an individual column as a column exposure time and varying the column exposure times along a horizontal dimension according to a periodic sequence, the sequence selected such within each individual row of pixels each set of pixels of a particular filter type has a corresponding periodic sequence of pixel exposure times with individual rows being read out in a line-by-line sequence.

16. The image sensing system of claim 15, further comprising a 3-line image buffer to receive line data from the pixel array and an image processor to process buffered line data.

17. The image sensing system of claim 16, wherein for each line of buffered line data the image processor combines raw pixel data from sets of adjacent pixels having the same filter type but at least two different exposure times to generate combined line data having half the effective line width and an extended dynamic range, the image processor performing color interpolation on the combined line data.

18. The image sensing system of claim 15, wherein individual pixels in the array are each assigned to particular colors by a color array filter pattern having a specified color sequence for each row that repeats after a selected number of columns.

19. The image sensing system of claim 17, wherein the color filter array pattern repeats the same color filter type every two pixels within an individual row and the periodic sequence is selected such that there is a periodic sequence of long exposure time pixels and short exposure time pixels for each color filter type.

20. The image sensing system of claim 17, wherein combined line data is generated using a weighted ratio of the contribution of pixels in each set, the weighting selected based on a combination of resolution and signal to noise ratio criteria.

21. The image sensing system of claim 15, wherein the image sensing system supports a full resolution mode in which the pixels have the same exposure time and line data is not combined.

22. An image sensing system, comprising:

a pixel array of photo-sensitive pixels organized into rows and columns with each pixel assigned to a color filter array filter;

a control block and address decoders to control pixel exposure time for each pixel with individual rows being read out in a line-by-line sequence;

a 3-line image buffer to buffer line data received from the pixel array for processing by an image processor;

the image sensing system having a first mode of operation in which each pixel has the same default exposure time and color interpolation is performed on the line data in the 3-line image buffer;

the image sensing system further having a high dynamic range mode of operation in which the control block selects a common exposure time for all of the pixels within an individual column as a column exposure time and varies the column exposure times along a horizontal dimension according to a sequence, the sequence selected such within each individual row of pixels each set of pixels of a particular filter type has a periodic sequence of pixel exposure times, with for each line of buffered pixel data in the 3-line memory the image processor combining raw pixel data from sets of adjacent pixels having the same filter type but at least two different exposure times to generate combined line data having half the effective line width and an extended dynamic range, the image processor performing color interpolation on the combined line data to determine color values at each pixel location

23. The image sensing system of claim 22, wherein the color filter array pattern repeats the same color filter type every two pixels within an individual row and the periodic sequence is selected such that there is a periodic sequence of long exposure time pixels and short exposure time pixels for each color filter type.

24. The image sensing system of claim 22, wherein combined line data is generated using a weighted ratio of the contribution of pixels in each set, the weighting selected based on a combination of resolution and signal to noise ratio criteria.

25. The image sensing system of claim 22, wherein the image sensing system supports a full resolution mode in which the pixels have the same exposure time and line data is not combined.