Imaging system with multiple sensors for producing high-dynamic-range images

• US 8,803,990 B2
• Filed: 02/28/2011
• Issued: 08/12/2014
• Est. Priority Date: 01/25/2011
• Status: Active Grant

• Current Assignee: Aptina Imaging Corporation (ON Semiconductor Corporation)
• Original Assignee: Aptina Imaging Corporation (ON Semiconductor Corporation)
• Inventors: Smith, Scott
• Primary Examiner(s): Misleh, Justin P
• Application Number:
  US13/036,573
• Publication Number:
  US 20120188392A1
• Time in Patent Office: 1,261 Days
• Field of Search: 348/47, 348/48, 348/218.1, 348/362, 348/364, 348/366, 348/367
• US Class Current: 348/218.1
• CPC Class Codes: H04N 23/741 by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 25/41 Extracting pixel data from a plurality of image sensors simultaneously picking up an image, e.g. for increasing the field of view by combining the outputs of a plurality of sensors
• CPC Class Codes: H04N 25/531 by controlling rolling shutters in CMOS SSIS

Abstract

An electronic device may have an array of image sensors that capture image data using different exposure times. Processing circuitry may be used to combine image data with a first exposure time and image data with a second exposure time to create a high-dynamic-range image. The image sensors may use electronic rolling shutter and global shutter image capture schemes. Using the electronic rolling shutter scheme, the reset signals for each sensor may be staggered and the read signals for each sensor may be aligned to allow synchronized readout from the image sensors. When using the global shutter scheme, image capture operations associated with a shorter exposure time may be centered in time within image capture operation associated with a longer exposure time to minimize motion artifacts. Multiple image sensors may also be used to capture short-exposure-time data that is spaced evenly in time within the longer exposure time data.

Claims

1. A method for capturing image data for a high-dynamic-range image using multiple image sensors in an electronic device each of which receives image light through a respective lens in a lens array in the electronic device, the method comprising:

with a first of the image sensors, capturing image data using a first exposure time;

with a second of the image sensors, capturing image data using a second exposure time that is longer than the first exposure time; and

in synchronization, reading out data from rows of image pixels in the first and second image sensors using control circuitry in the electronic device, wherein the first and second image sensors use an electronic rolling shutter scheme to capture images; and

asserting a first reset signal for the first image sensor at a first time, asserting a second reset signal for the second image sensor at a second time subsequent to the first time, wherein reading out the data from the first and second image sensors comprises asserting a read signal for a given row in the first image sensor at a third time and simultaneously asserting a read signal for the given row in the second image sensor at the third time, wherein the third time is after the first and second times.

2. The method defined in claim 1 further comprising:

with processing circuitry in the electronic device, combining the image data captured using the first exposure time with the image data captured using the second exposure time to produce the high-dynamic-range image.

3. The method defined in claim 2 wherein the multiple image sensors include a third image sensor, the method further comprising:

with the third image sensor, capturing image data using a third exposure time.

4. The method defined in claim 3 wherein the third exposure time is longer than the first exposure time and shorter than the second exposure time.

5. The method defined in claim 4 wherein combining the image data comprises combining the image data captured with the third exposure time with the image data captured using the first and second exposure times to produce the high-dynamic-range image.

6. A method for capturing image data for a high-dynamic-range image using multiple image sensors in an electronic device each of which receives image light through a respective lens in a lens array in the electronic device, the method comprising:

with a first of the image sensors, capturing image data with a first exposure time using a global shutter scheme;

with a second of the image sensors, capturing image data with a second exposure time that is longer than the first exposure time using a global shutter scheme, wherein the first and second exposure times overlap, wherein capturing the image data comprises capturing at least some of the image data with the second exposure time while simultaneously capturing at least some of the data with the first exposure time, and wherein capturing the image data comprises capturing the image data so that the image data with the first exposure time is centered in time with respect to the image data with the second exposure time.

7. The method defined in claim 6 further comprising:

with an image processor in the electronic device, combining the image data with the first exposure time and the image data with the second exposure time to produce the high-dynamic-range image.

8. The method defined in claim 7 wherein capturing the image data comprises asserting a transfer signal at a different time in each of the image sensors.

9. The method defined in claim 8 further comprising:

simultaneously reading out rows of image data from the first and second image sensors following assertion of the transfer signals in each of the image sensors.

10. The method defined in claim 6, wherein the image data with the first exposure time has a first center time, wherein the image data with the second exposure time has a second center time, and wherein capturing the image data further comprises:

capturing the image data so that the second center time varies from the first center time by less than twenty percent of the second exposure time.

11. The method defined in claim 10, wherein capturing the image data further comprises:

capturing the image data so that the second center time varies from the first center time by less than five percent of the second exposure time.

12. The method defined in claim 10, wherein capturing the image data further comprises:

asserting a first reset signal at a first time in the first image sensor;

asserting a second reset signal at a second time prior to the first time in the second image sensor;

asserting a first charge transfer signal at a third time that is after the first and second times in the first image sensor;

asserting a second charge transfer signal at a fourth time that is after the third time in the second image sensor;

asserting a first readout signal at a fifth time that is after the fourth time in the first image sensor; and

asserting a second readout signal at the fifth time in the second image sensor, wherein the first center time is centered in time between the first and third times, and wherein the second center time is centered in time between the second and fourth times so that image data with the first exposure time has similar motion characteristics as the image data with the second exposure time.

13. A method for capturing image data for a high-dynamic-range image using multiple image sensors in an electronic device each of which receives image light through a respective lens in a lens array in the electronic device, the method comprising:

with a given one of the image sensors, capturing image data with a first exposure time; and

with a plurality of the image sensors other than the given one of the image sensors, capturing image data with a second exposure time that is shorter than the first exposure time, wherein capturing the image data with the second exposure time comprises capturing image data from the plurality of image sensors other than the given image sensor using a sequence of identical exposure times that are evenly spaced among the first exposure time.

14. The method defined in claim 13 further comprising:

with an image processor in the electronic device, combining the image data with the first exposure time and the image data with the second exposure time to produce a high-dynamic-range image.

15. The method defined in claim 14 wherein the image data with the second exposure time comprises image data from each of the plurality of image sensors other than the given one of the image sensors and wherein combining the image data comprises summing the image data from each of the plurality of image sensors.

16. The method defined in claim 15 wherein the plurality of image sensors other than the given one of the image sensors include at least eight sensors.

17. The method defined in claim 13 further comprising capturing multiple frames of video data by repeatedly using the multiple image sensors to capture data with the first and second exposure times.

18. The method defined in claim 13, wherein each exposure time of the sequence of exposure times occurs during different respective time periods, and wherein each exposure time of the sequence of exposure times has an identical duration.

19. The method defined in claim 18, wherein capturing the image data from the plurality of image sensors other than the given image sensor using the sequence of identical exposure times comprises capturing the sequence of identical exposure times so that a sum of each of the identical exposure times in the sequence is equal to the first exposure time.