IMAGING DEVICE WITH A PLURALITY OF PIXEL ARRAYS

• US 20140078333A1
• Filed: 09/19/2012
• Published: 03/20/2014
• Est. Priority Date: 09/19/2012
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Miao, Xiaoyu
• Granted Patent: US 9,143,673 B2
• Time in Patent Office: Days
• US Class Current: 348/222.1
• CPC Class Codes: G02B 13/0015 characterised by the lens design
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: H04N 23/50 Constructional details
• CPC Class Codes: H04N 23/51 Housings
• CPC Class Codes: H04N 23/54 Mounting of pick-up tubes, electronic image sensors, deviation or focusing coils
• CPC Class Codes: H04N 23/60 Control of cameras or camera modules
• CPC Class Codes: H04N 23/69 Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
• CPC Class Codes: H04N 23/73 by influencing the exposure time
• CPC Class Codes: H04N 23/741 by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors
• CPC Class Codes: H04N 23/951 by using two or more images to influence resolution, frame rate or aspect ratio
• CPC Class Codes: H04N 25/41 Extracting pixel data from a plurality of image sensors simultaneously picking up an image, e.g. for increasing the field of view by combining the outputs of a plurality of sensors
• CPC Class Codes: H04N 5/265 Mixing

Abstract

An imaging device includes a first pixel array arrange to capture a first image and a second pixel array arranged to capture a second image. The first pixel array and the second pixel array face substantially a same direction. The imaging device also includes shutter control circuitry which is coupled to the first pixel array to initiate a first exposure period of the first pixel array to capture the first image. The shutter control circuitry is also coupled to the second pixel array to initiate a second exposure period of the second pixel array to capture the second image. The imaging device also includes processing logic coupled to receive first pixel data of the first image and coupled to receive second pixel data of the second image. The processing logic is configured to generate at least one image using the first pixel data and the second pixel data.

Claims

1. An imaging device comprising:

a first pixel array arranged to capture a first image;

a second pixel array arranged to capture a second image, wherein the first pixel array and the second pixel array are facing substantially a same direction;

shutter control circuitry coupled to the first pixel array to initiate a first exposure period of the first pixel array to capture the first image and coupled to the second pixel array to initiate a second exposure period of the second pixel array to capture the second image; and

processing logic coupled to receive first pixel data of the first image and coupled to receive second pixel data of the second image, wherein the processing logic is configured to generate at least one image using the first pixel data and the second pixel data.

2. The imaging device of claim 1, wherein the first exposure period is for a first duration that is shorter than a second duration of the second exposure period.

3. The imaging device of claim 2, wherein the processing logic is coupled to output a composite image generated by combining the first pixel data and the second pixel data.

4. The imaging device of claim 3, wherein the processing logic is configured to execute a high dynamic range (“

HDR”

) algorithm to combine the first pixel data and the second pixel data into the composite image.

5. The imaging device of claim 1, wherein the shutter control circuitry is configured to initiate the second exposure period after the first exposure period starts.

6. The imaging device of claim 5, wherein the processing logic is coupled to output a sequence of images generated by interleaving at least a first portion of the first pixel data with at least a second portion of the second pixel.

7. The imaging device of claim 1, wherein the shutter control circuitry is configured to initiate the first exposure period and the second exposure period at substantially a same start time, and wherein the first exposure period and the second exposure period is for substantially a same duration, the processing logic coupled to output a composite image generated by performing a super-resolution algorithm on the first pixel data and the second pixel data.

8. The imaging device of claim 1 further comprising:

a first color filter positioned to filter image light propagating toward the first pixel array, wherein the first color filter is not positioned to filter the image light from the second pixel array, and wherein the processing logic is coupled to output a light-enhanced image generated by combining the first pixel data and the second pixel data.

9. The imaging device of claim 1 further comprising:

a third pixel array arranged to capture a third image;

a fourth pixel array arranged to capture a fourth image, wherein the third pixel array and the fourth pixel array are facing substantially the same direction as the first pixel array,

and wherein the shutter control circuitry is coupled to initiate a third exposure period of the third pixel array and coupled to initiate a fourth exposure period of the fourth pixel array,

and wherein the processing logic is coupled to receive third pixel data of the third image and coupled to receive fourth pixel data of the fourth image, wherein the processing logic is configured to generate the at least one image using the first, second, third, and fourth pixel data.

10. The imaging device of claim 9, wherein the processing logic is configured to combine the first pixel data and the second pixel data into a first composite image and configured to combine the third pixel data and the fourth pixel data into a second composite image, and wherein the processing logic is configured to interleave the first composite image and the second composite image into a video.

11. The imaging device of claim 9 further comprising:

a lens system configured to focus image light onto the first pixel array and the second pixel array from a first focus distance and configured to focus the image light onto the third pixel array and the fourth pixel array from a second focus distance different from the first focus distance,

wherein the first exposure period is shorter than the second exposure period and the third exposure period is shorter than the fourth exposure period,

and wherein the processing logic is configured to perform a high dynamic range (“

HDR”

) algorithm on the first and second pixel data, configured to perform the HDR algorithm on the third and fourth pixel data, configured to perform an after capture focusing (“

ACF”

) algorithm on the first and third pixel data, and configured to perform the ACF algorithm on the second and fourth pixel data, the processing logic further configured to output an HDR/ACF image.

12. The imaging device of claim 9 further comprising:

a color filter positioned to filter image light from the first and the third pixel arrays, wherein the color filter array is not positioned to filter the image light from the second and the fourth pixel arrays.

13. The imaging device of claim 9, wherein the shutter control circuitry is configured to stagger initiating the first, the second, the third, and the fourth exposure periods, and wherein the processing logic is configured to interleave the first, the second, the third, and the fourth pixel data into a high-frame-rate video.

14. The imaging device of claim 9 further comprising a head mounted display (“

HMD”

), wherein the first pixel array, the second pixel array, the shutter control circuitry and the processing logic are disposed on the HMD, and a flexible Printed Circuit Board (“

PCB”

) embedded in a frame of the HMD carries the first pixel data and the second pixel data to the processing logic.

15. An imaging device comprising:

a first pixel array arranged to capture a first image;

a second pixel array arranged to capture a second image, wherein a first field of view of the first pixel array partially overlaps a second field of view of the second pixel array;

shutter control circuitry coupled to simultaneously initiate a first exposure period of the first pixel array to capture the first image and a second exposure period of the second pixel array to capture the second image; and

processing logic coupled to receive first pixel data of the first image and coupled to receive second pixel data of the second image, wherein the processing logic is coupled to output a composite image generated by stitching together the first pixel data and the second pixel data.

16. The imaging device of claim 15, wherein the first exposure period and the second exposure period are the same duration.

17. The imaging device of claim 15 further comprising:

a third pixel array arranged to capture a third image, wherein a third field of view of the third pixel array partially overlaps the second field of view of the second pixel array;

wherein the shutter control circuitry is coupled to initiate a third exposure period of the third pixel array to capture the third image, the first, second, and third exposure periods initiated simultaneously,

and wherein the processing logic is coupled to receive third pixel data of the third image and coupled to output the composite image generated by stitching together the first, second, and third pixel data, the composite image substantially including a combined field of view of the first, second, and third fields of view.

18. The imaging device of claim 15 further comprising a head mounted display (“

HMD”

), wherein the first pixel array, the second pixel array, the shutter control circuitry and the processing logic are disposed on the HMD, and a flexible Printed Circuit Board (“

PCB”

) embedded in a frame of the HMD carries the first pixel data and the second pixel data to the processing logic.

19. A method comprising:

initiating a first exposure period of a first pixel array to capture a first image;

initiating a second exposure period of a second pixel array to capture a second image, wherein the first pixel array and the second pixel array are disposed on a same semiconductor die and are facing substantially a same direction; and

generating at least one image based on first pixel data from the first image and second pixel data from the second image.

20. The method of claim 19 further comprising:

initiating a third exposure period of a third pixel array to capture a third image;

initiating a fourth exposure period of a fourth pixel array to capture a fourth image, wherein the third pixel array and the fourth pixel array are disposed on the same semiconductor die and are facing substantially the same direction as the first pixel array and the second pixel array, and wherein generating the at least one image is also based on third pixel data from the third image and fourth pixel data from the fourth image.

21. The method of claim 20, wherein the first exposure period is for a first duration that is shorter than a second duration of the second exposure period and the third exposure period is for a third duration that is shorter than a fourth duration of the fourth exposure period, and wherein generating the at least on image includes generating a first composite image with the first and second pixel data and generating a second composite image with the third and fourth pixel data and interleaving the first composite image with the second composite image into a sequence of images.

22. The method of claim 20, wherein the first, second, third, and fourth exposure periods are staggered, and wherein generating the at least one image includes interleaving the first, second, third, and fourth pixel data into a high-frame-rate video.

23. The method of claim 20, wherein the first pixel array and the second pixel array receive image light focused at a first optical power and the third pixel array and the fourth pixel array receive the image light focused at a second optical power,

and wherein the first exposure period is for a first duration that is shorter than a second duration of the second exposure period and the third exposure period is for a third duration that is shorter than a fourth duration of the fourth exposure period,

the generating the at least one image including performing a high dynamic range (“

HDR”

) algorithm and an after capture focusing (“

ACF”

) algorithm.

24. The method of claim 19, wherein the first exposure period is for a first duration that is shorter than a second duration of the second exposure period and wherein generating the at least one image includes generating a high dynamic range (“

HDR”

) image.

25. The method of claim 19, wherein the first exposure period and the second exposure period are staggered, and wherein generating the at least one image includes interleaving at least a first portion of the first pixel data and a second portion of the second pixel data to generate high-frame-rate video.

26. The method of claim 19, wherein generating the at least one image includes generating a composite image by performing a super-resolution algorithm on the first pixel data and the second pixel data.

27. The method of claim 19, wherein generating the at least one image includes generating a light-enhanced image by combining color data from the first pixel data and intensity data from the second pixel data, wherein the first pixel array receives image light through a color filter and the second pixel array does not receive the image light through the color filter.