High frame rate interleaved imagers

• US 8,797,439 B1
• Filed: 06/22/2011
• Issued: 08/05/2014
• Est. Priority Date: 06/22/2011
• Status: Expired due to Fees

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Rawles, LLC (Amazon.com, Inc.)
• Inventors: Coley, Christopher, Weatherford, William Thomas, Yuan, Robert A., Marason, Eric G.
• Primary Examiner(s): DIEP, TRUNG T
• Application Number:
  US13/166,588
• Time in Patent Office: 1,140 Days
• Field of Search: 348/211.11, 348/221.1, 348320-324, 348/367, 348/376
• US Class Current: 348/305
• CPC Class Codes: H04N 23/45 for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 23/73 by influencing the exposure time
• CPC Class Codes: H04N 23/74 by influencing the scene brightness using illuminating means
• CPC Class Codes: H04N 23/75 by influencing optical camera components
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 23/951 by using two or more images to influence resolution, frame rate or aspect ratio
• CPC Class Codes: H04N 25/40 Extracting pixel data from image sensors by controlling scanning circuits, e.g. by modifying the number of pixels sampled or to be sampled

Abstract

Two or more imagers having associated frame rates are each configured with a global shutter having an exposure interval less than a frame interval. The exposure interval for a first imager is offset from the exposure interval of a second imager such that the first imager is exposed while the second imager is not exposed. When interleaved, an effective frame rate exceeds the individual frame rates provided by the individual imagers. This interleaved imaging system may be used to recover, at high frame rates, images that are projected from spatial light modulators.

Claims

1. An imaging system comprising:

a first global shutter coupled to a first imager having a first frame rate;

a second global shutter coupled to a second imager having a second frame rate; and

an imaging controller configured to;

maintain a first exposure interval of the first global shutter with a duration less than the first frame rate;

maintain a second exposure interval of the second global shutter with a duration less than the second frame rate;

interleave open and closed global shutter exposure intervals of the first and second global shutters; and

read out images from each of the imagers during non-exposed intervals to generate an effective frame rate greater than the first frame rate or the second frame rate.

2. The system of claim 1, wherein the first exposure interval and the second exposure interval overlap.

3. The system of claim 1, wherein the global shutters are configured to:

(1) expose substantially all of an imaging portion of the respective imager at about the same time when open, and (2) substantially prevent exposure of the imaging portion of the respective image when closed.

4. The system of claim 1, wherein the global shutters comprise liquid crystal shutters.

5. The system of claim 1, wherein the first frame rate and the second frame rate are about the same value but offset from one another by a pre-determined interleaving offset time interval.

6. The system of claim 1, wherein the first frame rate and the second frame rate are about 60 frames per second and the first and second exposure intervals are about 1/120 of a second.

7. The system of claim 1, wherein the imaging controller is further configured to recover an image comprising a projected image;

and further comprising a spatial light modulator configured to project at least a portion of the projected image.

8. The system of claim 7, wherein the spatial light modulator comprises a digital micromirror device.

9. The system of claim 7, wherein the projected image comprises structured light.

10. The system of claim 7, wherein the effective frame rate is between about 60 frames per second and about 240 frames per second.

11. An imaging system comprising:

a plurality of imaging elements configured to share at least a portion of an optical path, each of the plurality of imaging elements being coupled to a respective global shutter; and

an imaging controller configured to interleave open and closed states of the global shutters across the plurality of imaging elements to generate an effective frame rate greater than an individual frame rate for each of the plurality of imaging elements.

12. The system of claim 11, further comprising an additional one or more imaging elements configured to share at least a portion of the optical path, the additional one or more imaging elements being free from a global shutter.

13. The system of claim 11, wherein a shutter exposure interval of the global shutters is less than or equal to about 1/n of the frame rate of the plurality of imaging elements where n equals a number of imaging elements.

14. The system of claim 11, wherein the frame rate is about 60 frames per second, a shutter exposure interval for the open global shutter state is about 1/120 of a second, and the effective frame rate is about 120 frames per second.

15. The system of claim 11, wherein one or more of the plurality of imagers are configured to detect infrared light.

16. The system of claim 11, wherein at least one of the global shutters comprises a mechanical assembly.

17. The system of claim 11, further comprising a spatial light modulator configured to generate a projected image detectable at least in part by the plurality of imaging elements.

18. The system of claim 17, wherein the projected image is configured to share at least a portion of the optical path.

19. One or more computer-readable media storing computer-executable instructions that, when executed, cause one or more processors to perform acts comprising:

setting a first imager to a first frame rate;

setting a first global shutter coupled to the first imager to a first exposure interval;

setting a second imager to a second frame rate interleaved with the first frame rate; and

setting a second global shutter coupled to the second imager to a second exposure interval interleaved with the first exposure interval to generate an effective frame rate greater than the first frame rate and greater than the second frame rate.

20. The one or more computer-readable storage media of claim 19, wherein the first exposure interval and the second exposure interval to overlap.

21. The one or more computer-readable storage media of claim 19, further comprising:

acquiring interleaved images from the first and second imagers which share at least a portion of a common optical axis;

determining a reference point appearing across the acquired interleaved images; and

aligning the image of the reference point among the interleaved images.

22. The one or more computer-readable storage media of claim 21, wherein the aligning comprises shifting electronically the interleaved images.

23. The one or more computer-readable storage media of claim 21, wherein the aligning comprises mechanically shifting the first imager, second imager, or both the first and second imagers.

24. A method comprising:

opening a first global shutter coupled to a first imager;

acquiring an image passed by the first global shutter at the first imager;

closing the first global shutter;

opening a second global shutter coupled to a second imager sharing at least a portion of an optical path with the first imager;

reading out image data from the first imager;

acquiring an image passed by the second global shutter at the second imager;

closing the second global shutter;

opening the first global shutter coupled to the first imager;

reading out image data from the second imager; and

interleaving the acquired image passed by the first global shutter at the first imager with the acquired image passed by the second global shutter at the second imager to generate an effective frame rate greater than a frame rate for the first or second imager.

25. The method of claim 24 wherein the opening of the first global shutter and the opening of the second global shutter overlap.

26. The method of claim 24 wherein the interval between opening and closing the global shutter is less than or equal to about 1/120 of a second.

27. The method of claim 24 wherein the image acquired by the first imager and the image acquired by the second imager are acquired at a frame rate of between 30 frames per second and 240 frames per second.