Methods and apparatus for using multiple optical chains in parallel with multiple different exposure times

• US 9,270,876 B2
• Filed: 01/05/2014
• Issued: 02/23/2016
• Est. Priority Date: 01/05/2013
• Status: Active Grant

• Current Assignee: Blue River Technology Inc. (Deere & Company)
• Original Assignee: The LightCo Inc.
• Inventors: Laroia, Rajiv
• Primary Examiner(s): Berardesca, Paul
• Application Number:
  US14/147,579
• Publication Number:
  US 20140192225A1
• Time in Patent Office: 779 Days
• US Class Current: 1/1
• CPC Class Codes: G02B 13/0065 having a beam-folding prism or mirror
• CPC Class Codes: G02B 13/16 for use in conjunction with image converters or intensifiers , or for use with projectors, e.g. objectives for projection TV
• CPC Class Codes: G02B 13/22 Telecentric objectives or lens systems
• CPC Class Codes: G02B 17/008 Systems specially adapted to form image relays or chained systems
• CPC Class Codes: G02B 17/0896 with variable magnification or multiple imaging planes, including multispectral systems systems with only refractive elements G02B15/14
• CPC Class Codes: G02B 3/0056 arranged along two different directions in a plane, e.g. honeycomb arrangement of lenses G02B3/0043 takes precedence; miniaturised objectives for electronic devices employing wafer level optics G02B13/0085
• CPC Class Codes: G02B 7/18 for prisms; for mirrors
• CPC Class Codes: G03B 17/17 with reflectors arranged in beam forming the photographic image, e.g. for reducing dimensions of camera
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: H04N 23/45 for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 23/54 Mounting of pick-up tubes, electronic image sensors, deviation or focusing coils
• CPC Class Codes: H04N 23/55 Optical parts specially adapted for electronic image sensors; Mounting thereof
• CPC Class Codes: H04N 23/57 Mechanical or electrical details of cameras or camera modules specially adapted for being embedded in other devices
• CPC Class Codes: H04N 23/73 by influencing the exposure time
• CPC Class Codes: H04N 23/741 by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors
• CPC Class Codes: H04N 23/75 by influencing optical camera components
• CPC Class Codes: H04N 5/265 Mixing

Abstract

Methods and apparatus for capturing or generating images using multiple optical chains operating in parallel are described. Pixel values captured by individual optical chains corresponding to the same scene area are combined to provide an image with at least some of the benefits which would have been provided by capturing an image of the scene using a larger lens than that of the individual lenses of the optical chain modules. By using multiple optical chains in parallel at least some benefits normally obtained from using a large lens can be obtained without the need for a large lens. Furthermore in at least some embodiments, a wide dynamic range can be supported through the use of multiple sensors with the overall supported dynamic range being potentially larger than that of the individual sensors. Some lens and/or optical chain configurations are designed for use in small handheld devices, e.g., cell phones.

Claims

1. A method of producing at least one image of a first scene area, the method comprising:

operating a plurality of optical chains in parallel to capture images of the first scene area, the step of operating a plurality of optical chains including;

operating a first optical chain including a first lens mounted on a lens platter, a first light redirection device for redirecting light through another optical element of the first optical chain, said another optical element being another lens of the first optical chain or a filter of the first optical chain, said another optical element of the first optical chain being mounted on a first cylinder through which light passes prior to reaching a first sensor of the first optical chain to capture a first image of the first scene area using a first exposure time; and

operating a second optical chain including a second lens mounted on the lens platter, a second light redirection device for redirecting light through another optical element of the second optical chain, said another optical element being another lens of the second optical chain or a filter of the second optical chain, said another optical element of the second optical chain being mounted on a second cylinder through which light passes prior to reaching a second sensor of the second optical chain to capture a second image of said first scene area using a second exposure time, at least said first and second exposure times being of different durations but overlapping in time; and

processing said first and second images to generate a first combined image of said first scene area.

2. The method of claim 1,

wherein processing said first and second images to generate a first combined image of said first scene area includes weighting pixel values of said first and second images corresponding to the same portion of said first scene area as a function of said first and second exposure times, respectively, and summing the weighted pixel values.

3. The method of claim 1, further comprising:

operating a third optical chain including a third lens mounted on the lens platter, a third light redirection device for redirecting light through another optical element of the third optical chain prior to reaching a third sensor of the third optical chain, to capture a third image of said first scene area using a third exposure time; and

processing said third image to generate said first combined image of said first scene area from the third image in addition to said first and second images.

4. The method of claim 3, wherein said third exposure time is different than said first and second exposure times.

5. The method of claim 3, further comprising:

operating one of said first, second or third optical chains to capture a fourth image of the first scene area prior to capture of said first, second and third images; and

displaying said fourth image on a display of a camera device including said first, second, third optical chains.

6. The method of claim 3, further comprising:

operating one of said first, second and third optical chains to capture a fourth image of the first scene area after capturing one of said first, second and third images; and

displaying said fourth image on a display of a device including said first, second, and third optical chains; and

wherein the combined image is generated in the device after the display of the fourth image on the device.

7. The method of claim 6, wherein said operating one of said first, second and third optical chains to capture said fourth image of the first scene area uses a fourth exposure time different from said first, second and third exposure times.

8. The method of claim 5, wherein said operating one of said first, second and third optical chains to capture a fourth image of the first scene area uses a fourth exposure time different from said first, second and third exposure times.

9. The method of claim 5, wherein at least some optical chains in said plurality of optical chains include single color filters at least two optical chains being provided for each color for which a single color filter is used.

10. The method of claim 9, wherein at least some optical chains do not include any color filters.

11. The method of claim 5, wherein different optical chains having single color filters corresponding to the same color have different exposure times.

12. An apparatus, comprising:

a plurality of optical chains that work in parallel to capture images of a first scene area, said plurality of optical chains including a first optical chain including a first lens mounted on a lens platter, a first light redirection device for redirecting light through another optical element of the first optical chain, said another optical element being another lens of the first optical chain or a filter of the first optical chain, said another optical element of the first optical chain being mounted on a first cylinder through which light passes prior to reaching a first sensor of the first optical chain, said first optical chain being configured to capture a first image of the first scene area using a first exposure time;

said plurality of optical chains further including a second optical chain including a second lens mounted on the lens platter, a second light redirection device for redirecting light through another optical element of the second optical chain, said another optical element being another lens of the second optical chain or a filter of the second optical chain, said another optical element of the second optical chain being mounted on a second cylinder through which light passes prior to reaching a second sensor of the second optical chain, said second optical chain being configured to capture a second image of said first scene area using a second exposure time, at least said first and second exposure times being of different durations but overlapping in time; and

an image processing module configured to process said first and second images to generate a first combined image of said first scene area.

13. The apparatus of claim 12,

wherein said image processing module is further configured to weight pixel values of said first and second images corresponding to the same portion of said first scene area as a function of said first and second exposure times, respectively and sum the weighted pixel values, as part of being configured to process said first and second images to generate a first combined image of said first scene area.

14. The apparatus of claim 12,

wherein said plurality of optical chains further comprises a third optical chain including a third lens mounted on the lens platter, a third light redirection device for redirecting light through another optical element of the third optical chain prior to reaching a third sensor of the third optical chain, said third optical chain being configured to capture a third image of said first scene area using a third exposure time;

and

wherein said image processing module is further configured to process said third image to generate said first combined image of said first scene area from the third image in addition to said first and second images.

15. The apparatus of claim 14, wherein said third exposure time is different than said first and second exposure times.

16. The apparatus of claim 14, wherein at least one of said first, second or third optical chains is further configured to capture a fourth image of the first scene area prior to capture of said first, second and third images, the apparatus further comprising:

a display device configured to display said fourth image.

17. The apparatus of claim 14,

wherein at least one of said first, second or third optical chains is further configured to capture a fourth image of the first scene area after one of said first, second or third images has been captured;

wherein the apparatus further comprises a display device configured to display said fourth image; and

wherein the combined image is generated after the display of the fourth image on the display device.

18. The apparatus of claim 17, wherein said at least one of said first, second and third optical chains uses a fourth exposure time different from said first, second and third exposure times to capture the fourth image of the first scene area.

19. The apparatus of claim 16, wherein said at least one of said first, second and third optical chains uses a fourth exposure time different from said first, second and third exposure times to capture said fourth image of the first scene area.

20. A non-transitory computer readable medium including computer executable instructions which when executed by a processor control an apparatus to:

operate a plurality of optical chains in parallel to capture images of a first scene area, wherein operating a plurality of optical chains includes controlling a first optical chain including a first lens mounted on a lens platter, a first light redirection device for redirecting light through another optical element of the first optical chain, said another optical element being another lens of the first optical chain or a filter of the first optical chain, said another optical element of the first optical chain being mounted on a first cylinder through which light passes prior to reaching first sensor of the first optical chain to capture a first image of the first scene area using a first exposure time, and controlling a second optical chain including a second lens mounted on the lens platter, a second light redirection device for redirecting light through another optical element of the second optical chain, said another optical element being another lens of the second optical chain or a filter of the second optical chain, said another optical element of the second optical chain being mounted on a second cylinder through which light passes prior to reaching a second sensor of the second optical chain to capture a second image of said first scene area using a second exposure time, at least said first and second exposure times being of different durations but overlapping in time; and

process said first and second images to generate a first combined image of said first scene area.