Methods and apparatus for using multiple optical chains in parallel to support separate color-capture

• US 9,568,713 B2
• Filed: 01/05/2014
• Issued: 02/14/2017
• Est. Priority Date: 01/05/2013
• Status: Active Grant

• Current Assignee: Blue River Technology Inc. (Deere & Company)
• Original Assignee: Light Labs Inc.
• Inventors: Laroia, Rajiv
• Primary Examiner(s): Peterson, Christopher K
• Application Number:
  US14/147,583
• Publication Number:
  US 20140192224A1
• Time in Patent Office: 1,136 Days
• Field of Search: 348/218.1, 348/335, 348/336, 348/340, 250/208.1
• US Class Current: 1/1
• CPC Class Codes: G02B 13/0065 having a beam-folding prism or mirror
• CPC Class Codes: G02B 13/16 for use in conjunction with image converters or intensifiers , or for use with projectors, e.g. objectives for projection TV
• CPC Class Codes: G02B 13/22 Telecentric objectives or lens systems
• CPC Class Codes: G02B 17/008 Systems specially adapted to form image relays or chained systems
• CPC Class Codes: G02B 17/0896 with variable magnification or multiple imaging planes, including multispectral systems systems with only refractive elements G02B15/14
• CPC Class Codes: G02B 3/0056 arranged along two different directions in a plane, e.g. honeycomb arrangement of lenses G02B3/0043 takes precedence; miniaturised objectives for electronic devices employing wafer level optics G02B13/0085
• CPC Class Codes: G02B 7/18 for prisms; for mirrors
• CPC Class Codes: G03B 17/17 with reflectors arranged in beam forming the photographic image, e.g. for reducing dimensions of camera
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: H04N 23/45 for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 23/54 Mounting of pick-up tubes, electronic image sensors, deviation or focusing coils
• CPC Class Codes: H04N 23/55 Optical parts specially adapted for electronic image sensors; Mounting thereof
• CPC Class Codes: H04N 23/57 Mechanical or electrical details of cameras or camera modules specially adapted for being embedded in other devices
• CPC Class Codes: H04N 23/73 by influencing the exposure time
• CPC Class Codes: H04N 23/741 by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors
• CPC Class Codes: H04N 23/75 by influencing optical camera components
• CPC Class Codes: H04N 5/265 Mixing

Abstract

Methods and apparatus for capturing or generating images using multiple optical chains operating in parallel are described. Pixel values captured by individual optical chains corresponding to the same scene area are combined to provide an image with at least some of the benefits which would have been provided by capturing an image of the scene using a larger lens than that of the individual lenses of the optical chain modules. By using multiple optical chains in parallel at least some benefits normally obtained from using a large lens can be obtained without the need for a large lens. Furthermore in at least some embodiments, a wide dynamic range can be supported through the use of multiple sensors with the overall supported dynamic range being potentially larger than that of the individual sensors. Some lens and/or optical chain configurations are designed for use in small handheld devices, e.g., cell phones.

Claims

1. A method of operating a camera device including at least seven optical chain modules, at least four of the seven optical chain modules having different filters, the method comprising:

operating a first optical chain module including a first autofocus drive, a first multi-megapixel sensor, and a multicolor filter including a red filter element, a green filter element and a blue filter element to capture a multicolor image for preview purposes, said multicolor filter filtering light passing through a first opening of said first optical chain module;

presenting, on a display of a device, the multicolor image captured by the first optical chain module including the multicolor filter as a preview image;

controlling separate autofocus drives in a plurality of additional separate optical chain modules in response to user input received while said preview image is displayed, said plurality of additional separate optical chain modules being located in said camera device around said first optical chain module, said additional separate optical chain modules including;

a second optical chain module having a second autofocus drive and a second opening through which light passes to a second multi-megapixel sensor included in said second optical chain module, a center of said second opening being a first distance from a center of the first opening of said first optical chain module; and

a third optical chain module having a third autofocus drive and a third opening through which light passes to a third multi-megapixel sensor included in said third optical chain module, said first, second and third sensors being separate sensors, said first, second and third autofocus drives being separate autofocus drives; and

wherein said plurality of additional separate optical chain modules includes;

i) two optical chain modules having a red filter ii) two optical chain modules having a green filter; and

iii) two optical chain modules having a blue filter; and

wherein said two optical chain modules having a red filter are positioned on opposite sides of said first optical chain module; and

wherein said plurality of additional separate optical chain modules are configured in a circle with the first optical chain module including said multicolor filter at the center of the circle, an opening of the two additional separate optical chain modules having a red filter being on said circle, an opening of the two separate additional optical chain modules having a green filter being on said circle and an opening of the two separate optical chain modules having a blue filter being on said circle.

2. The method of claim 1,

wherein said first, second and third multi-megapixel sensors are separate sensors having at least 8 megapixels.

3. The method of claim 2,

wherein controlling separate autofocus drives in the plurality of additional separate optical chain modules in response to user input includes operating an autofocus controller to control the individual optical chain modules in said plurality of additional separate optical chain modules to focus based on a distance from the camera device to a user selected object.

4. The method of claim 3,

wherein distance between optical centers of adjacent individual optical chain modules on said circle is non-uniform with the distance between optical centers of at least one pair of optical chain modules on said circle being different than the distance between optical centers of another pair of adjacent optical chain modules on said circle.

5. The method of claim 4, further comprising:

controlling the two optical chain modules having a red filter to use different exposure times.

6. The method of claim 1, wherein the camera device includes more camera modules with single color filters than with multicolor filters.

7. The method of claim 6, wherein said two optical chain modules having a green filter are positioned on opposite sides of said first optical chain module with the first optical chain module being in the center between the two optical chain modules having the green filter.

8. The method of claim 1, wherein said camera device is a camera.

9. The method of claim 1, wherein said camera device is a mobile cellular phone.

10. An apparatus, comprising:

a first optical chain module for capturing a first multicolor image, said first optical chain module including a first autofocus drive, a first multi-megapixel sensor, and a multicolor filter including a red filter element, a green filter element and a blue filter element for filtering light passing through a first opening of said first optical chain module;

a display for presenting the multicolor image captured by the first optical chain module including the multicolor filter as a preview image;

a controller configured to control a plurality of additional separate optical chain modules in response to user input received while said preview image is displayed;

said plurality of additional separate optical chain modules being located in said apparatus around said first optical chain module, said plurality of additional separate optical chain modules including;

a second optical chain module having a second autofocus drive and a second opening through which light passes to a second multi-megapixel sensor included in said second optical chain module; and

a third optical chain module having a third autofocus drive and a third opening through which light passes to a third multi-megapixel sensor included in said third optical chain module, said first, second and third autofocus drives being separate autofocus drives;

wherein said plurality of additional separate optical chain modules includes two optical chain modules having a red filter, two optical chain modules having a green filter, and two optical chain modules having a blue filter;

wherein said two optical chain modules having a red filter are positioned on opposite sides of said first optical chain module;

wherein said plurality of additional separate optical chain modules includes;

i) two optical chain modules having a red filter, ii) two optical chain modules having a green filter, and iii) two optical chain modules having a blue filter;

wherein said two optical chain modules having a red filter are positioned on opposite sides of said first optical chain module;

wherein said plurality of additional separate optical chain modules are configured in a circle with the first optical chain module including said multicolor filter at the center of the circle; and

an image processing module configured to process images captured by at least said second and third optical chain modules.

11. The apparatus of claim 10,

wherein said first, second and third multi-megapixel sensors are separate sensors having at least 8 megapixels.

12. The apparatus of claim 11,

wherein said controller is an autofocus controller configured to control the focus distance of individual optical chain modules in said plurality of optical chain modules based on a user selected object.

13. The apparatus of claim 12, wherein the apparatus includes more camera modules with single color filters than with multicolor filters.

14. The apparatus of claim 13, wherein said two optical chain modules having a green filter are positioned on opposite sides of said first optical chain module with the first optical chain module being in the center between the two optical chain modules having the green filter.

15. The apparatus of claim 12, wherein said apparatus is a camera;

wherein said first opening is a first lens opening for a first lens included in said first optical chain module and having a first optical center;

wherein said second opening is a second lens opening for a second lens included in said second optical chain module and having a second optical center; and

wherein said third opening is a third lens opening for a third lens included in said third optical chain module and having a third optical center, a distance from said second optical center to said first optical center being different from a distance from said third optical center to said first optical center.

16. The apparatus of claim 12, wherein said apparatus is a mobile cellular phone.

17. A non-transitory computer readable medium including computer executable instructions which when executed by a processor control an apparatus to:

operate a first optical chain module including a first autofocus drive, a first multi-megapixel sensor, and a multicolor filter including a red filter element, a green filter element and a blue filter element to capture a multicolor image for preview purposes, said multicolor filter filtering light passing through a first opening of said first optical chain module;

operate a display to present the multicolor image captured by the first optical chain module including the multicolor filter as a preview image;

operate a controller to control separate autofocus drives in a plurality of additional separate optical chain modules in response to user input received while said preview image is displayed, said plurality of additional separate optical chain modules being located in said apparatus around said first optical chain module, said additional separate optical chain modules including;

a second optical chain module having a second autofocus drive and a second opening through which light passes to a second sensor included in said second optical chain module, a center of said second opening being a first distance from a center of the first opening of said first optical chain module; and

a third optical chain module having a third autofocus drive and a third opening through which light passes to a third sensor included in said third optical chain module, said first, second and third sensors being separate sensors, said first, second and third autofocus drives being separate autofocus drives; and

wherein said plurality of additional separate optical chain modules includes;

i) two optical chain modules having a red filter, ii) two optical chain modules having a green filter, and iii) two optical chain modules having a blue filter; and

wherein said two optical chain modules having a red filter are positioned on opposite sides of said first optical chain module; and

wherein said plurality of additional separate optical chain modules are configured in a circle with the first optical chain module including said multicolor filter at the center of the circle.

18. The non-transitory computer readable medium of claim 17,

wherein said controller is an autofocus controller configured to control the focus distance of individual optical chain modules in said plurality of optical chain modules based on the user selected object.

19. The non-transitory computer readable medium of claim 18, wherein said apparatus is a camera.