Methods and apparatus for synchronized image capture using camera modules with different focal lengths

• US 10,009,530 B2
• Filed: 01/30/2017
• Issued: 06/26/2018
• Est. Priority Date: 10/18/2013
• Status: Active Grant

• Current Assignee: Blue River Technology Inc. (Deere & Company)
• Original Assignee: Light Labs Inc.
• Inventors: Laroia, Rajiv
• Primary Examiner(s): Vu, Ngoc-Yen
• Application Number:
  US15/419,186
• Publication Number:
  US 20170208230A1
• Time in Patent Office: 512 Days
• Field of Search: 3482181
• CPC Class Codes: G02B 13/0065 having a beam-folding prism or mirror
• CPC Class Codes: G02B 13/009 having zoom function
• CPC Class Codes: G02B 17/008 Systems specially adapted to form image relays or chained systems
• CPC Class Codes: G02B 7/04 with mechanism for focusing or varying magnification
• CPC Class Codes: G02B 7/09 adapted for automatic focusing or varying magnification
• CPC Class Codes: G02B 7/102 controlled by a microcomputer cameras with interchangeable lenses G03B17/14
• CPC Class Codes: G02B 7/1805 for prisms G02B7/181 takes precedence
• CPC Class Codes: G02B 7/182 for mirrors
• CPC Class Codes: G03B 13/36 Autofocus systems
• CPC Class Codes: G03B 17/17 with reflectors arranged in beam forming the photographic image, e.g. for reducing dimensions of camera
• CPC Class Codes: G03B 2205/0046 Movement of one or more optical elements for zooming
• CPC Class Codes: G03B 37/04 with cameras or projectors providing touching or overlapping fields of view
• CPC Class Codes: G03B 5/00 Adjustment of optical system relative to image or object surface other than for focusing
• CPC Class Codes: H04N 13/243 using three or more 2D image sensors
• CPC Class Codes: H04N 23/45 for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 23/51 Housings
• CPC Class Codes: H04N 23/54 Mounting of pick-up tubes, electronic image sensors, deviation or focusing coils
• CPC Class Codes: H04N 23/55 Optical parts specially adapted for electronic image sensors; Mounting thereof
• CPC Class Codes: H04N 23/57 Mechanical or electrical details of cameras or camera modules specially adapted for being embedded in other devices
• CPC Class Codes: H04N 23/631 Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters
• CPC Class Codes: H04N 23/67 : Focus control based on electronic image sensor signals
• CPC Class Codes: H04N 23/69 : Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
• CPC Class Codes: H04N 23/698 : for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 23/74 : by influencing the scene brightness using illuminating means
• CPC Class Codes: H04N 25/00 : Circuitry of solid-state image sensors [SSIS]; Control thereof
• CPC Class Codes: H04N 25/531 : by controlling rolling shutters in CMOS SSIS
• CPC Class Codes: H04N 25/745 : Circuitry for generating timing or clock signals
• CPC Class Codes: H04N 25/75 : Circuitry for providing, modifying or processing image signals from the pixel array
• CPC Class Codes: H04N 5/2622 : Signal amplitude transition in the zone between image portions, e.g. soft edges
• CPC Class Codes: H04N 5/2624 : for obtaining an image which is composed of whole input images, e.g. splitscreen
• CPC Class Codes: H04N 5/265 : Mixing

Abstract

Methods and apparatus relating to controlling optical chains (OCs) of a camera device to scan a scene area of interest, thereby capturing images of the scene area, in a synchronized manner are described. In various embodiments a synchronized rolling shutter read out of two or more image sensors included in two or more corresponding OCs is implemented controlling the sensors to read out rows of pixel values corresponding to a portion of the scene at the same time, e.g., concurrently. While two or more of the OCs are controlled to read out at the same time, some other OCs in the camera maybe controlled not to read out pixel values while other image sensors are reading out. In various embodiments the read out rate of the two or more sensors corresponding to two or more optical chains is controlled as a function of the focal lengths of the corresponding OCs.

Claims

1. A method of capturing images corresponding to a scene area of interest, the method comprising:

controlling a plurality of optical chains to capture pixel values corresponding to a portion of said scene area of interest in a synchronized manner, said plurality of optical chains including a first optical chain having a first focal length and a second optical chain having a second focal length, said second focal length being different from said first focal length, different portions of said scene area of interest being captured at different times, and

wherein said controlling includes controlling a first image sensor to scan the scene area at a first rate and controlling a second image sensor to scan the scene area at a second rate, the first rate being a function of the focal length of said first optical chain, the second rate being different from the first rate and being a function of the focal length of said second optical chain.

2. The method of claim 1, wherein said first and second image sensors include the same number of rows of pixels.

3. The method of claim 2, wherein a top edge of the first image sensor of the first optical chain corresponds to a row of pixel values that are captured as part of rolling shutter operation of said first image sensor, said rolling shutter operation including sequential capture of rows of pixel values beginning at the top edge of the first image sensor and proceeding to a bottom edge of said first image sensor;

and

wherein a top edge of the second image sensor of the second optical chain corresponds to a row of pixel values that are captured as part of rolling shutter operation of said second image sensor, said rolling shutter operation of the second image sensor including sequential capture of rows of pixel values beginning at the top edge of the second image sensor and proceeding to a bottom edge of said second image sensor, the scan direction of the first and second image sensors being the same.

4. The method of claim 1,

wherein said first and second image sensors include the same number of pixels.

5. The method of claim 4,

wherein the first focal length of the first optical chain is larger than the second focal length of the second optical chain;

and

wherein controlling the plurality of optical chains to capture pixel values corresponding to a portion of said scene area of interest includes;

controlling the first image sensor to capture a number of rows of pixels in said first image sensor in a period of time which is calculated as the ratio of the second focal length to the first focal length times the amount of time used to capture the same number of rows of pixels in said second image sensor.

6. The method of claim 5,

wherein the first focal length is twice the second focal length;

and

wherein the second image sensor scans the scene area of interest in a time period which is twice as long as a first time period used by said first image sensor to scan the scene area of interest.

7. The method of claim 6,

wherein controlling the plurality of optical chains to capture pixel values corresponding to a portion of said scene area of interest further includes:

controlling a third image sensor of a third optical chain to capture pixel values corresponding to a third portion of said scene area of interest after a last row of pixel values is captured by said first image sensor;

wherein said first image sensor captures a first portion of said scene area of interest;

wherein said second image sensor captures a second portion of said scene area of interest which is larger than said first portion; and

wherein said third portion of said scene area of interest is positioned below said first portion of said scene area of interest.

8. The method of claim 7,

wherein said second image sensor captures at least substantially the same image areas as said first and third image sensors capture but during a time period which is longer than an individual time period used by either of said first and third image sensors to capture pixel values of the scene area of interest.

9. A camera device for capturing images corresponding to a scene area of interest, comprising:

a plurality of optical chains; and

a controller configured to control said plurality of optical chains to capture pixel values corresponding to a portion of said scene area of interest in a synchronized manner, said plurality of optical chains including a first optical chain having a first focal length and a second optical chain having a second focal length, said second focal length being different from said first focal length, different portions of said scene area of interest being captured at different times under control of said controller, said controller controlling a first image sensor to scan the scene area at a first rate and controlling a second image sensor to scan the scene area at a second rate, the first rate being a function of the focal length of said first optical chain, the second rate being different from the first rate and being a function of the focal length of said second optical chain.

10. The camera device of claim 9, wherein said first and second image sensors include the same number of rows of pixels.

11. The camera device of claim 10, wherein a top edge of the first image sensor of the first optical chain corresponds to a row of pixel values that are captured as part of rolling shutter operation of said first image sensor, said rolling shutter operation including sequential capture of rows of pixel values beginning at the top edge of the first image sensor and proceeding to a bottom edge of said first image sensor;

and

wherein a top edge of the second image sensor of the second optical chain corresponds to a row of pixel values that are captured as part of rolling shutter operation of said second image sensor, said rolling shutter operation of the second image sensor including sequential capture of rows of pixel values beginning at the top edge of the second image sensor and proceeding to a bottom edge of said second image sensor, the scan direction of the first and second image sensors being the same.

12. The camera device of claim 9,

wherein said first and second image sensors include the same number of pixels.

13. The camera device of claim 12, wherein the first focal length is larger than the second focal length;

and

wherein said controller is further configured to control the first image sensor to capture a number of rows of pixels in said first image sensor in a period of time which is calculated as the ratio of the second focal length to the first focal length times the amount of time used to capture the same number of rows of pixels in said second image sensor.

14. The camera device of claim 13,

wherein the first focal length is twice the second focal length;

and

wherein the second image sensor scans the scene area of interest in a time period which is twice as long as a first time period used by said first image sensor to scan the scene area of interest.

15. The camera device of claim 14,

wherein said first image sensor captures a first portion of said scene area of interest;

wherein said second image sensor captures a second portion of said scene area of interest which is larger than said first portion; and

wherein said controller is further configured to control a third image sensor of a third optical chain to capture pixel values corresponding to a third portion of said scene area of interest after a last row of pixel values is captured by said first image sensor, said third portion of said scene area of interest being positioned below said first portion of said scene area of interest.

16. The camera device of claim 15,

wherein said second image sensor captures at least substantially the same image area as said first and third image sensors capture but during a time period which is longer than a time period used by either of said first and third image sensors to capture portions of the scene area of interest.

17. A non-transitory computer readable medium comprising computer executable instructions, said non-transitory computer readable medium comprising:

instructions, which when executed by a processor, control a plurality of optical chains to capture pixel values corresponding to a portion of said scene area of interest in a synchronized manner, said plurality of optical chains including a first optical chain having a first focal length and a second optical chain having a second focal length, said second focal length being different from said first focal length, different portions of said scene area of interest being captured at different times, and

wherein said controlling includes controlling a first image sensor to scan the scene area at a first rate and controlling a second image sensor to scan the scene area at a second rate, the first rate being a function of the focal length of said first optical chain, the second rate being different from the first rate and being a function of the focal length of said second optical chain.

18. The non-transitory computer readable medium of claim 17,

wherein said first and second image sensors include the same number of rows of pixels.