Synchronized image capture methods and apparatus

• US 9,557,520 B2
• Filed: 07/09/2014
• Issued: 01/31/2017
• Est. Priority Date: 10/18/2013
• Status: Active Grant

• Current Assignee: Blue River Technology Inc. (Deere & Company)
• Original Assignee: Light Labs Inc.
• Inventors: Laroia, Rajiv
• Primary Examiner(s): VU, NGOC YEN T
• Application Number:
  US14/327,518
• Publication Number:
  US 20150296154A1
• Time in Patent Office: 937 Days
• Field of Search: 348/218.1, 348/294, 348296-297, 348308-310, 348/302, 348/335, 348/337, 348373-374
• US Class Current: 1/1
• CPC Class Codes: G02B 13/0065 having a beam-folding prism or mirror
• CPC Class Codes: G02B 13/009 having zoom function
• CPC Class Codes: G02B 17/008 Systems specially adapted to form image relays or chained systems
• CPC Class Codes: G02B 7/04 with mechanism for focusing or varying magnification
• CPC Class Codes: G02B 7/09 adapted for automatic focusing or varying magnification
• CPC Class Codes: G02B 7/102 controlled by a microcomputer cameras with interchangeable lenses G03B17/14
• CPC Class Codes: G02B 7/1805 for prisms G02B7/181 takes precedence
• CPC Class Codes: G02B 7/182 for mirrors
• CPC Class Codes: G03B 13/36 Autofocus systems
• CPC Class Codes: G03B 17/17 with reflectors arranged in beam forming the photographic image, e.g. for reducing dimensions of camera
• CPC Class Codes: G03B 2205/0046 Movement of one or more optical elements for zooming
• CPC Class Codes: G03B 37/04 with cameras or projectors providing touching or overlapping fields of view
• CPC Class Codes: G03B 5/00 Adjustment of optical system relative to image or object surface other than for focusing
• CPC Class Codes: H04N 13/243 using three or more 2D image sensors
• CPC Class Codes: H04N 23/45 for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 23/51 Housings
• CPC Class Codes: H04N 23/54 Mounting of pick-up tubes, electronic image sensors, deviation or focusing coils
• CPC Class Codes: H04N 23/55 Optical parts specially adapted for electronic image sensors; Mounting thereof
• CPC Class Codes: H04N 23/57 Mechanical or electrical details of cameras or camera modules specially adapted for being embedded in other devices
• CPC Class Codes: H04N 23/631 Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters
• CPC Class Codes: H04N 23/67 : Focus control based on electronic image sensor signals
• CPC Class Codes: H04N 23/69 : Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
• CPC Class Codes: H04N 23/698 : for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 23/74 : by influencing the scene brightness using illuminating means
• CPC Class Codes: H04N 25/00 : Circuitry of solid-state image sensors [SSIS]; Control thereof
• CPC Class Codes: H04N 25/531 : by controlling rolling shutters in CMOS SSIS
• CPC Class Codes: H04N 25/745 : Circuitry for generating timing or clock signals
• CPC Class Codes: H04N 25/75 : Circuitry for providing, modifying or processing image signals from the pixel array
• CPC Class Codes: H04N 5/2622 : Signal amplitude transition in the zone between image portions, e.g. soft edges
• CPC Class Codes: H04N 5/2624 : for obtaining an image which is composed of whole input images, e.g. splitscreen
• CPC Class Codes: H04N 5/265 : Mixing

Abstract

Methods and apparatus relating to controlling optical chains (OCs) of a camera device to scan a scene area of interest, thereby capturing images of the scene area, in a synchronized manner are described. In various embodiments a synchronized rolling shutter read out of two or more image sensors included in two or more corresponding OCs is implemented controlling the sensors to read out rows of pixel values corresponding to a portion of the scene at the same time, e.g., concurrently. While two or more of the OCs are controlled to read out at the same time, some other OCs in the camera maybe controlled not to read out pixel values while other image sensors are reading out. In various embodiments the read out rate of the two or more sensors corresponding to two or more optical chains is controlled as a function of the focal lengths of the corresponding OCs.

Claims

1. A method of capturing images corresponding to a scene area of interest, the method comprising:

controlling a plurality of optical chains to read out pixel values corresponding to a portion of said scene area of interest at the same time, said plurality of optical chains including a first optical chain having a first focal length and a second optical chain having a second focal length, said second focal length being different from said first focal length, different portions of said scene area of interest being read out at different times, and

wherein said controlling includes implementing synchronized rolling shutter read out of a first image sensor included in said first optical chain and a second image sensor included in the second optical chain and controlling the read out rate of said first and second image sensors as a function of the focal length of the first and second optical chains.

2. The method of claim 1, wherein said first and second image sensors include the same number of rows of pixels.

3. The method of claim 2, wherein a top edge of the first image sensor of the first optical chain corresponds to a row of pixel values that are read out first as part of the rolling shutter read out of said first image sensor, said rolling shutter read out including sequential reading out of rows of pixel values beginning at the top edge of the first image sensor and proceeding to a bottom edge of said first image sensor;

and

wherein a top edge of the second image sensor of the second optical chain corresponds to a row of pixel values that are read out first as part of a rolling shutter read out of said second image sensor, said rolling shutter read out of the second image sensor including sequential reading out of rows of pixel values beginning at the top edge of the second image sensor and proceeding to a bottom edge of said second image sensor, the scan direction of the first and second image sensors being the same.

4. The method of claim 1,

wherein said first and second image sensors include the same number of pixels.

5. The method of claim 4, wherein the first focal length of the first optical chain is larger than the second focal length of the second optical chain, the method comprising:

controlling the first image sensor to read out a number of rows of pixels in said first image sensor in a period of time which is calculated as the ratio of the second focal length to the first focal length times the amount of time used to read out the same number of rows of pixels in said second image sensor.

6. The method of claim 5,

wherein the first focal length is twice the second focal length;

and

wherein the second image sensor is fully read out over a time period which is twice as long as a first time period used to fully read out said first image sensor.

7. The method of claim 6,

wherein controlling the plurality of optical chains to read out pixel values corresponding to a portion of said scene area of interest at the same time further comprises:

controlling a third image sensor of a third optical chain to read out pixel values corresponding to a third portion of said scene area of interest after a last row of pixel values is read out from said first image sensor;

wherein said first image sensor captures a first portion of said scene area of interest;

wherein said second image sensor captures a second portion of said scene area of interest which is larger than said first portion; and

wherein said third portion of said scene area of interest is positioned below said first portion of said scene area of interest.

8. The method of claim 7,

wherein said second image sensor captures at least substantially the same image areas as said first and third image sensors capture but during a readout time which is longer than an individual read out time of either of said first and third image sensors.

9. A camera device for capturing images corresponding to a scene area of interest, comprising:

a plurality of optical chains; and

a controller configured to control said plurality of optical chains to read out pixel values corresponding to a portion of said scene area of interest at the same time, said plurality of optical chains including a first optical chain having a first focal length and a second optical chain having a second focal length, said second focal length being different from said first focal length, different portions of said scene area of interest being read out at different times, said controller being further configured to implement synchronized rolling shutter read out of a first image sensor included in said first optical chain and a second image sensor included in the second optical chain and control the read out rate of said first and second image sensors as a function of the focal length of the first and second optical chains, as part of being configured to control said plurality of optical chains to read out pixel values corresponding to a portion of said scene area of interest at the same time.

10. The camera device of claim 9, wherein said first and second image sensors include the same number of rows of pixels.

11. The camera device of claim 10, wherein a top edge of the first image sensor of the first optical chain corresponds to a row of pixel values that are read out first as part of the rolling shutter read out of said first image sensor, said rolling shutter read out including sequential reading out of rows of pixel values beginning at the top edge of the first image sensor and proceeding to a bottom edge of said first image sensor;

and

wherein a top edge of the second image sensor of the second optical chain corresponds to a row of pixel values that are read out first as part of a rolling shutter read out of said second image sensor, said rolling shutter read out of the second image sensor including sequential reading out of rows of pixel values beginning at the top edge of the second image sensor and proceeding to a bottom edge of said second image sensor, the scan direction of the first and second image sensors being the same.

12. The camera device of claim 9,

wherein said first and second image sensors include the same number of pixels.

13. The camera device of claim 12, wherein the first focal length is larger than the second focal length;

and

wherein said controller is further configured to control the first image sensor to read out a number of rows of pixels in said first image sensor in a period of time which is calculated as the ratio of the second focal length to the first focal length times the amount of time used to read out the same number of rows of pixels in said second image sensor.

14. The camera device of claim 13,

wherein the first focal length is twice the second focal length;

and

wherein the controller is further configured to control the second image sensor to be fully read out over a time period which is twice as long as a first time period used to fully read out said first image sensor.

15. The camera device of claim 14,

wherein said first image sensor captures a first portion of said scene area of interest;

wherein said second image sensor captures a second portion of said scene area of interest which is larger than said first portion; and

wherein said controller is further configured to control a third image sensor of a third optical chain to read out pixel values corresponding to a third portion of said scene area of interest after a last row of pixel values is read out from said first image sensor, said third portion of said scene area of interest being positioned below said first portion of said scene area of interest.

16. The camera device of claim 15,

wherein said second image sensor captures at least substantially the same image area as said first and third image sensors capture but during a readout time which is longer than an individual read out time of either of said first and third image sensors.

17. A non-transitory computer readable medium comprising computer executable instructions, said non-transitory computer readable medium comprising:

instructions, which when executed by a processor, control a plurality of optical chains to read out pixel values corresponding to a portion of said scene area of interest at the same time, said plurality of optical chains including a first optical chain having a first focal length and a second optical chain having a second focal length, said second focal length being different from said first focal length, different portions of said scene area of interest being read out at different times; and

wherein said instructions, which when executed by said processor, control said plurality of optical chains to read out pixel values corresponding to a portion of said scene area of interest at the same time, include instructions for implementing synchronized rolling shutter read out of a first image sensor included in said first optical chain and a second image sensor included in the second optical chain and controlling the read out rate of said first and second image sensors as a function of the focal length of the first and second optical chains.

18. The non-transitory computer readable medium of claim 17,

wherein said first and second image sensors include the same number of rows of pixels.