Methods and apparatus for controlling sensors to capture images in a synchronized manner

• US 9,551,854 B2
• Filed: 07/09/2014
• Issued: 01/24/2017
• Est. Priority Date: 10/18/2013
• Status: Active Grant

• Current Assignee: Blue River Technology Inc. (Deere & Company)
• Original Assignee: Light Labs Inc.
• Inventors: Laroia, Rajiv
• Primary Examiner(s): SPINKS, ANTOINETTE T
• Application Number:
  US14/327,517
• Publication Number:
  US 20150296153A1
• Time in Patent Office: 930 Days
• Field of Search: 348/262, 348/218.1, 348/294, 348/335, 348/337, 348/373, 348/374
• US Class Current: 1/1
• CPC Class Codes: G02B 13/0065 having a beam-folding prism or mirror
• CPC Class Codes: G02B 13/009 having zoom function
• CPC Class Codes: G02B 17/008 Systems specially adapted to form image relays or chained systems
• CPC Class Codes: G02B 7/04 with mechanism for focusing or varying magnification
• CPC Class Codes: G02B 7/09 adapted for automatic focusing or varying magnification
• CPC Class Codes: G02B 7/102 controlled by a microcomputer cameras with interchangeable lenses G03B17/14
• CPC Class Codes: G02B 7/1805 for prisms G02B7/181 takes precedence
• CPC Class Codes: G02B 7/182 for mirrors
• CPC Class Codes: G03B 13/36 Autofocus systems
• CPC Class Codes: G03B 17/17 with reflectors arranged in beam forming the photographic image, e.g. for reducing dimensions of camera
• CPC Class Codes: G03B 2205/0046 Movement of one or more optical elements for zooming for optical zoom lenses in general G02B7/10
• CPC Class Codes: G03B 37/04 with cameras or projectors providing touching or overlapping fields of view
• CPC Class Codes: G03B 5/00 Adjustment of optical system relative to image or object surface other than for focusing
• CPC Class Codes: H04N 13/243 using three or more 2D image sensors
• CPC Class Codes: H04N 23/45 for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 23/51 Housings
• CPC Class Codes: H04N 23/54 Mounting of pick-up tubes, electronic image sensors, deviation or focusing coils
• CPC Class Codes: H04N 23/55 Optical parts specially adapted for electronic image sensors; Mounting thereof
• CPC Class Codes: H04N 23/57 Mechanical or electrical details of cameras or camera modules specially adapted for being embedded in other devices
• CPC Class Codes: H04N 23/631 Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters
• CPC Class Codes: H04N 23/67 : Focus control based on electronic image sensor signals
• CPC Class Codes: H04N 23/69 : Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
• CPC Class Codes: H04N 23/698 : for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 23/74 : by influencing the scene brightness using illuminating means
• CPC Class Codes: H04N 25/00 : Circuitry of solid-state image sensors [SSIS]; Control thereof
• CPC Class Codes: H04N 25/531 : by controlling rolling shutters in CMOS SSIS
• CPC Class Codes: H04N 25/745 : Circuitry for generating timing or clock signals
• CPC Class Codes: H04N 25/75 : Circuitry for providing, modifying or processing image signals from the pixel array
• CPC Class Codes: H04N 5/2622 : Signal amplitude transition in the zone between image portions, e.g. soft edges
• CPC Class Codes: H04N 5/2624 : for obtaining an image which is composed of whole input images, e.g. splitscreen
• CPC Class Codes: H04N 5/265 : Mixing

Abstract

Methods and apparatus relating to capturing images of a scene area in a synchronized manner using a plurality of optical chains are described. In various embodiments image sensors corresponding to the plurality of optical chains of a camera are operated in rolling shutter mode to read out rows of pixel values corresponding to a current scan position when the image sensors have a row of pixel values corresponding to the current scan position. In some embodiments a scene area of interest is captured by initiating a scan and thus image capture of the scene area by one or more optical chains which are operated in a coordinated manner. In some embodiments a controller controls the plurality of image sensors to perform a read out of pixel values in a synchronized manner, e.g., with rows of pixel values being read out sequentially in accordance with operation of a rolling shutter implementation.

Claims

1. A method of operating a camera device to scan a portion of a scene area of interest, the method comprising:

controlling a plurality of image sensors to scan the scene area of interest from a first edge of said scene area of interest to a second edge of said scene area of interest, said first and second edges of said scene area of interest being parallel, said scanning including controlling image capture according to a scan position which progresses from said first edge to said second edge of the scene area of interest, said controlling including operating a first image sensor in said plurality of image sensors in rolling shutter mode to read out a row of pixel values corresponding to a current scan position when said first image sensor has a row of pixel values corresponding to said current scan position, said current scan position changing over time, said controlling further including operating a second image sensor in said plurality of image sensors in rolling shutter mode to read out a row of pixel values corresponding to the current scan position when said second image sensor has a row of pixel values corresponding to said current scan position, said first and second image sensors capturing images of different portions of said scene area of interest;

wherein said first image sensor corresponds to an optical chain having a first focal length;

wherein said second image sensor corresponds to an optical chain having a second focal length; and

wherein said controlling includes controlling the rate at which pixel rows are read out of the first and second image sensors as a function of the focal length of the individual optical chains to which the first and second image sensors correspond.

2. The method of claim 1,

wherein said first image sensor captures an image of a portion of the scene area of interest which is not captured by the second image sensor;

and

wherein said second image sensor captures an image of a portion of said scene area of interest which is not captured by said first image sensor.

3. The method of claim 1, wherein said first and second image sensors correspond to first and second portions of the scene area of interest, respectively, the extent of the first portion of the scene area of interest being greater than the extent of the second portion of the scene area of interest in the direction of scan, said controlling further including:

controlling the duration of readout of the first image sensor to have a longer duration than the readout of the second image sensor.

4. The method of claim 1, wherein said first image sensor captures a smaller portion of said scene area of interest than said second image sensor.

5. The method of claim 1, wherein a pixel size of the first image sensor is the same as a pixel size of the second image sensor.

6. The method of claim 1, wherein the first focal length is larger than the second focal length, said first focal length being N times the second focal length, said step of controlling the first image sensor including controlling the first image sensor to read out rows of pixel values out of the first image sensor at a rate which is N times faster than the rate at which rows of pixel values are read out from the second image sensor.

7. The method of claim 1, wherein said plurality of image sensors include multiple image sensors which capture substantially non-overlapping portions of said scene area of interest and wherein a union of the substantially non-overlapping portions of said scene area of interest captured by said multiple image sensors covers the scene area of interest.

8. The method of claim 1,

wherein the first focal length is smaller than said second focal length;

and

wherein a first portion of said scene area of interest captured by the first image sensor begins at a scan location which precedes a second portion of the scene area of interest captured by the second image sensor.

9. The method of claim 1, wherein said plurality of image sensors includes at least 5 sensors, four of the at least 5 sensors capturing different substantially non-overlapping portions of said scene of interest and a fifth one of said 5 sensors capturing a scene area including the different substantially non-overlapping portions of said scene area of interest.

10. The method of claim 1, wherein said plurality of image sensors includes at least 5 sensors, four of which capture different corner portions of said scene area of interest.

11. The method of claim 1,

wherein said plurality of image sensors includes a third image sensor;

wherein said first image sensor captures a first portion of said scene area of interest;

wherein said third image sensor captures a second portion of said scene area of interest, wherein said first and second portions of said scene area of interest are partially non-overlapping; and

wherein said second image sensor captures the entire scene area of interest.

21. The method of claim 1,

wherein said first image sensor captures an image of a portion of the scene area of interest which is captured by the second image sensor.

22. The method of claim 1,

wherein said images of different portions of said scene area of interest captured by said first and second image sensors at least partially overlap, and

wherein operating the second image sensor includes operating the second image sensor to capture a first overlapping portion of the scene area of interest at the same time the first sensor is operated to capture the first overlapping portion of the scene area of interest.

23. The method of claim 22, wherein said images of different portions of said scene area of interest captured by said first and second image sensors at least partially overlap and partially do not overlap;

and

wherein operating the second image sensor includes operating the second image sensor to capture a first non-overlapping portion of the scene area of interest after said first sensor has completed capturing portions of said scene area of interest to which the first sensor corresponds.

24. The method of claim 22,

wherein the first and second image sensors have the same number of rows of pixels;

and

wherein said second image sensor is included in a second optical chain which has a focal length shorter than a focal length of the first optical chain which includes the first sensor.

12. A camera device, comprising:

a plurality of optical chains; and

a sensor controller configured to control a plurality of image sensors, corresponding to said plurality of optical chains, to scan a scene area of interest from a first edge of said scene area of interest to a second edge of said scene area of interest, said first and second edges of said scene area of interest being parallel, said sensor controller controlling, as part of said scan, image capture according to a scan position which progresses from said first edge to said second edge of the scene area of interest, said sensor controller controlling a first image sensor in said plurality of image sensors in rolling shutter mode to read out a row of pixel values corresponding to a current scan position when said first image sensor has a row of pixel values corresponding to said current scan position, said current scan position changing over time, said sensor controller further controlling a second image sensor in said plurality of image sensors in rolling shutter mode to read out a row of pixel values corresponding to the current scan position when said second image sensor has a row of pixel values corresponding to said current scan position, said first and second image sensors capturing images of different portions of said scene area of interest;

wherein said first image sensor corresponds to a first optical chain having a first focal length and said second image sensor corresponds to a second optical chain having a second focal length; and

wherein said sensor controller is configured to control the rate at which pixel rows are read out of the first and second image sensors as a function of the focal length of the individual optical chains to which the first and second image sensors correspond.

13. The camera device of claim 12, wherein said first image sensor captures an image of a portion of the scene area of interest which is not captured by the second image sensor;

and

wherein said second image sensor captures an image of a portion of said scene area of interest which is not captured by said first image sensor.

14. The camera device of claim 12, wherein said first and second image sensors correspond to first and second portions of the scene area of interest, respectively, the extent of the first portion of the scene area of interest being greater than the extent of the second portion of the scene area of interest in the direction of scan, said sensor controller being further configured to:

control the duration of readout of the first image sensor to have a longer duration than the readout of the second image sensor.

15. The camera device of claim 12, wherein said first image sensor corresponds to an optical chain having a first focal length which is smaller than the second focal length by a factor of N times.

16. The camera device of claim 12, wherein said plurality of image sensors include multiple image sensors which capture substantially non-overlapping portions of said scene area of interest and wherein a union of the portions of said scene area of interest captured by said multiple image sensors cover the scene area of interest.

17. The camera device of claim 12, wherein a first portion of said scene area of interest captured by the first image sensor begins at a scan location which precedes a second portion of the scene area of interest captured by the second image sensor.

18. The camera device of claim 17, wherein said plurality of image sensors includes at least 5 sensors, four different ones of the at least five sensors capturing substantially non-overlapping corner portions of said scene of interest and a fifth one of said 5 sensors capturing a scene area including the four substantially non-overlapping corner portions of said scene area of interest.

19. The camera device of claim 12,

wherein said plurality of image sensors includes a third image sensor which is part of a third optical chain having the first focal length;

wherein said first image sensor captures a first portion of said scene area of interest;

wherein said third image sensor captures a second portion of said scene area of interest, wherein said first and second portions of said scene area of interest are partially non-overlapping; and

wherein said second image sensor captures the entire scene area of interest.

20. A non-transitory computer readable medium, said non-transitory computer readable medium comprising computer executable instructions, said non-transitory computer readable medium comprising:

instructions, which when executed by a processor, control a plurality of image sensors to scan the scene area of interest from a first edge of said scene area of interest to a second edge of said scene area of interest, said first and second edges of said scene area of interest being parallel, said scanning including controlling image capture according to a scan position which progresses from said first edge to said second edge of the scene area of interest, said instructions causing said processor to control a first image sensor in said plurality of image sensors in rolling shutter mode to read out a row of pixel values corresponding to a current scan position when said first image sensor has a row of pixel values corresponding to said current scan position, said current scan position changing over time, said instructions further causing said processor to control a second image sensor in said plurality of image sensors in rolling shutter mode to read out a row of pixel values corresponding to the current scan position when said second image sensor has a row of pixel values corresponding to said current scan position, said first and second image sensors capturing images of different portions of said scene area of interest;

wherein said first image sensor corresponds to an optical chain having a first focal length;

wherein said second image sensor corresponds to an optical chain having a second focal length; and

wherein said processor is configured to control the rate at which pixel rows are read out of the first and second image sensors as a function of the focal length of the individual optical chains to which the first and second image sensors correspond.