Image capture control methods and apparatus

• US 10,274,706 B2
• Filed: 01/16/2017
• Issued: 04/30/2019
• Est. Priority Date: 10/18/2013
• Status: Active Grant

• Current Assignee: Blue River Technology Inc. (Deere & Company)
• Original Assignee: Light Labs Inc.
• Inventors: Laroia, Rajiv
• Primary Examiner(s): Fosselman, Joel W
• Application Number:
  US15/407,236
• Publication Number:
  US 20170126976A1
• Time in Patent Office: 834 Days
• Field of Search: 348340
• CPC Class Codes: G02B 13/001 Miniaturised objectives for electronic devices, e.g. portable telephones, webcams, PDAs, small digital cameras G02B23/243 takes precedence
• CPC Class Codes: G02B 13/0055 employing a special optical element
• CPC Class Codes: G02B 13/0065 having a beam-folding prism or mirror
• CPC Class Codes: G02B 13/009 having zoom function
• CPC Class Codes: G02B 13/02 Telephoto objectives, i.e. systems of the type + - in which the distance from the front vertex to the image plane is less than the equivalent focal length
• CPC Class Codes: G02B 15/00 Optical objectives with means for varying the magnification anamorphotic objectives G02B13/08
• CPC Class Codes: G02B 17/008 Systems specially adapted to form image relays or chained systems
• CPC Class Codes: G02B 17/0896 with variable magnification or multiple imaging planes, including multispectral systems systems with only refractive elements G02B15/14
• CPC Class Codes: G02B 27/1066 for enhancing image performance, like resolution, pixel numbers, dual magnifications or dynamic range, by tiling, slicing or overlapping fields of view
• CPC Class Codes: G02B 3/0037 characterized by the distribution or form of lenses
• CPC Class Codes: G02B 5/005 Diaphragms for cameras G03B9/02
• CPC Class Codes: G02B 5/045 Prism arrays
• CPC Class Codes: G02B 5/08 Mirrors vehicle mirrors involving special optical features B60R1/08
• CPC Class Codes: G02B 7/282 Autofocusing of zoom lenses
• CPC Class Codes: H04N 23/45 for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 23/51 Housings
• CPC Class Codes: H04N 23/55 Optical parts specially adapted for electronic image sensors; Mounting thereof
• CPC Class Codes: H04N 23/58 Means for changing the camera field of view without moving the camera body, e.g. nutating or panning of optics or image sensors
• CPC Class Codes: H04N 23/631 Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters
• CPC Class Codes: H04N 23/667 Camera operation mode switching, e.g. between still and video, sport and normal or high- and low-resolution modes
• CPC Class Codes: H04N 23/67 : Focus control based on electronic image sensor signals
• CPC Class Codes: H04N 23/6811 : based on the image signal
• CPC Class Codes: H04N 23/6812 : based on additional sensors, e.g. acceleration sensors
• CPC Class Codes: H04N 23/685 : performed by mechanical compensation
• CPC Class Codes: H04N 23/689 : Motion occurring during a rolling shutter mode
• CPC Class Codes: H04N 23/69 : Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
• CPC Class Codes: H04N 23/73 : by influencing the exposure time
• CPC Class Codes: H04N 23/741 : by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors
• CPC Class Codes: H04N 23/80 : Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 23/951 : by using two or more images to influence resolution, frame rate or aspect ratio
• CPC Class Codes: H04N 25/00 : Circuitry of solid-state image sensors [SSIS]; Control thereof
• CPC Class Codes: H04N 25/42 : by switching between different modes of operation using different resolutions or aspect ratios, e.g. switching between interlaced and non-interlaced mode
• CPC Class Codes: H04N 25/44 : by partially reading an SSIS array
• CPC Class Codes: H04N 25/70 : SSIS architectures; Circuits associated therewith
• CPC Class Codes: H04N 25/75 : Circuitry for providing, modifying or processing image signals from the pixel array
• CPC Class Codes: H04N 5/2628 : Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation
• CPC Class Codes: H04N 5/265 : Mixing
• CPC Class Codes: H04N 5/268 : Signal distribution or switching

Abstract

Methods and apparatus for controlling the read out of rows of pixel values from sensors corresponding to different optical chains used to capture portions of the same image area are described. The readout is controlled based on user input and/or determinations with regard to the rate of motion in captured images or portions of captured images. For a low rate of motion i, the readout rate of a sensor corresponding to a small focal length is slowed down while the pixel row readout rate of one or more sensors corresponding to one or more optical chains have larger focal lengths are allowed to proceed at a normal rate. For a high rate of motion, the read out rate of the sensor corresponding to the optical chain having the smaller focal length is allowed to proceed at the normal rate.

Claims

1. A method of capturing images using a camera device including at least an optical chain including a sensor, the method comprising:

determining a rate of motion based on images captured by the camera device;

making a decision to control the sensor to;

i) perform joint readouts of pixel elements corresponding to consecutive pixel element rows of the sensor or ii) perform individual readouts of consecutive rows of pixel elements of said sensor based on a determined rate of motion; and

controlling the sensor to read out pixel values of pixel elements of said sensor in accordance with the said decision, controlling said sensor including controlling said sensor to perform either;

i) joint readouts of pixel elements corresponding to consecutive pixel element rows of the sensor or ii) individual readouts of consecutive rows of pixel elements of said sensor.

2. The method of claim 1, wherein

determining said rate of motion is based on images sequentially captured by said camera device.

3. The method of claim 2, wherein said determining includes determining the rate of motion based on differences between a first image captured by said camera device during a first time period and a second image captured by said camera device during a second time period which follows said first time period.

4. The method of claim 2, wherein said step of making a decision includes:

deciding to perform joint readouts of pixel elements corresponding to consecutive pixel element rows of the sensor when the rate of motion is above a first threshold.

5. The method of claim 2, wherein said step of making a decision includes:

deciding to perform individual readouts of consecutive rows of pixel elements of said sensor when the rate of motion is below said first threshold.

6. The method of claim 2, wherein controlling the sensor to perform joint readouts of pixel elements corresponding to consecutive pixel element rows of the sensor causes the sensor to generate a lower resolution image than is generated when performing individual readouts of consecutive rows of pixel elements of said sensor.

7. A method of capturing images using a camera device including at least an optical chain including a sensor, the method comprising:

receiving user input indicating user input indicating one of a high rate of motion mode of operation or another mode of operation;

making a decision to control the sensor to;

i) perform joint readouts of pixel elements corresponding to consecutive pixel element rows of the sensor or ii) perform individual readouts of consecutive rows of pixel elements of said sensor, said decision being based on the received user input, said user input indicating a high rate of motion mode of operation or another mode of operation, said step of making a decision to control the sensor including;

i) deciding to perform joint readouts of pixel elements corresponding to consecutive pixel element rows of the sensor when said user input indicates a high rate of motion mode of operation; and

ii) deciding to perform individual readouts of consecutive rows of pixel elements of said second sensor when the user input indicates selection of a high resolution motion mode of operation; and

controlling the sensor to read out pixel values of pixel elements of said sensor in accordance with the said decision, controlling said sensor including controlling said sensor to perform either;

i) joint readouts of pixel elements corresponding to consecutive pixel element rows of the sensor or ii) individual readouts of consecutive rows of pixel elements of said sensor.

8. The method of claim 7, wherein controlling the sensor to perform joint readouts of pixel elements corresponding to consecutive pixel element rows of the sensor causes the sensor to generate a lower resolution image than is generated when performing individual readouts of consecutive rows of pixel elements of said sensor.

9. A camera device comprising:

a user input for receiving input from a user;

an optical chain including a sensor; and

a processor configured to;

determine a rate of motion from images captured by said optical chain;

make a decision to control the sensor to;

i) perform joint readouts of pixel elements corresponding to consecutive pixel element rows of the sensor or ii) perform individual readouts of consecutive rows of pixel elements of said sensor, said decision being based the determined rate of motion; and

control the sensor to read out pixel values of pixel elements of said sensor in accordance with the said decision, controlling said sensor including controlling said sensor to perform either;

i) joint readouts of pixel elements corresponding to consecutive pixel element rows of the sensor or ii) individual readouts of consecutive rows of pixel elements of said sensor.

10. The camera device of claim 9, wherein the processor, as part of determining the rate of motion is

configured to:

determine said rate of motion from images sequentially captured by said sensor.

11. The camera device of claim 10, wherein said processor, as part of determining said rate of motion is configured to determine the rate of motion based on differences between a first image captured by said sensor during a first time period and a second image captured by said sensor during a second time period which follows said first time period.

12. The camera device of claim 11, wherein said processor is configured, as part of said step of making a decision, to:

decide to perform joint readouts of pixel elements corresponding to consecutive pixel element rows of the sensor when the rate of motion is above a first threshold.

13. The camera device of claim 12, wherein said processor is configured, as part of said step of making a decision, to:

decide to perform individual readouts of consecutive rows of pixel elements of said sensor when the rate of motion is below said first threshold.

14. The camera device of claim 9, wherein when the processor controls the sensor to perform joint readouts of pixel elements corresponding to consecutive pixel element rows of the sensor, the sensor generates a lower resolution image than when performing individual readouts of consecutive rows of pixel elements of said sensor.

15. A camera device comprising:

a user input for receiving input from a user indicating one of a high rate of motion mode of operation or another mode of operation;

an optical chain including a sensor; and

a processor configured to;

make a decision to control the sensor to;

i) perform joint readouts of pixel elements corresponding to consecutive pixel element rows of the sensor or ii) perform individual readouts of consecutive rows of pixel elements of said sensor based on user input indicating a high rate of motion mode of operation or another mode of operation, said processor being configured, as part of making the decision, to;

decide to perform joint readouts of pixel elements corresponding to consecutive pixel element rows of the sensor when said user input indicates a high rate of motion mode of operation; and

decide to perform individual readouts of consecutive rows of pixel elements of said sensor when the user input indicates selection of a high resolution motion mode of operation; and

control the sensor to read out pixel values of pixel elements of said sensor in accordance with the said decision, controlling said sensor including controlling said sensor to perform either;

i) joint readouts of pixel elements corresponding to consecutive pixel element rows of the sensor or ii) individual readouts of consecutive rows of pixel elements of said sensor.

16. The camera device of claim 15, wherein when the processor controls the sensor to perform joint readouts of pixel elements corresponding to consecutive pixel element rows of the sensor, the sensor generates a lower resolution image than when performing individual readouts of consecutive rows of pixel elements of said sensor.