Methods and apparatus for synchronizing readout of multiple image sensors

• US 10,091,447 B2
• Filed: 04/16/2016
• Issued: 10/02/2018
• Est. Priority Date: 04/17/2015
• Status: Active Grant

• Current Assignee: Blue River Technology Inc. (Deere & Company)
• Original Assignee: Light Labs Inc.
• Inventors: Laroia, Rajiv, Shroff, Nitesh
• Primary Examiner(s): Hsu, Amy
• Application Number:
  US15/130,966
• Publication Number:
  US 20160309110A1
• Time in Patent Office: 899 Days
• CPC Class Codes: H04N 23/45 for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 23/60 Control of cameras or camera modules
• CPC Class Codes: H04N 23/6812 based on additional sensors, e.g. acceleration sensors
• CPC Class Codes: H04N 23/69 Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
• CPC Class Codes: H04N 23/73 by influencing the exposure time
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 23/951 by using two or more images to influence resolution, frame rate or aspect ratio
• CPC Class Codes: H04N 25/445 by skipping some contiguous pixels within the read portion of the array
• CPC Class Codes: H04N 25/745 Circuitry for generating timing or clock signals
• CPC Class Codes: H04N 25/75 Circuitry for providing, modifying or processing image signals from the pixel array
• CPC Class Codes: H04N 5/2624 for obtaining an image which is composed of whole input images, e.g. splitscreen

Abstract

Methods and apparatus for reading out pixel values from sensors in a synchronized manner are described. Readout of rows of pixel values from different sensors are controlled so that pixel values of different sensors corresponding to the same portion of a scene are read out in a way that the same portions of a scene are captured at the same or nearly the same time by different sensors. In one embodiment a first sensor which captures a large scene area alternates between reading out rows of pixel values from a top portion and a bottom portion of the first sensor while sensors corresponding to smaller areas of the scene read out rows of pixel values in a consecutive manner. Sensors may read out rows of pixel values at the same rate despite corresponding to optical chains with different focal lengths. The image captured by the first sensor facilitates image combining.

Claims

1. A method of operating an apparatus, the method comprising:

alternating between reading out a row of pixel values from a top portion of a first sensor and a row of pixel values from a bottom portion of the first sensor;

reading out a row of pixel values from a second sensor in parallel with the reading out of the row of pixel values from the top portion of the first sensor, wherein reading out said row of pixel values from a second sensor in parallel with the reading out of the row of pixel values from the top portion of the first sensor is performed during a first read out time period, said row of pixel values read out from the first sensor during the first read out time period corresponding to a scene area which overlaps a scene area to which said row of pixel values read out from the second sensor corresponds;

reading out during said first read out time period a row of pixel values from a third sensor capturing a scene area below the scene area to which said row of pixel values read out from the second sensor correspond;

during a second read out time period reading out an additional row of pixel values from the third sensor in parallel with reading out of a row of pixel values from the bottom portion of the first sensor, said row of pixel values from the bottom portion of the first sensor corresponding to a scene area which overlaps the scene area captured by the additional row of pixel values from the third sensor; and

during said second read out time period reading out an additional row of pixel values from the second sensor.

2. A method of operating an apparatus, the method comprising:

alternating between reading out a row of pixel values from a top portion of a first sensor and a row of pixel values from a bottom portion of the first sensor;

reading out a row of pixel values from a second sensor in parallel with the reading out of the row of pixel values from the top portion of the first sensor;

reading out a row of pixel values from a third sensor in parallel with reading out of a row of pixel values from the bottom portion of the first sensor; and

repeating, until an image is captured by said first sensor, said steps of;

i) alternating between reading out a row of pixel values from a top portion of the first sensor and a row of pixel values from a bottom portion of the first sensor, ii) reading out a row of pixel values from a second sensor in parallel with the reading out of the row of pixel values from the top portion of the first sensor; and

iii) reading out a row of pixel values from a third sensor in parallel with reading out of a row of pixel values from a bottom portion of the first sensor.

3. The method of claim 2, wherein reading out of rows of pixel values from said second sensor is performed by reading out rows of pixel values in a direction extending from a bottom row towards a top row of said second sensor.

4. The method of claim 3, wherein said first sensor is part of a first camera module, said second sensor is part of a second camera module, and said third sensor is part of a third camera module, said first camera module having a shorter focal length than said second and third camera modules.

5. A method of operating an apparatus, the method comprising:

alternating between reading out a row of pixel values from a top portion of a first sensor and a row of pixel values from a bottom portion of the first sensor;

reading out a row of pixel values from a second sensor in parallel with the reading out of the row of pixel values from the top portion of the first sensor; and

reading out a row of pixel values from a third sensor in parallel with reading out of a row of pixel values from the bottom portion of the first sensor, wherein reading out of rows of pixel values from said third sensor is performed by reading out rows of pixel values in a direction extending from a top row of said third sensor with row read out of said third sensor progressing towards a bottom row of said third sensor.

6. A method of operating an apparatus, the method comprising:

alternating between reading out a row of pixel values from a top portion of a first sensor and a row of pixel values from a bottom portion of the first sensor;

reading out a row of pixel values from a second sensor in parallel with the reading out of the row of pixel values from the top portion of the first sensor;

reading out a row of pixel values from a third sensor in parallel with reading out of a row of pixel values from the bottom portion of the first sensor; and

reading out rows of pixel values from a fourth sensor in parallel with reading out of a row of pixel values from the top portion of the first sensor.

7. The method of claim 6, wherein reading out of rows of pixel values from said fourth sensor is performed by reading out rows of pixel values in a direction extending from a bottom row towards a top row of said fourth sensor.

8. The method of claim 6, further comprising:

reading out rows of pixel values from a fifth sensor in parallel with reading out of pixel values from the bottom portion of the first sensor; and

wherein reading out of rows of pixel values from said fifth sensor is performed by reading out rows of pixel values in a direction extending from a top row towards a bottom row of said fifth sensor.

9. The method of claim 8,

wherein said second, third, fourth and fifth sensors correspond to second, third, fourth and fifth camera modules;

and

wherein said first sensor corresponds to a first camera module which has a shorter focal length than said second camera module.

10. The method of claim 9, wherein the focal length of said second camera module is at least twice the focal length of the first camera module.

11. The method of claim 9, further comprising:

generating a composite image from images captured by reading out pixel values from said second, third, fourth and fifth sensors; and

wherein generating a composite image includes using an image captured by said first camera module in aligning portions of said images captured by reading out pixel values from the second, third, fourth and fifth sensors as part of generating the composite image.

12. An apparatus, comprising:

a first image sensor;

a second image sensor; and

a controller configured to control reading out of pixel values from said first image sensor and second image sensor in a coordinated manner, the controller being configured to;

control the first image sensor to alternate between reading out a row of pixel values from a top portion of the first image sensor and a row of pixel values from a bottom portion of the first image sensor;

control the second image sensor to read out a row of pixel values from the second image sensor in parallel with the reading out of the row of pixel values from the top portion of the first image sensor;

control reading out said row of pixel values from the second image sensor in parallel with the reading out of the row of pixel values from the top portion of the first image sensor during a first read out time period, said row of pixel values read out from the first image sensor during the first read out time period corresponding to a scene area which overlaps a scene area to which said row of pixel values read out from the second image sensor corresponds;

control a third image sensor to read out during said first read out time period a row of pixel values from the third image sensor capturing a scene area below the scene area to which said row of pixel values read out from the second image sensor corresponds;

control the third image sensor to read out during a second read out time period an additional row of pixel values from the third image sensor in parallel with read out of a row of pixel values from the bottom portion of the first image sensor, said row of pixel values from the bottom portion of the first image sensor corresponding to a scene area which overlaps the scene area captured by the additional row of pixel values from the third image sensor; and

control the second image sensor to read out during said second read out time period an additional row of pixel values from the second image sensor.

13. The apparatus of claim 12, further comprising:

said third image sensor; and

wherein said controller is further configured to control the third image sensor to read out a row of pixel values from the third image sensor in parallel with read out of a row of pixel values from the bottom portion of the first image sensor.

14. An apparatus, comprising:

a first image sensor;

a second image sensor;

a third image sensor; and

a controller configured to control reading out of pixel values from said first image sensor and said second image sensor in a coordinated manner, the controller being configured to;

control the first image sensor to alternate between reading out a row of pixel values from a top portion of the first image sensor and a row of pixel values from a bottom portion of the first image sensor;

control the second image sensor to read out a row of pixel values from the second image sensor in parallel with the reading out of the row of pixel values from the top portion of the first image sensor; and

control the third image sensor to read out a row of pixel values from the third image sensor in parallel with read out of a row of pixel values from the bottom portion of the first image sensor; and

repeat the following operations until an image is captured by said first image sensor;

i) control the first image sensor to alternate between reading out a row of pixel values from a top portion of the first image sensor and a row of pixel values from a bottom portion of the first image sensor, ii) control the second image sensor to read out a row of pixel values from the second image sensor in parallel with the reading out of the row of pixel values from the top portion of the first image sensor, and iii) control the third image sensor to read out a row of pixel values from the third image sensor in parallel with reading out of a row of pixel values from a bottom portion of the first image sensor.

15. The apparatus of claim 14, further comprising:

a fourth image sensor;

wherein said controller is further configured to control the fourth image sensor to read out rows of pixel values from the fourth image sensor in parallel with read out of a row of pixel values from the top portion of the first image sensor;

a fifth image sensor; and

wherein said controller is further configured to control the fifth image sensor to read out rows of pixel values from the fifth image sensor in parallel with read out of pixel values from the bottom portion of the first image sensor.

16. The apparatus of claim 15, further comprising:

a processor configured to generate a composite image from images captured by reading out pixel values from said second, third, fourth and fifth image sensors, said processor being further configured to use an image captured by said first image sensor in aligning portions of said images captured by reading out pixel values from the second, third, fourth and fifth image sensors, as part of being configured to generate a composite image.

17. A non-transitory machine readable medium for use in a camera including a first image sensor and a second image sensor, the non-transitory machine readable medium including processor executable instructions which when executed by a processor control said processor to:

alternate between reading out a row of pixel values from a top portion of the first image sensor and a row of pixel values from a bottom portion of the first image sensor; and

read out a row of pixel values from said second image sensor in parallel with the reading out of the row of pixel values from the top portion of the first image sensor, wherein the read out said row of pixel values from said second image sensor in parallel with the reading out of the row of pixel values from the top portion of the first image sensor is performed during a first read out time period, said row of pixel values read out from the first image sensor during the first read out time period corresponding to a scene area which overlaps a scene area to which said row of pixel values read out from the second image sensor corresponds;

read out during said first read out time period a row of pixel values from a third sensor capturing a scene area below the scene area to which said row of pixel values read out from the second image sensor correspond;

read out, during a second read out time period, an additional row of pixel values from the third sensor in parallel with reading out of a row of pixel values from the bottom portion of the first image sensor, said row of pixel values from the bottom portion of the first image sensor corresponding to a scene area which overlaps the scene area captured by the additional row of pixel values from the third sensor; and

reading out, during said second read out time period, an additional row of pixel values from the second image sensor.