Methods and apparatus for synchronized image capture in a device including optical chains with different orientations

• US 9,948,832 B2
• Filed: 06/22/2016
• Issued: 04/17/2018
• Est. Priority Date: 06/22/2016
• Status: Active Grant

• Current Assignee: Blue River Technology Inc. (Deere & Company)
• Original Assignee: Light Labs Inc.
• Inventors: Laroia, Rajiv
• Primary Examiner(s): DIEP, TRUNG T
• Application Number:
  US15/190,156
• Publication Number:
  US 20170374240A1
• Time in Patent Office: 664 Days
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: H04N 23/45 for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 23/57 Mechanical or electrical details of cameras or camera modules specially adapted for being embedded in other devices
• CPC Class Codes: H04N 23/60 Control of cameras or camera modules
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 23/951 by using two or more images to influence resolution, frame rate or aspect ratio
• CPC Class Codes: H04N 5/04 Synchronising for television systems using pulse code modulation H04N7/56

Abstract

Camera device including multiple optical chains, e.g., camera modules, some of which have different orientations relative to the base of the camera device and methods of using such a camera device are described. The orientation of the camera modules within the camera device is taken into consideration when determining the direction in which lines of the sensors of the individual camera modules are read out. Information with regard to the read out direction to be used with particular camera modules is stored in some embodiments and used to control the readout of sensors, e.g., sensors which use a rolling shutter to capture images of a scene area. The sensors may have the same field of view or overlapping fields of view.

Claims

1. A method of operating a camera device including a plurality of optical chains, the method comprising:

controlling a first sensor of a first optical chain to capture lines of a first image of a scene area in a first order, said first optical chain having a first orientation in the camera device and including a first mirror, said first order being an order proceeding from a top of the first sensor to the bottom of the first sensor causing capture of the scene area by the first sensor in a first scene area capture direction;

controlling a second sensor of a second optical chain to capture lines of a second image of the scene area in a second order, said second optical chain having a second orientation in the camera device which is different from said first orientation and including a second mirror, said second sensor having a second light capture area which is oriented in a different direction in said camera device than a first light capture area of the first sensor, said second order being an order proceeding from a bottom of the second sensor to the top of the second sensor causing capture of the scene area by the second sensor in the first scene area capture direction; and

storing the first and second images in memory.

2. The method of claim 1,

wherein said first scene area capture direction is a top of scene area to bottom of scene area direction;

and

wherein said first optical chain is oriented in said camera device with the first light capture area of the first sensor facing in the direction of the second light capture area of the second sensor.

8. The method of claim 1, wherein the first sensor is mounted at a first angle relative to a base of said first optical chain;

wherein the second sensor is mounted at the first angle relative to a base of the second optical chain; and

wherein said first optical chain is mounted at a first angle relative to a bottom of the camera device and wherein said second optical chain is mounted at a second angle relative to the bottom of the camera device, the second angle relative to the bottom of the camera device being 180 degrees or approximately 180 degrees from the first angle relative to the bottom of the camera device.

9. The method of claim 8, wherein said first and second angles relative to the bottom of the camera device are angles that result in said first and second optical chains extending in directions which are not perpendicular to the bottom of said camera device.

10. The method of claim 8, wherein the camera device further includes:

i) a third optical chain which does not include a mirror and which includes a third sensor and ii) a fourth optical chain which does not include a mirror and which includes a fourth sensor; and

wherein the method further comprises;

controlling sensors of optical chains in said camera device without mirrors to read out lines in said first order.

21. The method of claim 2,

wherein the first and second optical chains have a configuration which is the same with respect to mirror position relative to sensor position within the optical chain;

and

wherein the first optical chain is oriented in said camera 180 degrees relative to said second optical chain.

3. A method of operating a camera device including a plurality of optical chains, the method comprising:

controlling a first sensor of a first optical chain having a first orientation in the camera device to read out lines of a first image of a scene area captured by said first sensor in a first order, said first order being an order proceeding from a top of the first sensor to the bottom of the first sensor;

controlling a second sensor of a second optical chain having a second orientation in the camera device to read out lines of a second image of the scene area captured by said second sensor in a second order, said second order being an order proceeding from a bottom of the second sensor to the top of the second sensor; and

storing the first and second images in memory;

wherein said first optical chain is oriented in said camera device 180 degrees relative to said second optical chain; and

wherein said first and second optical chains each include a mirror.

4. The method of claim 3, wherein said first and second optical chains are of the same type but differ in orientation in said camera device.

5. The method of claim 3, wherein said first and second optical chains have the same focal length and the first and second sensors are of the same size and have same number of lines of pixel elements for sensing pixel values.

6. The method of claim 5, further comprising:

storing in memory information indicating a sensor readout order to be used with individual optical chains in said camera device; and

controlling the order in which rows of pixels are read out of said first and second sensors in accordance with the stored information indicating the sensor readout order to be used.

7. The method of claim 6, wherein said first and second sensors are rolling shutter sensors, said first and second sensors reading out different rows of pixel values at different times in an order which matches the order in which each of the first and second sensor is controlled to read out rows of pixel values.

11. A camera device including a plurality of optical chains, comprising:

a first optical chain having a first orientation in the camera device, said first optical chain including a first sensor and a first mirror that directs light onto a light capture area of the first sensor;

a second optical chain having a second orientation in the camera device, said second optical chain including a second sensor and a second mirror that directs light on a light capture area of the second sensor, the first light capture area of the first sensor being oriented in a different direction within the camera device than the second light capture area of the second sensor;

a controller controlling reading out of pixel values from the first sensor of said first optical chain and the second sensor of said second optical chain, the controller being configured to;

control said first sensor to read out lines of a first image of a scene area captured by said first sensor in a first order, said first order controlling the first sensor to capture the scene area in a first scene area capture direction, said first order being an order proceeding from a top of the first sensor to the bottom of the first sensor; and

control said second sensor to read out lines of a second image of the scene area captured by said second sensor in a second order, said second order being an order proceeding from a bottom of the second sensor to the top of the second sensor, said second order controlling the second sensor to capture the scene area in the first scene area capture direction; and

memory including the first and second images.

12. The camera device of claim 11, wherein said first optical chain is oriented in said camera device 180 degrees relative to said second optical chain.

13. The camera device of claim 12, wherein said first and second optical chains each include a mirror.

14. The camera device of claim 13, wherein said first and second optical chains are of the same type but differ in orientation in said camera device.

15. The camera device of claim 13, wherein said first and second optical chains have the same focal length and the first and second sensors are of the same size and have same number of lines of pixel elements for sensing pixel values.

16. The camera device of claim 15, further comprising:

stored information indicating a sensor readout order to be used with individual optical chains in said camera device; and

wherein said controller is further configured to control the order in which rows of pixels are read out of said first and second sensors in accordance with the stored information indicating the sensor readout order to be used.

17. The camera device of claim 11, wherein the first sensor is mounted at a first angle relative to a base of said first optical chain;

wherein the second sensor is mounted at the first angle relative to a base of the second optical chain; and

wherein said first optical chain is mounted at a first angle relative to a bottom of the camera device and wherein said second optical chain is mounted at a second angle relative to the bottom of the camera device, the second angle relative to the bottom of the camera device being 180 degrees or approximately 180 degrees from the first angle relative to the bottom of the camera device.

18. The camera device of claim 17, wherein said first and second angles relative to the bottom of the camera device are angles that result in said first and second optical chains extending in directions which are not perpendicular to the bottom of said camera device.

19. The camera device of claim 17, further comprising:

i) a third optical chain which does not include a mirror and which includes a third sensor and ii) a fourth optical chain which does not include a mirror and which includes a fourth sensor; and

wherein the controller is further configured to control sensors of optical chains in said camera device without mirrors to read out lines in said first order.

20. A non-transitory machine readable medium for use in a camera device including a plurality of optical chains, the non-transitory machine readable medium including processor executable instructions which when executed by a processor cause said processor to:

control a first sensor of a first optical chain to capture lines of a first image of a scene area in a first order, said first optical chain having a first orientation in the camera device and including a first mirror, said first order being an order proceeding from a top of the first sensor to the bottom of the first sensor causing capture of the scene area by the first sensor in a first scene area capture direction;

control a second sensor of a second optical chain to capture lines of a second image of the scene area in a second order, said second optical chain having a second orientation in the camera device which is different from said first orientation and including a second mirror, said second sensor having a second light capture area which is oriented in a different direction in said camera device than a first light capture area of the first sensor, said second order being an order proceeding from a bottom of the second sensor to the top of the second sensor causing capture of the scene area by the second sensor in the first scene area capture direction; and

store the first and second images in memory.