Methods and apparatus for capturing images using optical chains and/or for using captured images

• US 9,578,252 B2
• Filed: 07/09/2014
• Issued: 02/21/2017
• Est. Priority Date: 10/18/2013
• Status: Active Grant

• Current Assignee: Blue River Technology Inc. (Deere & Company)
• Original Assignee: Light Labs Inc.
• Inventors: Laroia, Rajiv
• Primary Examiner(s): Daniels, Anthony J
• Application Number:
  US14/327,514
• Publication Number:
  US 20150109482A1
• Time in Patent Office: 958 Days
• Field of Search: 348/335, 348/345
• US Class Current: 1/1
• CPC Class Codes: G02B 13/001 Miniaturised objectives for electronic devices, e.g. portable telephones, webcams, PDAs, small digital cameras G02B23/243 takes precedence
• CPC Class Codes: G02B 13/0055 employing a special optical element
• CPC Class Codes: G02B 13/0065 having a beam-folding prism or mirror
• CPC Class Codes: G02B 13/009 having zoom function
• CPC Class Codes: G02B 13/02 Telephoto objectives, i.e. systems of the type + - in which the distance from the front vertex to the image plane is less than the equivalent focal length
• CPC Class Codes: G02B 15/00 Optical objectives with means for varying the magnification anamorphotic objectives G02B13/08
• CPC Class Codes: G02B 17/008 Systems specially adapted to form image relays or chained systems
• CPC Class Codes: G02B 17/0896 with variable magnification or multiple imaging planes, including multispectral systems systems with only refractive elements G02B15/14
• CPC Class Codes: G02B 27/1066 for enhancing image performance, like resolution, pixel numbers, dual magnifications or dynamic range, by tiling, slicing or overlapping fields of view
• CPC Class Codes: G02B 3/0037 characterized by the distribution or form of lenses
• CPC Class Codes: G02B 5/005 Diaphragms for cameras G03B9/02
• CPC Class Codes: G02B 5/045 Prism arrays
• CPC Class Codes: G02B 5/08 Mirrors vehicle mirrors involving special optical features B60R1/08
• CPC Class Codes: G02B 7/282 Autofocusing of zoom lenses
• CPC Class Codes: H04N 23/45 for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 23/51 Housings
• CPC Class Codes: H04N 23/55 Optical parts specially adapted for electronic image sensors; Mounting thereof
• CPC Class Codes: H04N 23/58 Means for changing the camera field of view without moving the camera body, e.g. nutating or panning of optics or image sensors
• CPC Class Codes: H04N 23/631 Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters
• CPC Class Codes: H04N 23/667 Camera operation mode switching, e.g. between still and video, sport and normal or high- and low-resolution modes
• CPC Class Codes: H04N 23/67 : Focus control based on electronic image sensor signals
• CPC Class Codes: H04N 23/6811 : based on the image signal
• CPC Class Codes: H04N 23/6812 : based on additional sensors, e.g. acceleration sensors
• CPC Class Codes: H04N 23/685 : performed by mechanical compensation
• CPC Class Codes: H04N 23/689 : Motion occurring during a rolling shutter mode
• CPC Class Codes: H04N 23/69 : Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
• CPC Class Codes: H04N 23/73 : by influencing the exposure time
• CPC Class Codes: H04N 23/741 : by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors
• CPC Class Codes: H04N 23/80 : Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 23/951 : by using two or more images to influence resolution, frame rate or aspect ratio
• CPC Class Codes: H04N 25/00 : Circuitry of solid-state image sensors [SSIS]; Control thereof
• CPC Class Codes: H04N 25/42 : by switching between different modes of operation using different resolutions or aspect ratios, e.g. switching between interlaced and non-interlaced mode
• CPC Class Codes: H04N 25/44 : by partially reading an SSIS array
• CPC Class Codes: H04N 25/70 : SSIS architectures; Circuits associated therewith
• CPC Class Codes: H04N 25/75 : Circuitry for providing, modifying or processing image signals from the pixel array
• CPC Class Codes: H04N 5/2628 : Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation
• CPC Class Codes: H04N 5/265 : Mixing
• CPC Class Codes: H04N 5/268 : Signal distribution or switching

Abstract

Methods and apparatus relating to image capture of image portions using optical chains with non-parallel optical axis are described. In some embodiments different portions of a scene area of interest captured by different optical chains operating in parallel are combined. The use of multiple optical chains in parallel facilitates generation of an image with a higher overall pixel count than would be possible using a single sensor of one of the optical chains and/or with more light capture than would be captured using a single one of the optical chains.

Claims

1. A method of capturing images, comprising:

capturing, using a first optical chain of a camera, a first image of a first portion of a scene area of interest, said first optical chain including a first mirror and a first sensor and having a first optical axis portion extending between the first mirror and the first sensor;

capturing, using a second optical chain of said camera, a second image of a second portion of said scene area of interest, said second optical chain having a second optical axis portion which is not parallel to the first optical axis portion of the first optical chain;

capturing, using a third optical chain of said camera, a third image, said third optical chain not including a mirror and having a focal length that is smaller than a first focal length of the first optical chain;

capturing, using a fourth optical chain of said camera a fourth image, said fourth optical chain not including a mirror, said fourth image including at least a portion of said scene area of interest;

capturing, using a fifth optical chain of the camera, a fifth image, said fifth optical chain including a third mirror and a fifth sensor, said fifth image being an image of a third portion of the scene area of interest, said fifth optical chain having a fifth optical axis portion extending between the third mirror and the fifth sensor, said fifth optical axis portion not being parallel to said second optical axis portion; and

capturing, using a sixth optical chain of the camera, a sixth image, said sixth optical chain including a fourth mirror and a sixth sensor, said sixth image being an image of a fourth portion of the scene area of interest, said sixth optical chain having a sixth optical axis portion extending between the fourth mirror and the sixth sensor, said sixth optical axis portion not being parallel to said first or fifth optical axis portions.

2. The method of claim 1, wherein the first and second optical chains have the same focal length;

and

wherein the second optical chain includes a second mirror and a second sensor, said second optical axis portion extending between the second mirror and the second sensor.

3. The method of claim 1,

wherein said first sensor is located in a first top corner portion of said camera;

wherein the second optical chain includes a second sensor that is located in a second bottom corner portion of said camera;

wherein said third optical chain includes a third sensor that is located in a center portion of said camera; and

wherein said third image captured using the third optical chain includes the scene area of interest.

4. The method of claim 3, wherein said third optical chain includes a third optical axis portion which extends straight from said third sensor through an opening in a front face of the camera.

5. The method of claim 4,

wherein the third optical axis portion is perpendicular to the front face of the camera;

wherein the first optical chain captures the first image using the first sensor; and

wherein the third optical chain captures the third image using the third sensor.

6. The method of claim 1,

wherein the fourth optical chain includes a fourth sensor and has a fourth optical axis portion;

wherein said third and fourth optical axis portions are parallel to each other and extend straight through different openings in a front face of the camera; and

wherein said third optical axis portion and fourth optical axis portion are not parallel to the first optical axis portion or the second optical axis portion.

7. The method of claim 1, further comprising:

generating a composite image by combining at least said first, second, fifth and sixth images.

8. The method of claim 7, wherein said composite image is a composite image of said scene area of interest, said composite image having a larger number of pixel values than either of said first, second, third and fourth images individually have.

9. The method of claim 1,

wherein said first, second, third, and fourth portions of said scene area of interest correspond to different corner portions of said scene area of interest;

wherein said first, second, fifth, and sixth images each have a first number of pixel values, and wherein the method further comprises;

generating a composite image from at least said first, second, third, fourth, fifth and sixth images, said composite image have a number of pixel values more than three times the first number of pixel values but less than six times the first number of pixel values.

10. The method of claim 9, wherein said composite image includes a number of pixel values which is less than the number of pixel values resulting from the sum of the number of pixel values combined to generate said composite image.

11. The method of claim 1, further comprising capturing a seventh image, using a seventh optical chain of the camera including a fifth mirror and a seventh sensor, said seventh optical chain having the same focal length as said first optical chain, said seventh optical chain having a seventh optical axis portion extending between the fifth mirror and the seventh sensor, said seventh optical axis portion extending at an angle relative to a bottom of said camera which is different from angles at which said first, second, fifth and sixth optical axis portions extend relative to the bottom of the camera.

12. The method of claim 1 wherein said first mirror is a movable mirror.

13. A camera, comprising:

a first optical chain including a first mirror and a first sensor, said first optical chain having a first optical axis portion extending between the first mirror and the first sensor, said first optical chain having a first focal length;

a second optical chain having a second optical axis portion which is not parallel to the first optical axis portion of the first optical chain; and

a third optical chain which does not include a mirror, said third optical chain having a focal length that is smaller than said first focal length of said first optical chain

a fourth optical chain of said camera, said fourth optical chain not including a mirror;

a fifth optical chain including a third mirror and a fifth sensor, said fifth optical chain having a fifth optical axis portion extending between the third mirror and the fifth sensor, said fifth optical axis portion not being parallel to said second optical axis portion; and

a sixth optical chain of the camera, said sixth optical chain including a fourth mirror and a sixth sensor, said sixth optical chain having a sixth optical axis portion extending between the fourth mirror and the sixth sensor, said sixth optical axis portion not being parallel to said first or fifth optical axis portions.

14. The camera of claim 13, wherein said third optical chain includes a third sensor and has a third optical axis portion which extends straight through an opening of the third optical chain in a face of said camera to the third sensor, said opening of the third optical chain being different from a first opening of the first optical chain through which light passes into the first optical chain.

15. The camera of claim 14,

wherein the first sensor of the first optical chain is located in a first corner portion of said camera;

and

wherein the third sensor of the third optical chain is located in a center portion of said camera.

16. The camera of claim 14, wherein the fourth optical chain which does not include a mirror includes a fourth sensor and has a fourth optical axis portion which extends straight through an opening of the fourth optical chain in the face of said camera;

and

wherein said third optical axis portion and said fourth optical axis portion are not parallel to the first or second optical axis portions.

17. The camera of claim 16, further comprising:

a processor configured to generate a composite image by combining at least a first image captured by said first optical chain, a second image captured by said second optical chain, a third image captured by said third optical chain and a fourth image captured by said fourth optical chain, said first image being an image of a first corner portion of a scene area of interest, said second image being an image of a second corner portion of said scene area of interest, said third image being an image of said scene area of interest and said fourth image including a second image of said scene area of interest, said first and second corner portions of the scene area of interest being different corner portions.

18. The camera of claim 17,

wherein said processor is further configured to generate the composite image from at least said first image, said second image, said third image, said fourth image, a fifth image captured by said fifth optical chain, and a sixth image captured by said sixth optical chain, said fifth image being an image of a third corner portion of the scene area of interest and said sixth image being an image of a fourth corner portion of the scene area of interest, said first, second, third and fourth corner portions of the scene area of interest being different corner portions.

19. The camera of claim 13, wherein said first mirror is a movable mirror.

20. The camera of claim 13, wherein said camera includes a set of at least two optical chains without mirrors in a center region of said camera, said third optical chain being in said set of at least two optical chains.

21. The camera of claim 20, wherein said set of at least two optical chains without mirrors includes at least four optical chains.

22. A non-transitory computer readable medium, said non-transitory computer readable medium comprising computer executable instructions, said non-transitory computer readable medium comprising:

instructions, which when executed by a processor, control a first optical chain of a camera to capture a first image of a first portion of a scene area of interest, said first optical chain including a first mirror and a first sensor and having a first optical axis portion extending between the first mirror and the first sensor;

instructions, which when executed by said processor, control a second optical chain of said camera to capture a second image of a second portion of said scene area of interest, said second optical chain including a second mirror and a second sensor and having a second optical axis portion extending between the second mirror and the second sensor, said second optical axis portion not being parallel to the first optical axis portion;

instructions, which when executed by said processor, control a third optical chain of said camera which does not include a mirror to capture a third image, said third optical chain having a focal length that is smaller than a first focal length of said first optical chain, said third image including said scene area of interest;

instructions which when executed by said processor, control a fourth optical chain of said camera to capture a fourth image;

instructions which when executed by said processor, control a fifth optical chain of said camera to capture a fifth image, the fifth optical chain including a third mirror and a fifth sensor, said fifth image being an image of a third portion of the scene area of interest, said fifth optical chain having a fifth optical axis portion extending between the third mirror and the fifth sensor, said fifth optical axis portion not being parallel to said second optical axis portion; and

instructions which when executed by said processor, control a sixth optical chain of the camera to capture a sixth image, said sixth optical chain including a fourth mirror and a sixth sensor, said sixth image being an image of a fourth portion of the scene area of interest, said sixth optical chain having a sixth optical axis portion extending between the fourth mirror and the sixth sensor, said sixth optical axis portion not being parallel to said first or fifth optical axis portions.

23. A camera, comprising:

a first optical chain including a first mirror and a first sensor, said first optical chain having a first optical axis portion extending between the first mirror and the first sensor, said first optical chain having a first focal length;

a second optical chain having a second optical axis portion which is not parallel to the first optical axis portion of the first optical chain; and

a third optical chain which does not include a mirror, said third optical chain having a focal length that is smaller than said first focal length of said first optical chain;

wherein said camera includes a set of at least two optical chains without mirrors in a center region of said camera, said third optical chain being in said set of at least two optical chains; and

wherein said set of at least two optical chains without mirrors is surrounded by optical chains which include mirrors, said first and second optical chains being in said optical chains which include mirrors which surround said set of at least two optical chains without mirrors.