Methods and apparatus for capturing images and/or for using captured images

• US 9,563,033 B2
• Filed: 07/09/2014
• Issued: 02/07/2017
• Est. Priority Date: 10/18/2013
• Status: Active Grant

• Current Assignee: Blue River Technology Inc. (Deere & Company)
• Original Assignee: Light Labs Inc.
• Inventors: Laroia, Rajiv
• Primary Examiner(s): Berhan, Ahmed A
• Application Number:
  US14/327,515
• Publication Number:
  US 20150293429A1
• Time in Patent Office: 944 Days
• US Class Current: 1/1
• CPC Class Codes: G02B 13/0065 having a beam-folding prism or mirror
• CPC Class Codes: G02B 13/009 having zoom function
• CPC Class Codes: G02B 17/008 Systems specially adapted to form image relays or chained systems
• CPC Class Codes: G02B 7/04 with mechanism for focusing or varying magnification
• CPC Class Codes: G02B 7/09 adapted for automatic focusing or varying magnification
• CPC Class Codes: G02B 7/102 controlled by a microcomputer cameras with interchangeable lenses G03B17/14
• CPC Class Codes: G02B 7/1805 for prisms G02B7/181 takes precedence
• CPC Class Codes: G02B 7/182 for mirrors
• CPC Class Codes: G03B 13/36 Autofocus systems
• CPC Class Codes: G03B 17/17 with reflectors arranged in beam forming the photographic image, e.g. for reducing dimensions of camera
• CPC Class Codes: G03B 2205/0046 Movement of one or more optical elements for zooming for optical zoom lenses in general G02B7/10
• CPC Class Codes: G03B 37/04 with cameras or projectors providing touching or overlapping fields of view
• CPC Class Codes: G03B 5/00 Adjustment of optical system relative to image or object surface other than for focusing
• CPC Class Codes: H04N 13/243 using three or more 2D image sensors
• CPC Class Codes: H04N 23/45 for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 23/51 Housings
• CPC Class Codes: H04N 23/54 Mounting of pick-up tubes, electronic image sensors, deviation or focusing coils
• CPC Class Codes: H04N 23/55 Optical parts specially adapted for electronic image sensors; Mounting thereof
• CPC Class Codes: H04N 23/57 Mechanical or electrical details of cameras or camera modules specially adapted for being embedded in other devices
• CPC Class Codes: H04N 23/631 Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters
• CPC Class Codes: H04N 23/67 : Focus control based on electronic image sensor signals
• CPC Class Codes: H04N 23/69 : Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
• CPC Class Codes: H04N 23/698 : for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 23/74 : by influencing the scene brightness using illuminating means
• CPC Class Codes: H04N 25/00 : Circuitry of solid-state image sensors [SSIS]; Control thereof
• CPC Class Codes: H04N 25/531 : by controlling rolling shutters in CMOS SSIS
• CPC Class Codes: H04N 25/745 : Circuitry for generating timing or clock signals
• CPC Class Codes: H04N 25/75 : Circuitry for providing, modifying or processing image signals from the pixel array
• CPC Class Codes: H04N 5/2622 : Signal amplitude transition in the zone between image portions, e.g. soft edges
• CPC Class Codes: H04N 5/2624 : for obtaining an image which is composed of whole input images, e.g. splitscreen
• CPC Class Codes: H04N 5/265 : Mixing

Abstract

Methods and apparatus relating to image capture of image portions using optical chains with different focal lengths are described. In some embodiments different portions of a scene area of interest captured by different optical chains operating in parallel are combined. The combining in some embodiments is performed using depth information generated from two images captured by two different optical chains which capture at least some common overlapping portions of the scene area of interest. The depth information maybe generated using an image of the entire scene and one or more other images of the scene or portions of the scene area of interest. The use of multiple optical chains in parallel facilitates generation of an image with a higher overall pixel count than would be possible using a single sensor of one of the optical chains and/or with more light capture than would be captured using a single optical chain.

Claims

1. A method of capturing images, comprising:

capturing a first image of a scene area of interest using a first optical chain of a camera, said first optical chain having a first focal length;

capturing a second image of a second scene area, said second scene area being a first portion of said scene area of interest, using a second optical chain of said camera, said second optical chain having a second focal length which is greater than said first focal length, said first portion being smaller than said scene area of interest;

capturing a third image of a third scene area, said third scene area being a second portion of said scene area of interest, using a third optical chain of said camera, said third optical chain having a third focal length which is greater than said first focal length, said second portion being smaller than said scene area of interest and being a different from said first portion, at least a portion of said third scene area being non-overlapping with the second scene area, said first and second scene areas being included in said scene area of interest;

capturing a fourth image of a fourth scene area using a fourth optical chain of said camera, said fourth scene area being a third portion of said scene area of interest, said fourth optical chain having said second focal length which is greater than said first focal length, said third portion of the scene area of interest being smaller than said scene area of interest and being different from said first and second portions of said scene area of interest;

capturing a fifth image of a fifth scene area using a fifth optical chain of said camera, said fifth scene area being a fourth portion of said scene area of interest, said fifth optical chain having said second focal length which is greater than said first focal length, said fourth portion being smaller than said scene area of interest and being different from said first, second, and third portions of said scene area of interest; and

storing or outputting said first, second, third, fourth and fifth images.

2. The method of claim 1, further comprising:

generating a composite image from at least said first, second, and third captured images.

3. The method of claim 2, further comprising:

generating depth information for a portion of said scene area of interest which is included in at least two of said captured images; and

wherein generating the composite image is performed as a function of said depth information.

4. The method of claim 1, further comprising:

capturing a sixth image, said sixth image being of the scene area of interest, using a sixth optical chain having a fourth focal length, said fourth focal length being equal to or smaller than the first focal length, said sixth image including at least said first and second scene portions and said scene area of interest.

5. A method of capturing images, comprising:

capturing a first image of a scene area of interest using a first optical chain of a camera, said first optical chain having a first focal length;

capturing a second image of a second scene area, said second scene area being a first portion of said scene area of interest, using a second optical chain of said camera, said second optical chain having a second focal length which is greater than said first focal length;

capturing a third image of a third scene area, said third scene area being a second portion of said scene area of interest, using a third optical chain of said camera, said third optical chain having a third focal length which is greater than said first focal length, at least a portion of said third scene area being non-overlapping with the second scene area;

capturing a fourth image, said fourth image being of the scene area of interest using a fourth optical chain having a fourth focal length, said fourth focal length being equal to or smaller than the first focal length; and

capturing a fifth image of a fifth scene area, said fifth scene area being a third portion of said scene area of interest, using a fifth optical chain having a fifth focal length which is greater than said first focal length; and

storing or outputting at least said first, second and third images.

6. The method of claim 5, further comprising:

capturing a sixth image of a sixth scene area, said sixth scene area being a fourth portion of said scene area of interest, using a sixth optical chain having a sixth focal length which is greater than said first focal length.

7. The method of claim 6, wherein said second, third, fifth and sixth focal lengths are the same.

8. The method of claim 7, wherein the first and fourth focal lengths are the same.

9. The method of claim 8, wherein the union of said second, third, fifth and sixth scene areas includes the scene area of interest.

10. The method of claim 9, further comprising:

capturing a seventh image of a seventh scene area, said seventh scene area being at the center of said scene area of interest, using a seventh optical chain having a seventh focal length which is greater than said first focal length.

11. The method of claim 10, wherein said seventh focal length is the same as said second, third, fifth and sixth focal lengths.

12. A camera, comprising:

a first optical chain having a first focal length and being configured in said camera to capture a first image of a scene area of interest;

a second optical chain having a second focal length and being configured to capture a second image of a second scene area, said second focal length being greater than said first focal length, said second scene area being a first portion of said scene area of interest and being smaller than said scene area of interest; and

a third optical chain having a third focal length and being configured to capture a third image of a third scene area, said third focal length being greater than said first focal length, said third scene area being a second portion of said scene area of interest which is smaller than said scene area of interest and different from said first portion, at least a portion of said third scene area being non-overlapping with the second scene area, said first and second scene areas being included in said scene area of interest;

a fourth optical chain of said camera configured to capture a fourth image of a fourth scene area, said fourth scene area being a third portion of said scene area of interest, said fourth optical chain having said second focal length which is greater than said first focal length, said third portion of the scene area of interest being smaller than said scene area of interest and being different from said first and second portions of said scene area of interest; and

a fifth optical chain of said camera, configured to capture a fifth image of a fifth scene area, said fifth scene area being a fourth portion of said scene area of interest, said fifth optical chain having said second focal length which is greater than said first focal length, said fourth portion being smaller than said scene area of interest and being different from said first, second and third portions of said scene area of interest.

13. The camera of claim 12, further comprising:

a processor configured to generate a composite image from at least said first image of said scene area of interest captured by said first optical chain, said second image of said second scene area captured using the second optical chain, and said third image of said third scene area captured using said third optical chain; and

a memory configured to store said first, second and third images.

14. The camera of claim 13,

wherein said processor is further configured to generate depth information for a portion of said scene area of interest which is included in at least two of the captured images;

and

wherein said processor is further configured to generate the composite image as a function of said depth information.

15. The camera of claim 12, further comprising:

a sixth optical chain having a fourth focal length, said fourth focal length being equal to or smaller than the first focal length.

16. The camera of claim 12,

wherein said first portion of said scene area of interest corresponds to an upper left corner portion of the scene area of interest;

wherein said second portion of said scene area of interest corresponds to an upper right corner portion of the scene area of interest;

wherein said third portion of said scene area of interest corresponds to a lower left corner portion of the scene area of interest; and

wherein said third portion of said scene area of interest corresponds to a lower right corner portion of the scene area of interest.

17. A camera, comprising:

a first optical chain having a first focal length, said first optical chain being arranged in the camera to capture a scene area of interest;

a second optical chain having a second focal length which is greater than said first focal length; and

a third optical chain having a third focal length which is greater than said first focal length;

a fourth optical chain having a fourth focal length, said fourth focal length being equal to or smaller than the first focal length;

a fifth optical chain having a fifth focal length which is greater than said first focal length;

a sixth optical chain having a sixth focal length which is greater than said first focal length;

wherein said first, second, third, fourth, fifth and sixth optical chains are separate optical chains; and

wherein said second, third, fifth and sixth optical chains are arranged to capture different portions of the scene area of interest captured by said first optical chain.

18. The camera of claim 17, wherein said second, third, fifth and sixth focal lengths are the same;

and wherein the first and fourth focal lengths are the same.

19. The camera of claim 18,

wherein said fifth optical chain is used to capture a fifth image of a fifth scene area, said fifth scene area being a third portion of said scene area of interest;

and

wherein said sixth optical chain is used to capture a sixth image of a sixth scene area, said sixth scene area being a fourth portion of said scene area of interest; and

wherein the union of said second, third, fifth and sixth scene areas includes the scene area of interest.

20. The camera of claim 19, further comprising:

a seventh optical chain having a seventh focal length which is greater than said first focal length, said seventh optical chain being used to capture a seventh image of a seventh scene area, said seventh scene area being at the center of said scene area of interest.

21. A non-transitory computer readable medium, said non-transitory computer readable medium comprising computer executable instructions, said non-transitory computer readable medium comprising:

instructions, which when executed by a processor, control a first optical chain of a camera to capture a first image of a scene area of interest, said first optical chain having a first focal length;

instructions, which when executed by said processor, control a second optical chain of said camera to capture a second image of a second scene area, said second optical chain having a second focal length which is greater than said first focal length, said second scene area being a first portion of said scene area of interest, said first portion being smaller than said scene area of interest;

instructions, which when executed by said processor, control a third optical chain to capture a third image of a third scene area, said third optical chain having a third focal length, said third focal length being greater than said first focal length, said third scene area being a second portion of said scene area of interest, said second portion being smaller than said scene area of interest and being a different from said first portion, at least a portion of said third scene area being non-overlapping with the second scene area, said first and second scene areas being included in said scene area of interest;

instructions, which when executed by said processor, control a fourth optical chain to capture a fourth image of a fourth scene area, said fourth scene area being a third portion of said scene area of interest, said fourth optical chain having said second focal length which is greater than said first focal length, said third portion of the scene area of interest being smaller than said scene area of interest and being different from said first and second portions of said scene area of interest;

instructions, which when executed by said processor, control a fifth optical chain to capture a fifth image of a fifth scene area, said fifth scene area being a fourth portion of said scene area of interest, said fifth optical chain having said second focal length which is greater than said first focal length, said fourth portion being smaller than said scene area of interest and being different from said first, second, and third portions of said scene area of interest; and

instructions, which when executed by said processor, control said processor to store said first, second, third, fourth and fifth captured images in a memory.

22. The non-transitory computer readable medium of claim 21,

wherein said first portion of said scene area of interest corresponds to an upper left corner portion of the scene area of interest;

wherein said second portion of said scene area of interest corresponds to an upper right corner portion of the scene area of interest;

wherein said third portion of said scene area of interest corresponds to a lower left corner portion of the scene area of interest; and

wherein said fourth portion of said scene area of interest corresponds to a lower right corner portion of the scene area of interest.