Wide field of view array camera for hemispheric and spherical imaging

• US 9,819,863 B2
• Filed: 06/15/2015
• Issued: 11/14/2017
• Est. Priority Date: 06/20/2014
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Osborne, Thomas Wesley, Georgiev, Todor Georgiev, Goma, Sergiu Radu
• Primary Examiner(s): Chen, Chia-Wei A
• Application Number:
  US14/739,311
• Publication Number:
  US 20150373279A1
• Time in Patent Office: 883 Days
• Field of Search: 3482181, 348373, 348374, 34821111
• CPC Class Codes: G02B 13/0075 having an element with variable optical properties
• CPC Class Codes: G02B 13/06 Panoramic objectives; So-called "sky lenses" including panoramic objectives having reflecting surfaces
• CPC Class Codes: G03B 17/17 with reflectors arranged in beam forming the photographic image, e.g. for reducing dimensions of camera
• CPC Class Codes: G03B 35/08 by simultaneous recording
• CPC Class Codes: G03B 37/04 with cameras or projectors providing touching or overlapping fields of view
• CPC Class Codes: H04N 13/243 using three or more 2D image sensors
• CPC Class Codes: H04N 23/45 for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 23/55 Optical parts specially adapted for electronic image sensors; Mounting thereof
• CPC Class Codes: H04N 23/58 Means for changing the camera field of view without moving the camera body, e.g. nutating or panning of optics or image sensors
• CPC Class Codes: H04N 23/60 Control of cameras or camera modules
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums

Abstract

Aspects relate to methods and systems for producing ultra-wide field of view images. In some embodiments, an image capture system for capturing wide field-of-view images comprises an aperture, a central camera positioned to receive light through the aperture, the center camera having an optical axis, a plurality of periphery cameras disposed beside the central camera and pointed towards a portion of the optical axis of the center camera, the plurality of cameras arranged around the center camera, and a plurality of extendible reflectors. The reflectors are configured to move from a first position to a second position and have a mirrored first surface that faces away from the optical axis of the center camera and a second black surface that faces towards the optical axis of the center camera, the plurality of periphery cameras arranged around the center camera.

Claims

1. An image capture system for capturing wide field-of-view images, comprising:

a first imaging system, comprising;

a first aperture;

a first center camera positioned to receive light through the first aperture, the first center camera having an optical axis that extends through the aperture;

a first plurality of periphery cameras arranged around the first center camera and pointed towards a portion of the optical axis of the first center camera; and

a first plurality of extendible reflectors configured to move from a first position to a second position, the first imaging system having a first FOV with the first plurality of extendible reflectors in the first position and having a second FOV with the first plurality of extendible reflectors in the second position, the second FOV being larger than the first FOV, the second position being closer to the optical axis of the first center camera, the first plurality of periphery cameras and the first plurality of extendible reflectors arranged relative to each other such that that at least a portion of light reflected from one of the first plurality of extendible reflectors enters a corresponding one of the first plurality of periphery cameras when the first plurality of extendible reflectors are in the second position.

2. The system of claim 1, further comprising a processor configured to create a hemispherical image using an image generated from the first center camera and an image from each of the first plurality of periphery cameras that was captured when the first plurality of extendible reflectors are positioned in the second position.

3. The system of claim 1, wherein the first plurality of extendible reflectors each comprise a mirrored first surface that faces away from the optical axis of the first center camera.

4. The system of claim 3, wherein the first plurality of extendible reflectors each comprise a second black surface that faces towards the optical axis of the first center camera.

5. The system of claim 3, wherein the mirrored first surface of the one of the first plurality of extendible reflectors reflects the at least a portion of the light reflected toward the corresponding one of the first plurality of periphery cameras when the first plurality of extendible reflectors are in the second position.

6. The system of claim 3, wherein the mirrored first surface each of the first plurality of extendible reflectors is one of a concave or convex surface.

7. The system of claim 1, wherein the one of the first plurality of extendible reflectors comprises a substantially elliptical shape corresponding to a FOV of the corresponding one of the first plurality of periphery cameras.

8. The system of claim 1, wherein each of the first plurality of periphery cameras and the first center camera comprise a lens assembly and an image sensor.

9. The system of claim 8, wherein the image sensor of each of the first plurality of periphery cameras and the first center camera is disposed on a common substrate, the common substrate forming a plane that is substantially orthogonal to the optical axis of the center camera.

10. The system of claim 9, wherein in the first position the first plurality of extendible reflectors are in a retracted configuration such that a reflective surface of each of the first plurality of reflectors is positioned substantially parallel to the substrate.

11. The system of claim 9, wherein in the second position the first plurality of extendible reflectors are in an extended configuration such that a reflective surface of each of the first plurality of reflectors is positioned at an angle parallel to the substrate, the angle having a predetermined value corresponding to producing a value of 180 degrees for the second FOV.

12. The system of claim 11, wherein at least a portion of rays approaching the optical axis of the first center camera in a direction parallel to the substrate are reflected off of the first plurality of extended reflectors toward the first plurality of periphery cameras when the first plurality of extendible reflectors are in the second position.

13. The system of claim 1, further comprising:

a second imaging system, comprising;

a second aperture;

a second central camera positioned to receive light through the second aperture, the second center camera having an optical axis that extends through the second aperture;

a second plurality of periphery cameras arranged around the second center camera and pointed towards a portion of the optical axis of the second center camera; and

a second plurality of extendible reflectors configured to move from a third position to a fourth position, the second imaging system having the first FOV with the second plurality of extendible reflectors in the third position and having the second FOV with the second plurality of extendible reflectors in the second position, the fourth position being closer to the optical axis of the second center camera than the third position, the second plurality of periphery cameras and the second plurality of extendible reflectors arranged relative to each other such that that at least a portion of light reflected from one of the second plurality of additional extendible reflectors enters a corresponding one of the second plurality of periphery cameras when the second plurality of extendible reflectors are in the fourth position.

14. The system of claim 13, wherein the second FOV of each of the first and second imaging systems is hemispherical-shaped and looks at a substantially different image scene than the hemispherical-shaped FOV of the other of the first and second imaging systems.

15. The system of claim 14, wherein the first and second imaging systems are disposed in a back-to-back arrangement such that a total FOV comprising a combination of the second FOV of each of the first and second imaging systems is spherical-shaped.

16. A method for capturing using a single array camera a standard field-of-view image or a wide field-of-view image, comprising:

controlling the positioning of a plurality of extendible reflectors positioned around a center camera, a plurality of periphery cameras also positioned around the center camera and each associated with one of the plurality of extendible reflectors, the center camera having an optical axis and the plurality of periphery cameras pointed toward a portion of the optical axis;

determining whether to capture the standard field-of-view image or the wide field-of-view image, the wide field-of-view image covering a greater field than the standard field-of-view image;

in response to determining to capture the standard field-of-view image;

positioning the plurality of extendible reflectors in a first position, and

capturing the standard field-of-view image using the center camera; and

in response to determining to capture the wide field-of-view image;

positioning the plurality of extendible reflectors in a second position such that at least a portion of light reflected from one of the plurality of extendible reflectors enters an associated one of the plurality of periphery cameras,

capturing a central portion of the wide field-of-view image using the center camera, and

capturing a plurality of periphery portions of the wide field-of-view image using the plurality of periphery cameras.

17. The method of claim 16, wherein, in response to determining to capture the standard field-of-view image, the method further comprises combining the central portion and the plurality of periphery portions into the wide field-of-view image.

18. The method of claim 16, wherein the central portion of the wide field-of-view image comprises a portion of the standard field-of-view image.

19. The method of claim 16, wherein positioning the plurality of extendible reflectors in the first position comprises retracting the plurality of extendible reflectors against a camera housing containing the center camera and the plurality of periphery cameras.

20. The method of claim 16, wherein positioning the plurality of extendible reflectors in the second position comprises angling the plurality of extendible reflectors relative to a camera housing containing the center camera and the plurality of periphery cameras such that a reflective surface of each of the plurality of extendible reflectors directs light representing the plurality of periphery portions toward the plurality of periphery cameras.

21. A non-transitory computer-readable medium storing instructions that, when executed, cause one or more processors to perform operations comprising, for controlling the positioning of a plurality of extendible reflectors positioned around a center camera and each associated with one of a plurality of periphery cameras also positioned around the center camera:

determining whether to capture a standard field-of-view image or a wide field-of-view image, the wide field-of-view image covering a greater field than the standard field-of-view image;

in response to determining to capture the standard field-of-view image;

generating first instructions to position the plurality of extendible reflectors in a first position, and

receiving first image data comprising the standard field-of-view image from the center camera; and

in response to determining to capture the wide field-of-view image;

generating second instructions to position the plurality of extendible reflectors in a second position such that at least a portion of light reflected from one of the plurality of extendible reflectors enters an associated one of the plurality of periphery cameras, and

receiving second image data comprising a central portion of the wide field-of-view image from the center camera and a plurality of periphery portions of the wide field-of-view image from the plurality of periphery cameras.

22. The non-transitory computer-readable medium of claim 21, wherein, in response to determining to capture the wide field-of-view image, the operations further comprise combining the central portion and the plurality of periphery portions into the wide field-of-view image.

23. The non-transitory computer-readable medium of claim 21, wherein generating the first instructions comprises generating instructions for an actuator associated with at least one extendible reflector of the plurality of extendible reflectors to position the at least one extendible reflector in a retracted configuration substantially against a camera housing containing the center camera and the plurality of periphery cameras.

24. The non-transitory computer-readable medium of claim 21, wherein generating the second instructions comprises generating instructions for an actuator associated with an extendible reflector of the plurality of extendible reflectors to position the extendible reflector in an extended configuration such that a reflective surface of the extendible reflector directs light representing one of the plurality of periphery portions toward an associated camera of the plurality of periphery cameras.

25. An apparatus for capturing a standard field-of-view image or a wide field-of-view image, the apparatus comprising:

housing means having at least an upper surface and a lower surface spaced apart from the upper surface;

image sensing means positioned between the upper surface and the lower surface;

light focusing means positioned below a first aperture in the upper surface of the housing means and above a central sensing area of the image sensing means;

a plurality of additional light focusing means positioned around the first light focusing means and below a corresponding plurality of additional apertures in the upper surface of the housing means and above a plurality of additional sensing areas of the image sensing means; and

a plurality of light reflecting means positioned around the first aperture above the first lens assembly, each of the plurality of light reflecting means associated with one of the plurality of additional light focusing means.

26. The apparatus of claim 25, the first light focusing means having an optical axis, wherein an optical axis of each of the plurality of additional light focusing means is positioned to intersect with the optical axis of the first light focusing means.

27. The apparatus of claim 25, further comprising actuating means for transitioning the plurality of light reflecting means between a first position corresponding to the standard field-of-view image and a second position corresponding to the wide field-of-view image.

28. The apparatus of claim 25, further comprising processing means in communication with the image sensing means.

29. The apparatus of claim 28, further comprising instructions provided to the processing means to combine image data from the central sensing area and the plurality of additional sensing areas into the wide field-of-view image.

30. The apparatus of claim 28, further comprising instructions provided to the processing means to generate instructions that configure actuating means to transition the plurality of extendible reflectors between a first position corresponding to the standard field-of-view image and a second position corresponding to the wide field-of-view image.