Parallax free multi-camera system capable of capturing full spherical images

• US 9,843,723 B2
• Filed: 06/18/2015
• Issued: 12/12/2017
• Est. Priority Date: 06/20/2014
• Status: Expired due to Fees

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Osborne, Thomas Wesley
• Primary Examiner(s): Geroleo, Francis G
• Application Number:
  US14/743,663
• Publication Number:
  US 20150373268A1
• Time in Patent Office: 908 Days
• Field of Search: 348 38
• CPC Class Codes: G02B 13/0075 having an element with variable optical properties
• CPC Class Codes: G02B 13/06 Panoramic objectives; So-called "sky lenses" including panoramic objectives having reflecting surfaces
• CPC Class Codes: G03B 17/17 with reflectors arranged in beam forming the photographic image, e.g. for reducing dimensions of camera
• CPC Class Codes: G03B 35/08 by simultaneous recording
• CPC Class Codes: G03B 37/04 with cameras or projectors providing touching or overlapping fields of view
• CPC Class Codes: H04N 13/243 using three or more 2D image sensors
• CPC Class Codes: H04N 23/45 for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 23/55 Optical parts specially adapted for electronic image sensors; Mounting thereof
• CPC Class Codes: H04N 23/58 Means for changing the camera field of view without moving the camera body, e.g. nutating or panning of optics or image sensors
• CPC Class Codes: H04N 23/60 Control of cameras or camera modules
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums

Abstract

Methods and systems for producing spherical field-of-view images. In some examples, an imaging system includes a front camera having a first field-of-view (FOV) in a first direction and an optical axis that extends through the first FOV, a back camera having an optical axis that extends through the first FOV, a plurality of side cameras disposed between the front camera and the back camera, a back light re-directing reflective mirror component disposed between the back camera and plurality of side cameras, the back light re-directing reflective mirror component further disposed perpendicular to the optical axis of the back camera, and a plurality of side light re-directing reflective mirror components, each of the plurality of side cameras positioned to receive light re-directed reflected from one of the plurality of light redirecting reflective mirror components.

Claims

1. An imaging system for capturing a plurality of images which together depict spherical image, comprising:

a front camera positioned to capture an image in a first field-of-view (FOV) around a projected optical axis of the front camera, the projected optical axis of the front camera being in a first direction;

a back camera positioned to receive light re-directed by a back re-directing reflective mirror component disposed between the front camera and the back camera, the back camera positioned to capture an image in a second FOV around a projected optical axis of the back camera, the projected optical axis of the back camera being in the first direction;

a set of first cameras disposed between the front camera and the back camera in a polygon-shaped arrangement, the first cameras collectively configured to capture images in a third FOV, the third FOV circular-shaped around the projected optical axis of the front camera and projecting outward away from the projected optical axis of the front camera, and at least a portion of the third FOV being between the first FOV and the second FOV;

a set of second cameras disposed between the set of first cameras and the back camera and in a polygon-shaped arrangement, the set of second cameras collectively configured to capture images in a fourth FOV, the fourth FOV circular-shaped around the projected optical axis of the front camera and projecting outward away from the projected optical axis of the front camera, and at least a portion of the fourth FOV being between the third FOV and the second FOV; and

a set of third cameras disposed between the set of second cameras and the back camera and in a polygon-shaped arrangement, and collectively configured to capture images in a fifth FOV, the fifth FOV circular-shaped around the projected optical axis of the front camera and projecting outward away from the projected optical axis of the front camera, and at least a portion of the fifth FOV being between the fourth FOV and the second FOV,

the front camera, back camera, the set of first cameras, the set of second cameras and the set of third cameras configured such that images captured in the first, second, third, fourth and fifth FOV collectively represent a spherical image as seen from a perspective of the imaging system.

2. The imaging system of claim 1, further comprising:

a memory component;

a processor coupled to the memory component and to each of the cameras, the processor and the memory component collectively configured to save an image from each one of the cameras, and further configured to generate the spherical image that includes at least a portion of the images from each of the cameras.

3. The imaging system of claim 1, wherein the back re-directing reflective mirror component is a reflector disposed perpendicular to the projected optical axis of the front camera.

4. The imaging system of claim 1, further comprising a plurality of light re-directing reflective mirror components, each camera in the set of first cameras, the set of second cameras and the set of third cameras positioned to receive light re-directed reflected from one of the plurality of light redirecting reflective mirror components.

5. The imaging system of claim 4, wherein the plurality of light re-directing reflective mirror components comprises a plurality of reflectors.

6. The imaging system of claim 5, wherein the plurality of light re-directing reflective mirror components include a first set of light re-directing reflective mirror components arranged around a longitudinal axis of the imaging system that includes the first optical axis and arranged in a first plane, a second set of light re-directing reflective mirror components arranged around the longitudinal axis of the imaging system that includes the first optical axis and arranged in a second plane, and a third set of light re-directing reflective mirror components arranged around a longitudinal axis of the imaging system that includes the first optical axis and arranged in a third plane, the first second and third planes being disposed between the front camera and the back camera.

7. The imaging system of claim 1, wherein the set of first cameras are disposed at an offset from the set of second cameras and the set of third cameras, the set of second cameras are disposed at an offset from the set of first cameras and the set of third cameras, and the set of third cameras are disposed at an offset from the set of first cameras and the set of second cameras.

8. The imaging system of claim 1, wherein the set of first cameras are disposed in a first plane, the set of second cameras are disposed in a second plane, and the set of third cameras are disposed in a third plane.

9. The imaging system of claim 8, wherein the first, second and third planes are parallel.

10. The imaging system of claim 1, wherein the set of first cameras comprises eight cameras, the set of second cameras comprise eight cameras, and the set of third cameras comprises eight cameras.

11. The imaging system of claim 1, wherein the set of first cameras are disposed equally spaced apart, the set of second cameras are disposed equally spaced apart, and the set of third cameras are disposed equally spaced apart.

12. An imaging system, comprising:

a front camera having a first field-of-view (FOV) in a first direction and an optical axis that extends through the first FOV in a first direction;

a back camera having an optical axis, the back camera positioned such that the optical axis of the back camera is aligned in the first direction to extend through the first FOV and positioned along a line congruent with the optical axis of the front camera;

a plurality of side cameras disposed between the front camera and the back camera, the plurality of side cameras having a combined FOV that extends around the optical axis of the front camera and away from the optical axis of the front camera;

a back light re-directing reflective mirror component disposed between the back camera and plurality of side cameras, the back camera and the back light re-directing reflective mirror component positioned such that the optical axis of the back camera is pointed at the back light re-directing reflective mirror component such that the back camera receives light re-directed reflected by the back light re-directing reflective mirror component along the optical axis of the back camera; and

a plurality of side light re-directing reflective mirror components, each of the plurality of side cameras positioned to receive light re-directed from one of the plurality of light re-directing reflective mirror components.

13. The imaging system of claim 12, wherein the plurality of side cameras are offset from the optical axis of the front camera.

14. The imaging system of claim 12, wherein the plurality of side cameras are arranged to form three concentric rings.

15. The imaging system of claim 12, wherein the plurality of side cameras comprise a first set of array cameras, a second set of array cameras, and a third set of array cameras, each of the first, second and third set of array cameras collectively having a FOV that includes at least a portion of a target scene.

16. The imaging system of claim 15, wherein each array camera includes an image sensor, and wherein the image sensors of the first set of array cameras are disposed on a first substrate, the image sensors of the second set of array cameras are disposed on a second substrate, and the third set of array cameras are disposed on a third substrate.

17. The imaging system of claim 16, wherein the first, second and third substrates are disposed in planes that are parallel.

18. A method of generating an image depicting a spherical field-of-view (FOV), the method comprising:

generating a front image in a front camera positioned to capture an image in a first field-of-view (FOV) around a projected optical axis of the front camera, the projected optical axis of the front camera being in a first direction;

generating a back image in a back camera positioned to receive light re-directed by a back re-directing reflective mirror component disposed between the front camera and the back camera, the back camera positioned to capture an image in a second FOV;

generating first images in a set of first cameras disposed between the front camera and the back camera in a polygon-shaped arrangement, the set of first cameras collectively configured to capture images in a third FOV, the third FOV circular-shaped around the projected optical axis of the front camera and projecting outward away from the projected optical axis of the front camera and at least a portion of the third FOV being between the first FOV and the second FOV;

generating second images in a set of second cameras disposed between the set of first cameras and the back camera and in a polygon-shaped arrangement, the set of second cameras collectively configured to capture images in a fourth FOV, the fourth FOV circular-shaped around the projected optical axis of the front camera and projecting outward away from the projected optical axis of the front camera and at least a portion of the fourth FOV being between the third FOV and the second FOV;

generating third images in a set of third cameras disposed between the set of second cameras and the back camera and in a polygon-shaped arrangement, and collectively configured to capture images in a fifth FOV, the fifth FOV circular-shaped around the projected optical axis of the front camera projecting outward away from the projected optical axis of the front camera and at least a portion of the fifth FOV being between the fourth FOV and the second FOV; and

receiving the front image, back image, first images, second images, and third images in at least one processor, and generating a mosaicked image with the at least one processor that includes at least a portion of the front image, back image, first images, second images, and third images.

19. The method of claim 18, wherein the mosaicked image is generated to depict a spherical image as seen from a perspective viewpoint within the spherical image.

20. The method of claim 18, wherein generating the mosaicked image comprises stitching together at least a portion of the front image, back image, first images, second images, and third images.

21. The method of claim 18, wherein the first, second, third, fourth, and fifth FOV at least partially overlap.

22. The method of claim 18, wherein the set of first cameras, the set of second cameras, and the set of third cameras are arranged to form three concentric rings.

23. The method of claim 18, wherein the back camera is a folded optic multi-sensor assembly.

24. An imaging device, comprising:

means for generating a front image in a front camera positioned to capture an image in a first field-of-view (FOV) around a projected optical axis of the front camera, the projected optical axis of the front camera being in a first direction;

means for generating a back image in a back camera positioned to receive light re-directed reflected by a back re-directing reflective mirror component disposed between the front camera and the back camera, the back camera positioned to capture an image in a second FOV;

means for generating first images in a set of first cameras disposed between the front camera and the back camera in a polygon-shaped arrangement, the set of first cameras collectively configured to capture images in a third FOV, the third FOV circular-shaped around the projected optical axis of the front camera and projecting outward away from the projected optical axis of the front camera first cameras and at least a portion of the third FOV being between the first FOV and the second FOV;

means for generating second images in a set of second cameras disposed between the first cameras and the back camera and in a polygon-shaped arrangement, the set of second cameras collectively configured to capture images in a fourth FOV, the fourth FOV circular-shaped around the projected optical axis of the front camera and projecting outward away from the projected optical axis of the front camera and at least a portion of the fourth FOV being between the third FOV and the second FOV;

means for generating third images in a set of third cameras disposed between the set of second cameras and the back camera and in a polygon-shaped arrangement, and collectively configured to capture images in a fifth FOV, the fifth FOV circular-shaped around the projected optical axis of the front camera projecting outward away from the projected optical axis of the front camera and at least a portion of the fifth FOV being between the fourth FOV and the second FOV; and

means for receiving the front image, back image, first images, second images, and third images and generating a mosaicked image that includes at least a portion of the front image, back image, first images, second images, and third images.

25. The imaging device of claim 24, wherein the mosaicked image is generated to depict a spherical image as seen from a perspective viewpoint within the spherical image.

26. The method of claim 24, wherein generating the mosaicked image comprises stitching together at least a portion of the front image, back image, first images, second images, and third images.

27. The method of claim 24, wherein the first, second, third, fourth, and fifth FOV at least partially overlap.

28. The method of claim 24, wherein the set of first cameras, the set of second cameras, and the set of third cameras are arranged to form three concentric rings.

29. The method of claim 24, wherein the back camera is a folded optic multi-sensor assembly.