Multi-camera system using folded optics

• US 10,165,183 B2
• Filed: 11/03/2017
• Issued: 12/25/2018
• Est. Priority Date: 10/19/2012
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Georgiev, Todor Georgiev, Osborne, Thomas Wesley, Goma, Sergiu Radu
• Primary Examiner(s): Jerabek, Kelly L
• Application Number:
  US15/802,994
• Publication Number:
  US 20180084193A1
• Time in Patent Office: 417 Days
• Field of Search: 348 36, 348 38- 39, 348143, 3482181, 348239, 348262
• CPC Class Codes: G02B 13/0065 having a beam-folding prism or mirror
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: H04N 23/57 Mechanical or electrical details of cameras or camera modules specially adapted for being embedded in other devices
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 25/41 Extracting pixel data from a plurality of image sensors simultaneously picking up an image, e.g. for increasing the field of view by combining the outputs of a plurality of sensors
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Abstract

Described herein are methods and devices that employ a plurality of image sensors to capture a target image of a scene. As described, positioning at least one reflective or refractive surface near the plurality of image sensors enables the sensors to capture together an image of wider field of view and longer focal length than any sensor could capture individually by using the reflective or refractive surface to guide a portion of the image scene to each sensor. The different portions of the scene captured by the sensors may overlap, and may be aligned and cropped to generate the target image.

Claims

1. An optical system comprising:

a first camera arranged to have a first field of view, the first camera comprising;

a light redirector configured to redirect light;

a first lens assembly comprising a first lens and a first actuator for moving the first lens, the first lens assembly being disposed optically downstream of the light redirector;

a first image sensor configured to receive light representing the first field of view, the first sensor being disposed optically downstream of both the light redirector and the first lens assembly, the first image sensor comprising a first photosensitive face defining a first reference plane;

a second camera arranged to have a second field of view that at least partially overlaps with the first field of view, the second camera comprising;

a second lens assembly comprising a second lens and a second actuator for moving the second lens;

a second image sensor configured to receive light representing the second field of view, the second sensor being disposed optically downstream of the second lens assembly, the second image sensor comprising a second photosensitive face defining a second reference plane, the first reference plane being different from the second reference plane.

2. The optical system of claim 1, wherein the first reference plane is non-parallel with the second reference plane such that the first reference plane and the second reference plane intersect at a non-zero angle.

3. The optical system of claim 2, wherein the first photosensitive face is configured to receive at least one light ray propagating normal to the first reference plane and the second photosensitive face is configured to receive at least one light ray propagating normal to the second reference plane.

4. The optical system of claim 3, wherein the first lens assembly is disposed between the light redirector and the first image sensor, the first lens assembly defining a first optical axis intersecting the first image sensor.

5. The optical system of claim 3, wherein the first lens assembly defines a first optical axis and the light redirector is configured to accept light incoming at a non-zero angle with respect to the first optical axis and to output light collinear with the first optical axis.

6. The optical system of claim 5, wherein the light redirector comprises a prism configured to re-angle light through refraction.

7. The optical system of claim 6, wherein the first actuator is configured to move the first lens in response to receiving a first command and the second is configured to move the second lens in response to receiving a second command.

8. The optical system of claim 7, comprising one or more processors configured to:

issue the first command to the first actuator based on light measured with the first image sensor;

issue the second command to the second actuator based on light measured with the second image sensor.

9. The optical system of claim 8, wherein the one or more processors are configured to:

prepare a first image of a scene based on light measured with the first image sensor and a second image of the scene based on light measured with the second image sensor, the first image having the first field of view, the second image having the second field of view;

align at least one of the first and second images with respect to the other of the first and second images.

10. The optical system of claim 1, wherein the first lens has a different focal length than the second lens.

11. The optical system of claim 1, wherein the light redirector comprises at least one of a mirror and a prism.

12. The optical system of claim 1, comprising one or more processors configured to:

issue a first command to the first actuator based on light measured with the first image sensor;

issue a second command to the second actuator based on light measured with the second image sensor;

prepare a first image of a scene based on light measured with the first image sensor and a second image of the scene based on light measured with the second image sensor, the first image having the first field of view, the second image having the second field of view;

align at least one of the first and second images with respect to the other of the first and second images;

wherein the first actuator is configured to move the first lens in response to receiving the first command and the second actuator is configured to move the second lens in response to receiving the second command.

13. The optical system of claim 12, wherein the one or more processors are configured to:

composite the aligned first image with the aligned second image;

save the composite image in memory.

14. The optical system of claim 13, wherein the composite image has a composite field of view, the composite field of view comprising at least some of the overlap between the first field of view and the second field of view.

15. The optical system of claim 12, wherein the one or more processors are configured to:

prepare a third image having a third field of view consisting of overlap between the first field of view and the second field of view.

16. A method of imaging comprising:

with a first camera of an optical system, the first camera arranged to have a first field of view, and the first camera comprising;

(a) a light redirector, (b) a first lens assembly comprising a first lens and a first actuator for moving the first lens, and (c) a first image sensor comprising a first photosensitive face defining a first reference plane;

redirecting first light representing the first field of view with the light redirector, flowing the redirected first light through the first lens, and receiving the flowed first light with the first image sensor;

with a second camera of the optical system, the second camera arranged to have a second field of view that at least partially overlaps with the first field of view, and the second camera comprising;

(a) a second lens assembly comprising a second lens and a second actuator for moving the second lens and (b) a second image sensor comprising a second photosensitive face defining a second reference plane, the first reference plane being different than the second reference plane;

flowing second light representing the second field of view though the second lens, and receiving the flowed second light with the second image sensor.

17. The method of claim 16, wherein the first reference plane is non-parallel with the second reference plane such that the first reference plane and the second reference plane intersect at a non-zero angle.

18. The method of claim 17, wherein the first photosensitive face is configured to receive at least one light ray propagating normal to the first reference plane and the second photosensitive face is configured to receive at least one light ray propagating normal to the second reference plane.

19. The method of claim 18, wherein the first lens assembly is disposed between the light redirector and the first image sensor, the first lens assembly defining a first optical axis intersecting the first image sensor.

20. The method of claim 18, wherein the first lens assembly defines a first optical axis and the light redirector is configured to accept first light incoming at a non-zero angle with respect to the first optical axis and to output first light collinear with the first optical axis.

21. The method of claim 20, wherein the light redirector comprises a prism configured to re-angle light through refraction.

22. The method of claim 21, wherein the first actuator is configured to move the first lens in response to receiving a first command and the second is configured to move the second lens in response to receiving a second command.

23. The method of claim 22, comprising:

measuring received first light with the first image sensor;

measuring received second light with the second image sensor;

issuing the first command to the first actuator based on measured first light;

issuing the second command to the second actuator based on measured second light.

24. The method of claim 23, comprising:

after the first actuator has moved in response to the first command, preparing a first image of a scene based on measured first light such that the first image has the first field of view;

after the second actuator has moved in response to the second command, preparing a second image of the scene based on measured second light such that the second image has the second field of view;

aligning at least one of the first and second images with respect to the other of the first and second images.

25. The method of claim 16, wherein the first lens has a different focal length than the second lens.

26. The method of claim 16, wherein the light redirector comprises at least one of a mirror and a prism.

27. The method of claim 16, comprising:

issuing a first command to the first actuator based first light measured with the first image sensor, the first actuator moving in response to the first command;

issuing a second command to the second actuator based on second light measured with the second image sensor, the second actuator moving in response to the second command;

preparing a first image of a scene based on first light measured with the first image sensor, the first image having the first field of view;

preparing a second image of the scene based on second light measured with the second image sensor, the second image having the second field of view;

aligning at least one of the first and second images with respect to the other of the first and second images.

28. The method of claim 27, comprising:

compositing the aligned first image with the aligned second image;

saving the composite image in memory.

29. The optical system of claim 28, wherein the composite image has a composite field of view, the composite field of view comprising at least some of the overlap between the first field of view and the second field of view.

30. The optical system of claim 27, comprising:

preparing a third image having a third field of view consisting of overlap between the first field of view and the second field of view.