DUAL APERTURE ZOOM DIGITAL CAMERA

• US 20180359423A1
• Filed: 07/28/2018
• Published: 12/13/2018
• Est. Priority Date: 06/13/2013
• Status: Active Grant

• Current Assignee: Corephotonics Ltd. (Samsung Electronics Co. Ltd.)
• Original Assignee: Corephotonics Ltd. (Samsung Electronics Co. Ltd.)
• Inventors: Shabtay, Gal, Goldenberg, Ephraim, Gigushinski, Oded, Cohen, Noy
• Granted Patent: US 10,225,479 B2
• Time in Patent Office: Days
• CPC Class Codes: G02B 13/0015 characterised by the lens design
• CPC Class Codes: G02B 13/009 having zoom function
• CPC Class Codes: G02B 27/0075 with means for altering, e.g. increasing, the depth of field or depth of focus
• CPC Class Codes: H04N 23/00 Cameras or camera modules comprising electronic image sensors; Control thereof
• CPC Class Codes: H04N 23/45 for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 23/58 Means for changing the camera field of view without moving the camera body, e.g. nutating or panning of optics or image sensors
• CPC Class Codes: H04N 23/667 Camera operation mode switching, e.g. between still and video, sport and normal or high- and low-resolution modes
• CPC Class Codes: H04N 23/67 Focus control based on electronic image sensor signals
• CPC Class Codes: H04N 23/69 Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
• CPC Class Codes: H04N 23/951 by using two or more images to influence resolution, frame rate or aspect ratio
• CPC Class Codes: H04N 23/958 for extended depth of field imaging

Abstract

A dual-aperture zoom digital camera operable in both still and video modes. The camera includes Wide and Tele imaging sections with respective lens/sensor combinations and image signal processors and a camera controller operatively coupled to the Wide and Tele imaging sections. The Wide and Tele imaging sections provide respective image data. The controller is configured to combine in still mode at least some of the Wide and Tele image data to provide a fused output image from a particular point of view, and to provide without fusion continuous zoom video mode output images, each output image having a given output resolution, wherein the video mode output images are provided with a smooth transition when switching between a lower zoom factor (ZF) value and a higher ZF value or vice versa, and wherein at the lower ZF the output resolution is determined by the Wide sensor while at the higher ZF value the output resolution is determined by the Tele sensor.

Claims

1. A dual-aperture digital camera for imaging an object or scene, comprising:

a) a Wide camera comprising a Wide lens and a Wide image sensor, the Wide camera having a respective field of view FOV_W and being operative to provide a Wide image of the object or scene;

b) a Tele camera comprising a Tele lens and a Tele image sensor, the Tele camera having a respective field of view FOV_T narrower than FOV_W and being operative to provide a Tele image of the object or scene, wherein the Tele lens has a respective effective focal length EFL_T and total track length TTL_T fulfilling the condition EFL_T/TTL_T>

1;

c) a first autofocus (AF) mechanism coupled mechanically to, and used to perform an AF action on the Wide lens;

d) a second AF mechanism coupled mechanically to, and used to perform an AF action on the Tele lens; and

e) a camera controller operatively coupled to the first and second AF mechanisms and to the Wide and Tele image sensors and configured to control the AF mechanisms and to process the Wide and Tele images to create a fused image, wherein areas in the Tele image that are not focused are not combined with the Wide image to create the fused image and wherein the camera controller is further operative to output the fused image with a point of view (POV) of the Wide camera by mapping Tele image pixels to matching pixels within the Wide image.

2. The dual-aperture digital camera of claim 1, wherein the camera controller is further configured to perform rectification of the Wide and Tele images by aligning these images to be on an approximately epipolar line to obtain rectified Wide and Tele images.

3. The dual-aperture digital camera of claim 2, wherein the camera controller is further configured to perform mapping between the rectified Wide and Tele images to produce a registration map.

4. The dual-aperture digital camera of claim 3, wherein the camera controller is further configured to perform resampling of the Tele image according to the registration map to provide a re-sampled Tele image.

5. The dual-aperture digital camera of claim 4, wherein the camera controller is further configured to process the re-sampled Tele image and the Wide image to detect an error in the registration and to provide a decision output.

6. The dual-aperture digital camera of claim 5, wherein, if an error is detected, the camera controller is further configured to choose Wide pixel values to be used in the output image for pixels that caused the error.

7. The dual-aperture digital camera of claim 1, wherein the Wide and Tele image sensors have pixels with identical pixel counts.

8. The dual-aperture digital camera of claim 7, wherein the pixel count is 12 MP.

9. The dual-aperture digital camera of claim 1, wherein the Wide and Tele image sensors have pixels with respective pixel sizes Pixel size_Wide and Pixel size_Tele and wherein Pixel size_Wide is equal to Pixel size_Tele.

10. The dual-aperture digital camera of claim 1, wherein the Wide and Tele image sensors have pixels with respective pixel sizes Pixel size_Wide and Pixel size_Tele and wherein Pixel size_Wide is not equal to Pixel size_Tele.

11. The dual-aperture digital camera of claim 1, wherein the Wide and Tele lenses have different F numbers F#_Wide and F#_Tele.

12. The dual-aperture digital camera of claim 11, wherein the camera controller is further configured to synchronize scanning of the Wide and Tele image sensors such that matching FOVs in the Wide and Tele images are scanned at the same time.

13. The dual-aperture digital camera of claim 11, wherein the camera controller is further configured to synchronize the Wide and Tele image sensors to start exposure at the same time.

14. The dual-aperture digital camera of claim 1, wherein the Wide and Tele lenses have respective F numbers F#_Wide and F#_Tele and wherein the camera controller is further configured to set respective Wide and Tele image sensor exposure times ET_Wide and ET_Tele to fulfill the condition ET_Tele=ET_Wide×

(F#_Tele/F#_Wide)²×

(Pixel size_Wide/Pixel size_Tele)².

15. The dual-aperture digital camera of claim 1, wherein the Wide and Tele lenses have respective F numbers F#_Wide and F#_Tele and wherein the camera controller is further configured to set respective Wide and Tele image sensor exposure times ET_Wide and ET_Tele to be equal.

16. The dual-aperture digital camera of claim 6, wherein the Wide and Tele image sensors have pixels with identical pixel counts and with respective pixel sizes Pixel size_Wide and Pixel size_Tele wherein Pixel size_Wide is equal to Pixel size_Tele, wherein the Wide and Tele lenses have different F numbers F#_Wide and F#_Tele, and wherein the camera controller is further configured to synchronize the Wide and Tele image sensors to start exposure at the same time.

17. The dual-aperture digital camera of claim 6, wherein the Wide and Tele image sensors have pixels with identical pixel counts and with respective pixel sizes Pixel size_Wide and Pixel size_Tele wherein Pixel size_Wide is not equal to Pixel size_Tele, wherein the Wide and Tele lenses have different F numbers F#_Wide and F#_Tele, and wherein the camera controller is further configured to synchronize the Wide and Tele image sensors to start exposure at the same time.

18. The dual-aperture digital camera of claim 6, wherein the Wide and Tele image sensors have pixels with respective pixel sizes Pixel size_Wide and Pixel size_Tele and wherein Pixel size_Wide is not equal to Pixel size_Tele, wherein the Wide and Tele lenses have different F numbers F#_Wide and F#_Tele, and wherein the camera controller is further configured to synchronize the Wide and Tele image sensors to start exposure at the same time.

19. A dual-aperture digital camera for imaging an object or scene, comprising:

a) a Wide camera comprising a Wide lens and a Wide image sensor, the Wide camera having a respective field of view FOV_W and being operative to provide a Wide image of the object or scene;

b) a Tele camera comprising a Tele lens and a Tele image sensor, the Tele camera having a respective field of view FOV_T narrower than FOV_W and being operative to provide a Tele image of the object or scene, wherein the Tele lens has a respective effective focal length EFL_T and total track length TTL_T fulfilling the condition EFL_T/TTL_T>

1;

c) a first autofocus (AF) mechanism coupled mechanically to, and used to perform an AF action on the Wide lens;

d) a second AF mechanism coupled mechanically to, and used to perform an AF action on the Tele lens, wherein the Wide and Tele lenses have different F numbers F#_Wide and F#_Tele, wherein the Wide and Tele image sensors have pixels with respective pixel sizes Pixel size_Wide and Pixel size_Tele wherein Pixel size_Wide is not equal to Pixel size_Tele, and wherein the Tele camera has a Tele camera depth of field (DOF_T) shallower than a DOF of the Wide camera (DOF_W); and

e) a camera controller operatively coupled to the first and second AF mechanisms and to the Wide and Tele image sensors and configured to control the AF mechanisms, to process the Wide and Tele images to find translations between matching points in the images to calculate depth information and to create a fused image suited for portrait photos, the fused image having a DOF shallower than DOF_T and having a blurred background.

20. The dual-aperture digital camera of claim 19, wherein the Tele lens includes five lens elements along an optical axis from an object side to an image side, starting from the object side with a first lens element with positive power, a second lens element with negative power, a fourth lens element with negative power and a fifth lens element, wherein the largest distance between consecutive lens elements along the optical axis is a distance between the fourth lens element and the fifth lens element.

21. The dual-aperture digital camera of claim 20, wherein the fused image having a DOF shallower than DOF_T is output as a portrait photo similar to a portrait photo taken with a digital single-lens reflex (DSLR) camera.

22. The dual-aperture digital camera of claim 21, wherein the DSLR has a focal length between 50-80 mm.

23. A method comprising:

a) providing a dual-camera comprising a Wide camera and a Tele camera, the Wide and Tele cameras having respective Wide and Tele lenses, Wide and Tele image sensors and Wide and Tele fields of view FOV_W and FOV_T, wherein FOV_T is narrower than FOV_W, wherein the Tele lens has a respective effective focal length EFL_T and total track length TTL_T fulfilling the condition EFL_T/TTL_T>

1;

b) acquiring a Wide image with the Wide sensor and a Tele image with the Tele sensor;

c) processing the Wide and Tele images to create a fused image, wherein areas in the Tele image that are not focused are not combined with the Wide image to create the fused image; and

d) outputting the fused image with a point of view (POV) of the Wide camera by mapping Tele image pixels to matching pixels within the Wide image.

24. The method of claim 23, further comprising rectifying the Wide and Tele images by aligning these images to be on an approximately epipolar line to obtain rectified Wide and Tele images.

25. The method of claim 24, further comprising mapping between the rectified Wide and Tele images to produce a registration map.

26. The method of claim 25, further comprising resampling the Tele image according to the registration map to provide a re-sampled Tele image.

27. The method of claim 26, further comprising processing the re-sampled Tele image and the Wide image to detect an error in the registration and to provide a decision output.

28. The method of claim 27, wherein if an error is detected, further comprising choosing Wide pixel values to be used in the output image for those pixels that caused the error.

29. The method of claim 23, wherein the Wide and Tele image sensors have pixels with identical pixel counts.

30. The method of claim 29, wherein the pixel count is 12 MP.

31. The method of claim 23, wherein the Wide and Tele image sensors have pixels with respective pixel sizes Pixel size_Wide and Pixel size_Tele and wherein Pixel size_Wide is equal to Pixel size_Tele.

32. The method of claim 23, wherein the Wide and Tele image sensors have pixels with respective pixel sizes Pixel size_Wide and Pixel size_Tele and wherein Pixel size_Wide is not equal to Pixel size_Tele.

33. The method of claim 23, wherein the Wide and Tele lenses have different F numbers F#_Wide and F#_Tele.

34. The method of claim 33, further comprising synchronizing scanning of the Wide and Tele image sensors such that matching FOVs in the Wide and Tele images are scanned at the same time.

35. The method of claim 33, further comprising synchronizing the Wide and Tele image sensors to start exposure at the same time.

36. The method of claim 23, wherein the Wide and Tele lenses have respective F numbers F#_Wide and F#_Tele, the method further comprising setting respective Wide and Tele image sensor exposure times ET_Wide and ET_Tele to fulfill the condition ET_Tele=ET_Wide×

(F#_Tele/F#_Wide)²×

(Pixel size_Wide/Pixel size_Tele)².

37. The method of claim 23, wherein the Wide and Tele lenses have respective F numbers F#_Wide and F#_Tele, the method further comprising setting respective Wide and Tele image sensor exposure times ET_Wide and ET_Tele to be equal.

38. The method of claim 28, wherein the Wide and Tele image sensors have pixels with identical pixel counts and with respective pixel sizes Pixel size_Wide and Pixel size_Tele wherein Pixel size_Wide is equal to Pixel size_Tele and wherein the Wide and Tele lenses have different F numbers F#_Wide and F#_Tele, the method further comprising synchronizing the Wide and Tele image sensors to start exposure at the same time.

39. The method of claim 28, wherein the Wide and Tele image sensors have pixels with identical pixel counts and with respective pixel sizes Pixel size_Wide and Pixel size_Tele wherein Pixel size_Wide is not equal to Pixel size_Tele and wherein the Wide and Tele lenses have different F numbers F#_Wide and F#_Tele, the method further comprising synchronizing the Wide and Tele image sensors to start exposure at the same time.

40. The method of claim 28, wherein the Wide and Tele image sensors have pixels with respective pixel sizes Pixel size_Wide and Pixel size_Tele wherein Pixel size_Wide is not equal to Pixel size_Tele and wherein the Wide and Tele lenses have different F numbers F#_Wide and F#_Tele, the method further comprising synchronizing the Wide and Tele image sensors to start exposure at the same time.