Application processor for disparity compensation between images of two cameras in digital photographing apparatus

• US 10,616,549 B2
• Filed: 01/24/2018
• Issued: 04/07/2020
• Est. Priority Date: 06/23/2017
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Park, Dong Jin, Kang, Seok Jae, Lee, Jee Hong, Im, Joon Hyuk, Kim, Chae Sung
• Primary Examiner(s): Dang, Philip P.
• Application Number:
  US15/878,540
• Publication Number:
  US 20180376122A1
• Time in Patent Office: 804 Days
• CPC Class Codes: G01S 19/42 Determining position
• CPC Class Codes: G01S 19/53 Determining attitude
• CPC Class Codes: H04N 13/111 Transformation of image signals corresponding to virtual viewpoints, e.g. spatial image interpolation
• CPC Class Codes: H04N 13/128 Adjusting depth or disparity
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/271 wherein the generated image signals comprise depth maps or disparity maps
• CPC Class Codes: H04N 13/296 Synchronisation thereof; Control thereof
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals
• CPC Class Codes: H04N 23/45 for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 23/611 where the recognised objects include parts of the human body
• CPC Class Codes: H04N 23/62 Control of parameters via user interfaces
• CPC Class Codes: H04N 23/63 by using electronic viewfinders
• CPC Class Codes: H04N 23/631 Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters
• CPC Class Codes: H04N 23/635 Region indicators; Field of view indicators
• CPC Class Codes: H04N 23/951 by using two or more images to influence resolution, frame rate or aspect ratio
• CPC Class Codes: H04N 5/2621 Cameras specially adapted for the electronic generation of special effects during image pickup, e.g. digital cameras, camcorders, video cameras having integrated special effects capability
• CPC Class Codes: H04N 5/2628 Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation
• CPC Class Codes: H04N 5/265 Mixing

Abstract

A digital photographing device may include a plurality of cameras on a common side of the device, an application processor for switching image capture between the camera, and a display. The application processor may switch images output on the display when the cameras are switched. During the image transition, one or more virtual viewpoint images are output between a pre-transition image and a post-transition image. The virtual viewpoint images interpolate a disparity between the pre-transition image and the post-transition image caused by the different cameras being located at different positions, and result in a smooth visual transition. When a camera switching input includes a zoom factor signal, the virtual viewpoint images may be compensated images according to the input zoom factor and a disparity.

Claims

1. An application processor comprising:

an input/output (I/O) interface configured to receive a user input including a zoom factor; and

a central processor configured to;

while a first image of a scene acquired from a first camera of a photographing apparatus is displayed, the first camera being disposed at a first position and having a first angle of view, receive from the I/O interface a signal indicative of a first zoom factor greater than a minimum zoom factor of a switchover region, and in response generate a control signal so that a current first image acquired by the first camera is digitally zoomed to generate a zoomed first image, and a second image of at least a portion of the scene is acquired by a second camera disposed at a second position of the photographing apparatus and having a narrower angle of view, the second image being acquired by the second camera according to the first zoom factor substantially simultaneously with the acquiring of the first image by the first camera;

generate one or more third images, which are virtually acquired at one or more third positions between the first position and the second position, using the zoomed first image and the second image, and cause the one or more third images to be sequentially displayed after the displaying of the first image; and

if the zoom factor becomes a maximum zoom factor of the switchover region, cause a current second image acquired by the second camera according to the maximum zoom factor to be displayed.

2. The application processor of claim 1, wherein the minimum zoom factor of the switchover region is a default zoom factor of the first camera, and the maximum zoom factor is a set zoom factor.

3. The application processor of claim 1, wherein the central processor causes the one or more third images to be displayed in order from a third image acquired at a third position closer to the first position than to the second position, to a third image acquired at a third position closer to the second position than to the first position.

4. The application processor of claim 1, further comprising an image signal processor (ISP) configured to perform image signal processing on the first image and the second image before the one or more third images are generated.

5. The application processor of claim 4, wherein the central processor receives a user input corresponding to a zoom factor between the minimum zoom factor of the switchover region and the maximum zoom factor of the switchover region through the input/output interface, and generates a control signal for switching cameras in response to the user input.

6. The application processor of claim 5, further comprising an image generator for camera switching configured to generate a disparity map between the first image having undergone image signal processing and the second image having undergone image signal processing in response to the control signal of the central processor, and to generate the one or more third images based on the disparity map.

7. The application processor of claim 5, further comprising a memory configured to store a camera switching program,

wherein the central processor generates a disparity map between the first image having undergone image signal processing and the second image having undergone image signal processing by executing the camera switching program, and generates the one or more third images based on the disparity map.

8. The application processor of claim 1, wherein before generating the one or more third images, the central processor aligns the zoomed first image and the second image and generates a disparity map based on the aligned zoomed first image and second image.

9. The application processor of claim 1, wherein a biometric signal of a user sensed through an input section is received through the input/output interface and transferred to the central processor, and

the central processor generates a control signal for switching camera corresponding to the biometric signal of the user.

10. The application processor of claim 1, wherein when any control signal generated in response to a signal received from the I/O interface indicative of a zoom factor between minimum zoom factor and the maximum zoom factor is generated for a predetermined time or more, the central processor causes the second image to be displayed.

11. The application processor of claim 1, wherein the central processor sets a frame rate of the one or more third images to be different from a frame rate of the first image and the second image.

12. The application processor of claim 1, wherein:

the control signal is generated in response to a zoom request signal for a specific region in the first image while the first image is displayed; and

the one or more third images are generated using image elements of the zoomed first image including the specific region, and image elements of the second image including the specific region.

13. An application processor comprising:

an input/output (I/O) interface configured to receive a user input including a zoom factor; and

a central processor configured to;

while a first image of a scene acquired from a first camera of a photographing apparatus is displayed, the first camera being disposed at a first position and having a first angle of view, receive from the I/O interface a signal indicative of a first zoom factor less than a maximum zoom factor of a switchover region, and in response generate a control signal so that a second image acquired by a second camera is digitally zoomed to generate a zoomed second image, and a current first image is acquired by the first camera according to the first zoom factor, substantially simultaneously with the acquiring of the second image by the second camera, wherein the first camera has a narrower angle of view than that of the second camera and the second camera is disposed at a second position of the photographing apparatus;

generate one or more third images, which are virtually acquired at one or more third positions between the first position and the second position, using the zoomed second image and the current first image, and cause the one or more third images to be sequentially displayed after the displaying of the first image; and

if the zoom factor becomes a minimum zoom factor of the switchover region, cause a current second image acquired by the second camera to be displayed according to the minimum zoom factor.

14. The application processor of claim 13, wherein the central processor causes the one or more third images to be displayed in order from a third image acquired at a third position closer to the second position than to the first position, to a third image acquired at a third position closer to the first position than to the second position.

15. The application processor of claim 13, wherein when any control signal generated in response to a signal received from the I/O interface indicative of a zoom factor between the minimum zoom factor and the maximum zoom factor is generated for a predetermined time or more, the central processor causes the zoomed first image to be displayed.

16. A photographing apparatus comprising:

a wide-angle camera disposed at a first position and a second camera disposed at a second position;

an application processor comprising;

an input/output (I/O) interface configured to receive a user input including a zoom factor; and

a central processor configured to;

acquire a wide-angle image captured by the wide-angle camera, and display the wide-angle image;

receive a switching input from the I/O interface, which includes a first input zoom factor that is greater than a minimum zoom factor of a switchover region;

acquire a current wide-angle image captured by the wide-angle camera and digitally zoom the current wide-angle image with the first input zoom factor to generate a zoomed wide-angle image;

acquire a second image captured by the second camera according to the first input zoom factor;

generate one or more third images, which are virtually acquired at one or more third positions between the first position and the second position, using elements of the zoomed wide-angle image and the second image, in response to the switching input; and

cause the one or more third images to be sequentially displayed after the displaying of the wide-angle image.

17. The photographing apparatus of claim 16, wherein the central processor causes the one or more third images to be displayed in order from a third image acquired at a third position closer to the first position than to the second position, to a third image acquired at a third position closer to the second position than to the first position.

18. The photographing apparatus of claim 16, wherein the application processor further comprises an image signal processor (ISP) configured to perform image signal processing on the wide-angle image and the second image.

19. The photographing apparatus of claim 18, further comprising an image generator for camera switching configured to generate a disparity map between the wide-angle image having undergone image signal processing and the second image having undergone image signal processing in response to a control signal, and to generate the one or more third images based on the disparity map.

20. The photographing apparatus of claim 18, wherein the application processor further comprises a memory configured to store a camera switching program,

wherein the central processor generates a disparity map between the wide-angle image having undergone image signal processing and the second image having undergone image signal processing by executing the camera switching program, and generates the one or more third images based on the disparity map.