Image processing apparatus and method for three-dimensional image zoom

• US 9,396,590 B2
• Filed: 08/23/2013
• Issued: 07/19/2016
• Est. Priority Date: 03/08/2013
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Jung, Seung Won, Choi, Ouk
• Primary Examiner(s): Liu, Zhengxi
• Application Number:
  US13/974,677
• Publication Number:
  US 20140253542A1
• Time in Patent Office: 1,061 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 2203/04806 Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: G06T 2219/2016 Rotation, translation, scaling
• CPC Class Codes: G06T 3/40 Scaling of whole images or parts thereof, e.g. expanding or contracting

Abstract

An image processing apparatus and method for zooming in on a partial area in a three-dimensional (3D) image selects a zoom mode from among a two-dimensional (2D) zoom mode, a 3D zoom mode, and an intermediate dimensional zoom mode between the 2D zoom mode and the 3D zoom mode. The image processing apparatus may include a mode selecting unit to select a zoom mode to be applied to a zoom area in a color image and a depth image among the 2D zoom mode, the 3D zoom mode, and the intermediate dimensional zoom mode between the 2D zoom mode and the 3D zoom mode, and a scaling unit to scale the zoom area using a zoom factor indicating magnification or minification of the zoom area and the selected zoom mode.

Claims

1. An image processing apparatus comprising:

a processor; and

a memory including computer readable instructions, which when executed by the processor cause the processor to,

select a zoom mode to be applied to a zoom area in a color image and a depth image, the zoom mode being selected from among a two-dimensional (2D) zoom mode, a three-dimensional (3D) zoom mode, and an intermediate dimensional zoom mode between the 2D zoom mode and the 3D zoom mode; and

scale the zoom area based on a zoom factor indicating magnification or minification of the zoom area and the selected zoom mode, wherein the processor is configured to select the zoom mode by,

estimating a range of depth information of the zoom area to be scaled using the intermediate dimensional zoom mode, and

selecting the intermediate dimensional zoom mode if a difference between the estimated range of depth information and a range of depth information of a current zoom area is less than equal to a threshold value.

2. The image processing apparatus of claim 1, wherein the intermediate dimensional zoom mode has a fewer number of calculations than the 3D zoom mode.

3. The image processing apparatus of claim 1, wherein, in the intermediate dimensional zoom mode, the processor is configured to scale the depth information of the zoom area based on the zoom factor.

4. The image processing apparatus of claim 3, wherein the processor is configured to scale the depth information of the points included in the zoom area in inverse proportion to a size of the zoom area to be magnified if the processor magnifies the zoom area using the zoom factor.

5. The image processing apparatus of claim 1, wherein the processor is configured to determine the range of depth information based on a difference between a greatest amount of depth information and a least amount of depth information in the zoom area.

6. The image processing apparatus of claim 1, wherein, in the 3D zoom mode, the processor is configured to shift the depth information of the zoom area based on the zoom factor.

7. The image processing apparatus of claim 6, wherein, in the 3D zoom mode, the processor is configured to generate a scaled color image and a scaled depth image by transforming 3D coordinates of the zoom area based on the zoom factor and by projecting the transformed 3D coordinates into a 2D space.

8. The image processing apparatus of claim 1, wherein the processor is configured to estimate a range of depth information of the zoom area to be scaled using the intermediate dimensional zoom mode, and select the 3D zoom mode if a difference between the estimated range of depth information and a range of depth information of a current zoom area exceeds a threshold value.

9. The image processing apparatus of claim 1, wherein in the 2D zoom mode, the processor is configured to scale a size of the zoom area in proportion to the zoom factor.

10. The image processing apparatus of claim 1, wherein the processor is configured to determine whether the depth information of the zoom area to be scaled is included in a depth zone, and select the 2D zoom mode if the depth information of the zoom area to be scaled is outside of the desired depth zone.

11. The image processing apparatus of claim 10, wherein the depth zone corresponds to a comfort zone or a Panum'"'"'s fusional area in which fatigue experienced by a user viewing a 3D image is minimized.

12. The image processing apparatus of claim 1, wherein the processor is configured to select the 2D zoom mode if the zoom area includes additional information of the color image or information lacking the depth information.

13. The image processing apparatus of claim 1, wherein the processor is configured to,

set a zoom area of a color image and a zoom area of a depth image on an area in a 3D image based on a selection by a user, and

set a zoom factor based on color information and the depth information in the zoom area, and information received from the user.

14. The image processing apparatus of claim 13, wherein the processor is configured to limit a zoom factor based on a resolution of the 3D image.

15. An image processing apparatus comprising:

a processor; and

a memory including computer readable instructions, which when executed by the processor cause the processor to,

set a zoom area in a color image and a depth image to be zoomed; and

generate a scaled color image and a scaled depth image based on a selected zoom mode from among a two-dimensional (2D) zoom mode, a three-dimensional (3D) zoom mode, and an intermediate dimensional zoom mode between the 2D zoom mode and the 3D zoom mode, wherein the processor is configured to select a zoom mode by,

estimating a range of depth information of the zoom area to be scaled using the intermediate dimensional zoom mode, and

selecting the intermediate dimensional zoom mode if a difference between the estimated range of depth information and a range of depth information of a current zoom area is less than or equal to a threshold value.

16. The image processing apparatus of claim 15, wherein in the 3D zoom mode, the processor is configured to transform 3D coordinates of the zoom area based on a zoom factor and by projecting the transformed 3D coordinates into a 2D space.

17. The image processing apparatus of claim 15, wherein, in the 3D zoom mode, the processor is configured to transform the 3D coordinates of the zoom area by shifting the depth information of the zoom area based on the zoom factor.

18. The image processing apparatus of claim 15, wherein, in the intermediate dimensional zoom mode, the processor is configured to scale the depth information of the zoom area based on a zoom factor by scaling the depth information of points included in the zoom area in inverse proportion to a size of the zoom area to be magnified.

19. The image processing apparatus of claim 15, wherein, in the 2D zoom mode, the processor is configured to scale the size of the zoom area in proportion to a zoom factor.

20. An image processing method comprising:

selecting a zoom mode to be applied to a zoom area in a color image and a depth image from among a two-dimensional (2D) zoom mode, a three-dimensional (3D) zoom mode, and an intermediate dimensional zoom mode between the 2D zoom mode and the 3D zoom mode; and

scaling the zoom area based on a zoom factor and the selected zoom mode, wherein the selecting the zoom mode includes,

estimating a range of depth information of the zoom area to be scaled using the intermediate dimensional zoom mode, and

selecting the intermediate dimensional zoom mode if a difference between the estimated range of depth information and a range of depth information of a current zoom area is less than or equal to a threshold value.

21. The image processing method of claim 20, wherein the intermediate dimensional zoom mode has a fewer number of calculations than the 3D zoom mode.

22. The image processing method of claim 20, wherein in the intermediate dimensional zoom mode, the scaling comprises scaling the depth information of the zoom area based on the zoom factor.

23. The image processing method of claim 20, wherein, in the 3D zoom mode, the scaling comprises shifting the depth information of the zoom area based on the zoom factor.

24. An image processing method comprising:

setting a zoom area based on an area in an image selected by a user;

selecting a zoom factor for the set zoom area based on an input from the user; and

adaptively changing between zoom modes to be applied to the zoom area, among a plurality of zoom modes, in response to a change in the selected zoom factor, wherein the plurality of zoom modes include a two-dimensional (2D) zoom mode, a three-dimensional (3D) zoom mode, and an intermediate dimensional zoom mode between the 2D zoom mode and the 3D zoom mode, wherein the adaptively changing includes,

estimating a range of depth information of the zoom area to be scaled using the intermediate dimensional zoom mode, and

selecting the intermediate dimensional zoom mode if a difference between the estimated range of depth information and a range of depth information of a current zoom area is less than or equal to a threshold value.

25. The image processing method of claim 24, wherein the adaptively changing comprises:

changing to the intermediate dimensional zoom mode or the 3D zoom mode based upon a likelihood of distortion occurring in the zoom area in the intermediate dimensional zoom mode.

26. The image processing method of claim 24, wherein the adaptively changing comprises:

changing to the intermediate dimensional zoom mode or the 3D zoom mode based upon whether a just noticeable depth difference occurs in the intermediate dimensional zoom mode.

27. The image processing method of claim 24, wherein the adaptively changing comprises:

determining a first depth range of the zoom area by calculating a difference between minimum depth information and maximum depth information in the zoom area;

determining a second depth range corresponding to the zoom area to be scaled based upon the selected zoom factor, by calculating a difference between the minimum depth information and the maximum depth information in the zoom area to be scaled;

calculating a difference between the first depth range and the second depth range; and

adaptively changing to the intermediate dimensional zoom mode or 3D zoom mode based upon the calculated difference between the first depth range and the second depth range.