Apparatus and method for depth manipulation of stereoscopic 3D image

• US 9,445,074 B2
• Filed: 05/30/2013
• Issued: 09/13/2016
• Est. Priority Date: 10/26/2012
• Status: Active Grant

• Current Assignee: Korea Advanced Institute of Science and Technology
• Original Assignee: Korea Advanced Institute of Science and Technology
• Inventors: Noh, Jun Yong, Lee, Sang Woo, Kim, Young Hui, Lee, Jung Jin, Kim, Kye Hyun, Lee, Kyung Han
• Primary Examiner(s): Li, Tracy
• Assistant Examiner(s): Rahman, Mohammad J
• Application Number:
  US13/905,418
• Publication Number:
  US 20140118507A1
• Time in Patent Office: 1,202 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: H04N 13/128 Adjusting depth or disparity

Abstract

Provided are an apparatus and a method for manipulating depth of a stereoscopic image that enable a user to manipulate depth of a stereoscopic image easily and intuitively. The apparatus may include: a histogram analyzing unit acquiring a depth distribution chart by performing histogram analysis of a disparity map corresponding to an input image; a Gaussian mixture model (GMM) fitting unit acquiring multiple depth layers by performing GMM fitting of the depth distribution chart; a disparity adjusting unit adjusting at least one of a position and a volume of at least one of the multiple depth layers and calculating a disparity mapping function on which a result of the adjustment is reflected in response to a user request; and a disparity map recalculating unit calculating a new disparity map by optimizing the disparity mapping function.

Claims

1. A stereoscopic image depth manipulating apparatus comprising:

a histogram analyzing unit acquiring a depth distribution chart by performing histogram analysis of a disparity map corresponding to an input image;

a Gaussian mixture model (GMM) fitting unit acquiring multiple depth layers by performing GMM fitting of the depth distribution chart;

a disparity adjusting unit adjusting at least one of a position and a volume of at least one of the multiple depth layers and calculating a disparity mapping function on which a result of the adjustment is reflected in response to a user request; and

a disparity map recalculating unit calculating a new disparity map by optimizing the disparity mapping function, which is calculated by the disparity adjusting unit, wherein the disparity adjusting unit calculates a disparity mapping function d_i′

defined by the following equation in a case where a user selects one of the multiple depth layers and adjusts the volume of the selected depth layer,

5. The stereoscopic image depth manipulating apparatus of claim 1, wherein the disparity adjusting unit calculates a disparity mapping function defined by the following equation in a case where a user selects one of the multiple depth layers and adjusts the position of the selected depth layer,

11. The stereoscopic image depth manipulating apparatus of claim 1, wherein the histogram analyzing unit further includes a function of manipulating a window size used for the histogram analysis in response to a user request.

2. A stereoscopic image depth manipulating apparatus comprising:

a histogram analyzing unit acquiring a depth distribution chart by performing histogram analysis of a disparity map corresponding to an input image;

a Gaussian mixture model (GMM) fitting unit acquiring multiple depth layers by performing GMM fitting of the depth distribution chart;

a disparity adjusting unit adjusting at least one of a position and a volume of at least one of the multiple depth layers and calculating a disparity mapping function on which a result of the adjustment is reflected in response to a user request; and

a disparity map recalculating unit calculating a new disparity map by optimizing the disparity mapping function, which is calculated by the disparity adjusting unit, wherein the disparity adjusting unit calculates a disparity mapping function defined by the following equation in a case where a user selects one of the multiple depth layers and adjusts the position of the selected depth layer,

12. The stereoscopic image depth manipulating apparatus of claim 2, wherein the histogram analyzing unit further includes a function of manipulating a window size used for the histogram analysis in response to a user request.

3. A stereoscopic image depth manipulating apparatus comprising:

a histogram analyzing unit acquiring a depth distribution chart by performing histogram analysis of a disparity map corresponding to an input image;

a Gaussian mixture model (GMM) fitting unit acquiring multiple depth layers by performing GMM fitting of the depth distribution chart;

a disparity adjusting unit adjusting at least one of a position and a volume of at least one of the multiple depth layers and calculating a disparity mapping function on which a result of the adjustment is reflected in response to a user request; and

a disparity map recalculating unit calculating a new disparity map by optimizing the disparity mapping function, which is calculated by the disparity adjusting unit, wherein the disparity map recalculating unit optimizes the disparity mapping function by acquiring a solution of an energy function defined by the following equation using an energy minimization technique,

4. The stereoscopic image depth manipulating apparatus of claim 3, wherein the disparity map recalculating unit can change at least one of λ

_o, λ

_T, and Ω

_j in accordance with a request from a user.

13. The stereoscopic image depth manipulating apparatus of claim 3, wherein the histogram analyzing unit further includes a function of manipulating a window size used for the histogram analysis in response to a user request.

6. A method of manipulating depth of a stereoscopic image, the method comprising:

acquiring a depth distribution chart by determining a window size used for histogram analysis with a user'"'"'s approval and performing the histogram analysis of a disparity map using the window size;

acquiring multiple depth layers from the depth distribution chart;

selecting at least one of the multiple depth layers with a user'"'"'s approval, adjusting at least one of a position and a volume of the selected depth layer, and then recalculating a disparity mapping function on which a result of the adjustment is reflected; and

calculating a new disparity map by optimizing the disparity mapping function, which is recalculated in the step of selecting the multiple depth layers, wherein, in the recalculating of a disparity mapping function, a disparity mapping function d_i′

defined by the following equation is calculated in a case where a user selects one of the multiple depth layers and adjusts the volume of the selected depth layer,

10. The method of manipulating depth of a stereoscopic image of claim 6, wherein, in the recalculating of a disparity mapping function, a disparity mapping function defined by the following equation is calculated in a case where a user selects one of the multiple depth layers and adjusts the position of the selected depth layer,

7. A method of manipulating depth of a stereoscopic image, the method comprising:

acquiring a depth distribution chart by determining a window size used for histogram analysis with a user'"'"'s approval and performing the histogram analysis of a disparity map using the window size;

acquiring multiple depth layers from the depth distribution chart;

selecting at least one of the multiple depth layers with a user'"'"'s approval, adjusting at least one of a position and a volume of the selected depth layer, and then recalculating a disparity mapping function on which a result of the adjustment is reflected; and

calculating a new disparity map by optimizing the disparity mapping function, which is recalculated in the step of selecting the multiple depth layers, wherein, in the recalculating of a disparity mapping function, a disparity mapping function defined by the following equation is calculated in a case where a user selects one of the multiple depth layers and adjusts the position of the selected depth layer,

8. A method of manipulating depth of a stereoscopic image, the method comprising:

acquiring a depth distribution chart by determining a window size used for histogram analysis with a user'"'"'s approval and performing the histogram analysis of a disparity map using the window size;

acquiring multiple depth layers from the depth distribution chart;

selecting at least one of the multiple depth layers with a user'"'"'s approval, adjusting at least one of a position and a volume of the selected depth layer, and then recalculating a disparity mapping function on which a result of the adjustment is reflected; and

calculating a new disparity map by optimizing the disparity mapping function, which is recalculated in the step of selecting the multiple depth layers, wherein, in the calculating of a new disparity map, the disparity mapping function is optimized by acquiring a solution of an energy function defined by the following equation using an energy minimization technique,

9. The method of manipulating depth of a stereoscopic image of claim 8, wherein, in the recalculating of a disparity mapping function, at least one of λ

_o, λ

_T, and Ω

_j is changed in accordance with a request from a user.