METHOD OF CREATING A PARALLAX VIDEO FROM A STILL IMAGE

• US 20150172627A1
• Filed: 07/02/2014
• Published: 06/18/2015
• Est. Priority Date: 12/13/2013
• Status: Active Grant

• Current Assignee: HTC Corporation
• Original Assignee: HTC Corporation
• Inventors: LEE, Huai-Che, CHEN, Li-Cheng, TSENG, Yung-Chao, CHUEH, Hsin-Ti
• Granted Patent: US 9,979,952 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: H04N 13/106 Processing image signals for multi-view video sequence encoding H04N19/597
• CPC Class Codes: H04N 13/261 with monoscopic-to-stereoscopic image conversion

Abstract

An automatic calibration method suitable for an electronic apparatus comprising a proximity sensor is disclosed. The automatic calibration method includes steps of: resetting data comprising a minimal index and a maximal index when a specific function is activated; updating the minimal index and the maximal index according to readings from the proximity sensor; resetting the minimal index when the proximity sensor detects a far event according to a threshold value of the proximity sensor; resetting the maximal index when the proximity sensor detects a near event according to the threshold value; and, re-calculating the threshold value according to the minimal index or the maximal index every time the minimal index or the maximal index is updated.

Claims

1. A method for creating a parallax video from a still image, the method comprising:

performing an image segmentation for separating at least a foreground object within the still image from an image background of the still image;

filling a blank region of the image background, the blank region corresponding to where the separated foreground object locates within the still image;

creating a virtual three-dimensional (3D) scene consisting of the foreground object and the image background;

analyzing the virtual 3D scene for determining a camera path; and

creating the parallax video by observing the virtual 3D scene with a virtual camera moving along the camera path.

2. The method of claim 1, wherein, in the virtual 3D scene, the foreground object is configured to be located in front of the image background by a depth difference.

3. The method of claim 1, wherein the step of filling the blank region comprises:

analyzing neighborhood pixels adjacent to tee the blank region of the image background; and

filling the blank region with a suitable pattern analyzed from the neighborhood pixels.

4. The method of claim 1, wherein the step of analyzing the virtual 3D scene for determining a camera path comprises:

determining whether the blank region of the image background is at least partially filled as an in-painting region; and

estimating different leakage ratios of the in-painting region corresponding to how much the in-painting region is observable from different candidate nodes by the virtual camera in the virtual 3D scene.

5. The method of claim 4, wherein the camera path comprises an initial node, a terminal node and at least one intermediate node, the step of determining the camera path further comprises:

setting the initial node at an original viewing point of the still image;

setting the terminal node according to a position and a size of the foreground object within the still image; and

determining the at least one intermediate node between the initial node and the terminal node.

6. The method of claim 5, wherein the at least one intermediate node of the camera path is determined node-by-node from the initial node to the terminal node, the step of determining each of the at least one intermediate node comprises:

finding a target vector from the initial node or a current intermediate node toward the terminal node;

creating a plurality of refraction vectors around the target vector; and

finding an optimal vector from the refraction vectors and target vector, and assigning an end of the optimal vector as a next intermediate node.

7. The method of claim 6, wherein the step of finding the optimal vector from the refraction vectors and target vector further comprises:

assigning ends of the refraction vectors and target vector as the candidate nodes;

verifying whether each visual window of the virtual camera from each of the candidate nodes is completely covered by the virtual 3D scene, wherein if any visual window exceeds the virtual 3D scene, the corresponding candidate node is disqualified;

calculating the leakage ratios corresponding to the qualified candidate nodes; and

finding the optimal vector by selecting an optimal node from the qualified candidate nodes according to the leakage ratios, such that the leakage ratio of the optimal node of the camera path is less than a criteria upper bound.

8. The method of claim 7, wherein the criteria upper bound is that 50% of the in-painting region is observable by the virtual camera from one of the candidate nodes.

9. The method of claim 6, wherein the refraction vectors are created along an orientation of the target vector with different angles deviated from the target vector.

10. The method of claim 5, wherein the terminal node is set to a right side from the initial node when the position of the foreground object is located at a right half of the still image, and the terminal node is set to a left side from the initial node when the position of the foreground object is located at a left half of the still image.

11. An electronic apparatus, comprising:

a data storage module, configured for storing a still image; and

a control module, coupled with the data storage module, the control module comprising;

a segmentation unit configured for separating at least a foreground object within the still image from an image background of the still image;

an in-painting unit configured for filling a blank region of the image background, the blank region corresponding to where the separated foreground object locates within the still image;

a scene simulation unit configured for creating a virtual three-dimensional (3D) scene consisting of the foreground object and the image background;

a scene analyzer unit configured for analyzing the virtual 3D scene for determining a camera path; and

a video generator unit configured for generating a parallax video by observing the virtual 3D scene with a virtual camera moving along the camera path.

12. The electronic apparatus of claim 11, wherein the scene simulation unit locates the foreground object in front of the image background by a depth difference in the virtual 3D scene.

13. The electronic apparatus of claim 11, wherein the in-painting unit analyzes neighborhood pixels adjacent to the blank region of the image background, and the in-painting unit fills the blank region with a suitable pattern analyzed from the neighborhood pixels.

14. The electronic apparatus of claim 11, wherein the scene analyzer unit determines whether the blank region of the image background is filled as an in-painting region, the scene analyzer unit estimates different leakage ratios of the in-painting region corresponding to how much the in-painting region is observable from different candidate nodes by the virtual camera in the virtual 3D scene, and the scene analyzer unit determines the camera path according to the leakage ratios.

15. The electronic apparatus of claim 14, wherein the camera path comprises an initial node, a terminal node and at least one intermediate node, the scene analyzer unit sets the initial node at an original viewing point of the still image, and the scene analyzer unit sets the terminal node according to a position and a size of the foreground object within the still image.

16. The electronic apparatus of claim 15, wherein the at least one intermediate node of the camera path is determined node-by-node from the initial node to the terminal node, the scene analyzer unit determines each of the at least one intermediate node by finding a target vector from the initial node or a current intermediate node toward the terminal node, creating a plurality of refraction vectors around the target vector, finding an optimal vector from the refraction vectors and target vector, and assigning an end of the optimal vector as a next intermediate node.

17. A non-transitory computer readable storage medium, containing instructions for performing a method comprising:

performing an image segmentation for separating at least a foreground object within the still image from an image background of the still image;

filling a blank region of the image background, the blank region corresponding to where the separated foreground object locates within the still image;

creating a virtual three-dimensional (3D) scene consisting of the foreground object and the image background;

analyzing the virtual 3D scene for determining a camera path; and

creating a parallax video by observing the virtual 3D scene with a virtual camera moving along the camera path.