Method and apparatus for generating enhanced 3D-effects for real-time and offline applications

• US 10,250,864 B2
• Filed: 12/27/2017
• Issued: 04/02/2019
• Est. Priority Date: 10/30/2013
• Status: Active Grant

• Current Assignee: VEFXi Corporation
• Original Assignee: VEFXi Corporation
• Inventors: Barkatullah, Javed Sabir
• Primary Examiner(s): Vaughn, Jr., William C
• Assistant Examiner(s): Jean Baptiste, Jerry T
• Application Number:
  US15/855,756
• Publication Number:
  US 20180139432A1
• Time in Patent Office: 461 Days
• CPC Class Codes: H04N 13/128 Adjusting depth or disparity
• CPC Class Codes: H04N 13/261 with monoscopic-to-stereoscopic image conversion

Abstract

A method for adjusting and generating enhanced 3D-effects for 2D to 3D image and video conversion applications includes controlling a depth location of a zero parallax plane within a depth field of an image scene to adjust parallax of objects in the image scene, controlling a depth volume of objects in the image scene to one of either exaggerate or reduce 3D-effect of the image scene, controlling a depth location of a segmentation plane within the depth field of the image scene, dividing the objects in the image scene into a foreground group and a background group, selectively increasing or decreasing depth volume of objects in the foreground group, selectively increasing or decreasing depth separation of objects in the foreground group relative to the objects in the background group, and generating an updated depth map file for a 2D-image.

Claims

1. A method for adjusting and generating enhanced 3D-effects for real time and offline 2D to 3D image and video conversion applications consisting of:

(a) selectively controlling a depth location of a zero parallax plane within a depth field of an image scene to adjust parallax of objects in the image scene;

(b) selectively controlling a depth volume of objects in the image scene to one of either exaggerate or reduce 3D-effect of the image scene;

(c) selectively controlling a depth location of a segmentation plane within the depth field of the image scene, wherein said depth location is a non-zero depth location, dividing the objects in the image scene into a foreground group and a background group based on a location of the objects relative to the segmentation plane wherein an object of said foreground group is in said background group when said depth location of said segmentation plane is moved forward and wherein an object of said background group is in said foreground group when said depth location of said segmentation plane is moved backward, wherein said segmentation plane is moved from a zero location to a different location where as a result of moving said segmentation plane to said different location at least one of,

(i) objects that were in said foreground group when said segmentation plane was at said zero location are moved to said background group when said segmentation plane is moved to said different location, and;

(ii) objects that were in said background group when said segmentation plane was at said zero location are moved to said foreground group when said segmentation plane is moved to said different location;

(d) selectively increasing or decreasing depth volume of objects in the foreground group independently of selectively increasing or decreasing depth volume of objects in the background group, wherein said depth volume of objects in said foreground group is modified to change available foreground volume in which objects to be rendered are mapped, wherein said depth volume of objects in said background group is modified to change available background volume in which objects to be rendered are mapped,

(i) wherein objects that were in said foreground group when said segmentation plane was at said zero location that are moved to said background group when said segmentation plane is moved to said different location are said selectively increased or decreased in said depth volume as objects in said background group, and

(ii) wherein objects that were in said background group when said segmentation plane was at said zero location that are moved to said foreground group when said segmentation plane is moved to said different location are said selectively increased or decreased in said depth volume as objects in said foreground group;

(e) selectively increasing or decreasing depth separation of objects in the foreground group relative to the objects in the background group, where said separation includes both a step offset and a slope scaling, wherein said step offset and said slope scaling is relative to said available foreground volume being fixed, wherein said step offset and said slope scaling is relative to said available background volume being fixed, wherein objects in said foreground group and said background group include a continuous range of available depths prior to said selectively increasing or decreasing said depth separation of objects in said foreground group relative to said objects in said background group and wherein objects in said foreground group and said background group include a discontinuous range of available depths after said selectively increasing or decreasing said depth separation of objects in said foreground group relative to said objects in said background group, wherein said discontinuous range includes a prohibited range of depths within said continuous range of available depths being said step offset;

(f) generating an updated depth map file for a 2D-image based upon the controlling the depth location, the controlling the depth volume, the increasing and decreasing depth volume, and the increasing and decreasing depth separation;

(g) rendering an enhanced 3D-image using the updated depth map.

2. The method of claim 1, wherein the method further comprises a software application running on a computing device.

3. The method of claim 2, wherein the computing device comprises one of a server computer, personal computer, tablet computer or smart-phone, graphics processor unit.

4. The method of claim 1, further comprising receiving a 2D-still image or a streaming 2D-video from a network with an associated depth map.

5. The method of claim 1, further comprising reading a 2D-still image or a 2D-video from a file stored on a local or remote storage device with the associated depth map image.

6. The method of claim 1, further comprising generating a depth map for each 2D-still image or a sequence of depth maps for each frame in a 2D-video.

7. The method of claim 1, further comprising reading meta-instructions for depth map enhancement for the 2D-image or video from a file stored on a local or remote storage device.

8. The method of claim 1, further comprising enabling a user to enhance the depth map through one of a set of graphical user interfaces (GUI), command line instructions, and custom input devices.

9. The method of claim 1, wherein rendering a 3D image comprises one of rendering an anaglyph, stereo-3D or auto-stereo 3D using the enhanced depth map.

10. The method of claim 1, further comprising one of displaying generated 3D image or video on and attached 3D display in real time, and storing the 3D image on local or remote storage device(s) for offline viewing.

11. The method of claim 1, further comprising storing the generated enhanced depth map as grey scale images on a storage device.

12. The method of claim 1, further comprising storing user modifications of the depth map as a sequence of instructions associated with each image in a control file using a pre-defined syntax.

13. The method of claim 1, wherein the method is executed by a dedicated hardware device.

14. The method of claim 1, wherein the method is executed by hardware contained in a stand-alone converter box.

15. The method of claim 1, wherein the method is implemented as one of a circuit board, a daughter card or any other plug-in card or module.