METHOD AND DEVICE FOR PROVIDING A LAYERED DEPTH MODEL OF A SCENE

• US 20110199379A1
• Filed: 10/15/2009
• Published: 08/18/2011
• Est. Priority Date: 10/21/2008
• Status: Abandoned Application

• Current Assignee: Koninklijke Philips Electronics N.V. (Koninklijke Philips N.V.)
• Original Assignee: Koninklijke Philips Electronics N.V. (Koninklijke Philips N.V.)
• Inventors: Benien, Christian, Uerlings, Harald Helmut, Mehl, Stefanie, Gremse, Felix, Helbing, Marc, Barenbrug, Bart Gerard Bernard
• Application Number:
  US13/124,283
• Publication Number:
  US 20110199379A1
• Time in Patent Office: Days
• US Class Current: 345/427
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06T 15/10 Geometric effects
• CPC Class Codes: G06T 15/20 Perspective computation
• CPC Class Codes: G06T 15/205 Image-based rendering
• CPC Class Codes: G06T 17/00 Three dimensional [3D] modelling, e.g. data description of 3D objects
• CPC Class Codes: G06T 19/00 Manipulating 3D models or images for computer graphics
• CPC Class Codes: H04N 13/128 Adjusting depth or disparity
• CPC Class Codes: H04N 13/161 Encoding, multiplexing or demultiplexing different image signal components for multi-view video sequence encoding H04N19/597
• CPC Class Codes: H04N 19/46 Embedding additional information in the video signal during the compression process H04N19/517, H04N19/68, H04N19/70 take precedence
• CPC Class Codes: H04N 19/597 specially adapted for multi-view video sequence encoding
• CPC Class Codes: H04N 2213/003 Aspects relating to the "2D+depth" image format

Abstract

The invention relates to a device and method of providing a layered depth model of a scene, the layers of the depth model comprising primary view information for a primary view of the scene from a primary viewing direction and occlusion information associated with the primary view information for use in rendering in other views of the scene, wherein: the primary view information comprises layer segments of the model which are depth-wise closest with respect to the primary viewing direction, and the occlusion information comprises further layer segments of the model and wherein the occlusion in formation comprises a safety region (SR1, SR2, SR3, SR4) adjacent to a depth transition for which occlusion information is provided (J1, J2, J3, J4), and wherein the safety region comprises corresponding segments of the primary view information, and wherein the safety region (SR1, SR2, SR3, SR4) is located on that side of the respective depth transition which is depth-wise farthest away with respect to the primary viewing direction. The invention further relates to a signal comprising a layered depth model of a scene as described above, as well as a computer program product implementing the method according to the invention.

Claims

1. Method of providing (510) a layered depth model (605) of a scene, the layers of the depth model (605) comprising primary view information for a primary view of the scene from a primary viewing direction (PVD) and occlusion information associated with the primary view information for use in rendering other views of the scene, wherein:

the primary view information comprises layer segments of the model which are depth-wise closest with respect to the primary viewing direction, and

the occlusion information comprises further layer segments of the model and wherein the occlusion information comprises a safety region (SR) adjacent to a depth transition for which occlusion information is provided (J1, J2, J3, J4), and wherein the safety region comprises corresponding segments of the primary view information, and wherein the safety region (SR) is located on that side of the respective depth transition which is depth-wise farthest away with respect to the primary viewing direction.

2. Method of claim 1, wherein

the primary view information and the occlusion information each comprise both image information and depth information.

3. Method of claim 1, wherein

the layered depth model (605) is a two layer model comprising:

a top layer (TL) encoding the primary view information and

an occlusion layer (OL2) encoding the occlusion information.

4. Method of claim 3, wherein

the occlusion layer (OL2) comprises corresponding segments of the top layer (TL) whenever there is no other information available.

5. Method of claim 1, wherein

the size of the occlusion information is based on one of:

a width based on a maximum deviation from the viewing angle of the primary view and the depth information,

a predetermined width on the one side of a depth transition and

a predetermined width on the one side and a further predetermined width on the other side of a depth transition.

6. Method of claim 1, wherein

the size of the safety region (SR) is based on one of

a predetermined maximum downscale factor and

a granularity of an occlusion information codec.

7. Method of claim 1, wherein

the layered depth model (605) further comprises

transparency information for the primary view information, wherein the transparency value in the safety region (SR) adjacent to the depth transition is substantially transparent and the transparency value in the safety region at other end of the safety region is substantially non-transparent.

8. Method of claim 7, wherein

the transparency values in the safety region (SR) comprise one of

a gradient from substantially transparent adjacent to the depth transition, to substantially non-transparent at the other end of the safety region,

a predetermined number of consecutive substantially transparent pixels adjacent to the depth transition,

a predetermined number of consecutive substantially transparent pixels adjacent to the depth transition and a further predetermined number of consecutive substantially non-transparent pixels at the other end of the safety region,

a predetermined number of consecutive substantially transparent pixels adjacent to the depth transition and a gradient from substantially transparent adjacent to the consecutive fully transparent pixels to substantially non-transparent,

a predetermined number of consecutive substantially transparent pixels adjacent to the depth transition and a gradient from substantially transparent adjacent to the consecutive fully transparent pixels to substantially non-transparent followed by a further predetermined number of consecutive substantially non-transparent pixels and

a predetermined number of consecutive substantially transparent pixels adjacent to the depth transition and a sharp transition from substantially transparent adjacent to the consecutive fully transparent pixels to substantially non-transparent, the transition aligned with a block boundary of a video codec, the transition followed by a further predetermined number of consecutive substantially non-transparent pixels.

9. Method of claim 7, wherein

in the primary view information a depth hole protection region (DHP) is provided for a region comprising a depth hole having a width below a first threshold width, and where in the depth hole protection region:

the primary view depth information (D1,D2) is based on both high edge depth values of the depth hole and

the transparency values (T1, T2) between the adjacent depth jumps are set to substantially transparent.

10. Method of claim 7, wherein

in the primary view information a depth spike protection region (DSP) is provided for a region comprising a depth spike having a width below a second threshold width, and where in the depth spike protection region:

the primary view depth information is set to the depth value of the top of the depth spike within the depth spike protection region, and

the transparency values in the depth spike protection region outside the depth spike are set to substantially transparent.

11. Method of processing (500) a three dimensional model of a scene, the method comprising:

obtaining (505) a three dimensional model of a scene,

providing (510) a layered depth model of the scene according to claim 1 and

processing (515) the three dimensional model of the scene.

16. A computer program product, comprising program instructions for executing the method claim 1.

12. Signal (615) comprising a layered depth model (605) of a scene, the layers of the depth model (605) comprising:

encoded primary view information for a primary view of the scene from a primary viewing direction (PVD) the primary view information comprising layer segments of the model which are depth-wise closest with respect to the primary viewing direction and

encoded occlusion information, associated with the primary view information for use in rendering other views of the scene, wherein the occlusion information comprises further layer segments of the model and wherein the occlusion information comprises a safety region (SR) adjacent to a depth transition for which occlusion information is provided (J1, J2, J3, J4), and wherein the safety region comprises corresponding segments of the primary view information, and wherein the safety region (SR) is located on that side of the respective depth transition which is depth-wise farthest away with respect to the primary viewing direction.

13. Signal of claim 12, wherein

the layered depth model is a two layer model comprising:

a top layer (TL) encoding the primary view information and

an occlusion layer (OL2) encoding the occlusion information.

14. Device (610) for providing a layered depth model (605) of a scene, the layers of the depth model (605) comprising primary view information for a primary view of the scene from a primary viewing direction (PVD) and occlusion information associated with the primary view information for use in rendering other views of the scene, the device (610) arranged to provide:

the primary view information such that it comprises layer segments of the model which are with respect to their depth closest to the viewpoint of the primary view, and

the occlusion information such that it comprises further layer segments of the model and

wherein the occlusion information comprises a safety region (SR) adjacent to depth transition for which occlusion information is provided (J1, J2, J3, J4), and wherein the safety region (SR) comprises corresponding segments of the primary view information, and wherein the safety regions (SR) is located on that side of the respective depth transitions which is depth-wise farthest away with respect to the primary viewing direction.

15. Device (600) for processing a three dimensional model of a scene, the device comprising:

receiving means (670) arranged to receive a three dimensional model of a scene,

device (610) for providing a layered depth model (605) of a scene according to claim 14 arranged to provide a layered depth model based on the three dimensional model of the scene and

processing (680) means arranged to process the layered depth model.