Three dimensional image data processing

• US 8,890,868 B2
• Filed: 09/18/2009
• Issued: 11/18/2014
• Est. Priority Date: 09/25/2008
• Status: Expired due to Fees

• Current Assignee: Koninklijke Philips Electronics N.V. (Koninklijke Philips N.V.)
• Original Assignee: Koninklijke Philips N.V.
• Inventors: Zwaans, Bernardus A. M., Barenburg, Bart G. B.
• Primary Examiner(s): McDowell, Jr., Maurice L
• Application Number:
  US13/119,269
• Publication Number:
  US 20110169823A1
• Time in Patent Office: 1,887 Days
• Field of Search: 345/422, 345/419
• US Class Current: 345/422
• CPC Class Codes: G06T 15/205 Image-based rendering

Abstract

An apparatus generates three dimensional image data. Specifically, image data for a first image layer and image data for a second image layer is generated together with depth indication data for at least one of the first image layer and the second image layer. The two image layers may be used e.g. for dependent images, such as foreground and background images, or for independent images, such as initial and final images for a cross fade. A dependency indicator which is indicative of a dependency between the image layers is generated and included in the three dimensional data. A rendering device may receive the three dimensional data and generate images for a three dimensional representation based on the data. The three dimensional processing is adapted in dependence on the dependency indicator. For example, interlayer de-occlusion processing may only be performed if the dependency indicator is indicative of dependent image layers.

Claims

1. A method of generating a three dimensional image data stream comprising a plurality of image layers for use in rendering picture elements of views of a three dimensional image, the method comprising:

providing first image data for a first image layer;

providing second image data for a second image layer;

providing depth indication data for at least one of the first image layer and the second image layer;

generating a dependency indicator, the dependency indicator being indicative of whether a first image of the first image layer contains supplementary three dimensional layered image data for a second simultaneous image of the second image layer; and

generating the three dimensional image data stream by combining the first image data, the second image data, the depth indication data and the dependency indicator.

2. The method of claim 1 wherein the dependency indicator is indicative of the first image layer and the second image layer being dependent when the first image layer is an occlusion layer for the second image.

3. The method of claim 2 wherein the dependency indicator is indicative of the first image layer and the second image layer being independent when the first image layer is not an occlusion layer for the second image.

4. The method of claim 1 wherein the depth indication data comprises a first depth indication map for the first image layer and a second depth indication map for the second image layer;

and wherein the dependency indicator is indicative of the first image layer and the second image layer being independent when depth indication values of the first depth indication map are inconsistent with depth indication values of the second depth indication map.

5. The method of claim 4 wherein the depth indication values of the first depth indication map are inconsistent with the depth indication values of the second depth indication map if the depth indication values of the first depth indication map represent deeper depth levels than depth indication values of the second indication map in at least one image region and the depth indication values of the second depth indication map represent deeper depth levels than the depth indication values of the first indication map in at least one other image region.

6. The method of claim 1 wherein the dependency indicator is indicative of the first image layer and the second image layer being independent when the first image layer and the second image layer comprises cross fade images, and is indicative of the first image layer and the second image layer being dependent when the first image layer and the second image layer do not comprise cross fade images.

7. The method of claim 1 wherein the dependency indicator is a binary data value having a first value indicating that the first image layer and the second image layer are dependent and a second value indicating that the first image layer and the second image layer are independent.

8. The method of claim 7 wherein the dependency indicator is set to the first value if the first image layer comprises a foreground image for an image of the second image layer and to the second value if the first image layer does not comprise a foreground image for the image of the second image layer.

9. The method of claim 1 wherein the first image layer belongs to a first image layer group and the second image layer belongs to a second image layer group, at least one of the first image layer group and the second image layer group comprising a plurality of image layers;

and the dependency indicator is indicative of a dependency between image layers of the first image layer group and image layers of the second image layer group.

10. The method of claim 9 wherein image layers of an image layer group are dependent image layers.

11. The method of claim 9 wherein image data for image layers in an image layer group are of adjacent layers and the dependency indicator is indicative of a layer location in the three dimensional image data steam of a next independent image layer group.

12. The method of claim 1 further comprising providing transparency data for at least one of the first image layer and the second image layer and including the transparency data in the three dimensional image data stream.

13. A method of rendering image elements comprising:

receiving a three dimensional image data stream comprising first image data for a first image layer, second image data for a second image layer, depth indication data for at least one of the first image layer and the second image layer, and a dependency indicator indicative of a dependency between the first image layer and the second image layer; and

rendering image elements of multiple views of the three dimensional image data stream for use in visualizing a three dimensional image by processing the first image data, the second image data and the depth indication data in response to the dependency indicator.

14. The method of claim 13 wherein the rendering comprises performing interlayer three dimensional image processing of simultaneous images of the first image layer and the second image layer only when the dependency indicator is indicative of the first image layer and the second image layer being dependent.

15. The method of claim 13 wherein the rendering comprises performing de-occlusion for a first image of the first image layer using information from a second image of the second image layer only if the dependency indicator is indicative of the first image layer and the second image layer being dependent.

16. The method of claim 13 wherein the rendering comprises combining a first image of the first image layer and a second image of the second image layer, the combination being in response to three dimensional image information when the dependency indicator is indicative of the first image layer and the second image layer being dependent and not being in response to three dimensional image information when the dependency indicator is indicative of the first image layer and the second image layer being independent.

17. An apparatus for generating a three dimensional image data stream comprising a plurality of image layers for use in rendering picture elements of views of a three dimensional image, the apparatus comprising:

a first image layer processor configured to provide first image data for a first image layer;

a second image layer processor configured to provide second image data for a second image layer;

a depth map processor configured to provide depth indication data for at least one of the first image layer and the second image layer;

a dependency processor configured to generate a dependency indicator indicative of a dependency between the first image layer and the second image layer; and

a data combiner configured to generate the three dimensional image data stream by combining the first image data, the second image data, the depth indication data and the dependency indicator.

18. An apparatus for rendering image elements, the apparatus comprising:

a receiver configure to receive a three dimensional image data stream comprising first image data for a first image layer, second image data for a second image layer, depth indication data for at least one of the first image layer and the second image layer, and a dependency indicator indicative of a dependency between the first image layer and the second image layer; and

a rendering unit configured to render image elements of multiple views of the three dimensional image data stream for use in visualizing a three dimensional image by processing the first image data, the second image data and the depth indication data in response to the dependency indicator.

19. A method of generating a three dimensional image data stream comprising a plurality of image layers for use in rendering picture elements of views of a three dimensional image, the method comprising:

providing a first image layer comprising image data that at least for some images represents a composite image comprising a contribution from a plurality of images;

providing a first depth layer comprising a composite depth map for the composite image;

providing a second image layer comprising image data for a further image contributing to the composite image;

providing a second depth layer comprising a further depth map for the further image;

providing a transparency map representing a contribution of the further image to the composite image;

generating a dependency indicator reflecting whether an image of the second image layer comprises at least one of a foreground image and a background image for at least one image contributing to a composite image of the first image layer or whether the image of the second image layer is not one of a foreground image and a background image of any image contributing to the composite image; and

generating the three dimensional image data stream by combining the first image layer, the first depth layer, the second image layer, the second depth layer, the transparency map and the dependency indicator.

20. A method of rendering image elements comprising:

receiving a three dimensional image data stream comprising

a first image layer comprising image data that at least for some images represents a composite image comprising a contribution from a plurality of images;

a first depth layer comprising a composite depth map for the composite image;

a second image layer comprising image data for a further image contributing to the composite image;

a second depth layer comprising a further depth map for the further image;

a transparency map representing a contribution of the further image to the composite image;

a dependency indicator reflecting whether an image of the second image layer comprises at least one of a foreground image and a background image for at least one image contributing to a composite image of the first image layer or whether the image of the second image layer is not one of a foreground image and a background image of any image contributing to the composite image; and

rendering image elements of multiple views of the three dimensional image data stream for use in visualizing a three dimensional image by three dimensional processing of the three dimensional image data stream wherein de-occlusion for an image of the first image layer is in response to a simultaneous image of the second image layer only if the dependency indicator is indicative of the image of the second image layer being at least one of a background image and a foreground image of an image contributing to the image of the first image layer.