Three dimensional image data processing

• US 10,043,304 B2
• Filed: 09/23/2009
• Issued: 08/07/2018
• Est. Priority Date: 09/25/2008
• Status: Active Grant

• Current Assignee: Koninklijke Philips Electronics N.V. (Koninklijke Philips N.V.)
• Original Assignee: Koninklijke Philips N.V.
• Inventors: Barenbrug, Bart Gerard Bernard, Zwaans, Bernardus Antonius Maria, Van Woerkom, Harald Hermanus Albertus
• Primary Examiner(s): Tryder, Gregory J
• Assistant Examiner(s): LEE, KWANG B
• Application Number:
  US13/119,773
• Publication Number:
  US 20110181588A1
• Time in Patent Office: 3,240 Days
• Field of Search: 345419
• CPC Class Codes: G06T 15/205 Image-based rendering

Abstract

Three dimensional image data is provided which includes data representing a first image which specifically may be a background image. A mixed image which is a combination of the first image and a second image which specifically may be a foreground image, a transparency map related to the mixed image comprising transparency values for pixels of the mixed image and a depth indication map for the mixed image comprising depth indication values for pixels of the mixed image. The use of a mixed image may allow three dimensional processing while at the same time allowing 2D backwards compatibility. Image quality around image objects may be improved by modifying transparency values in response to depth indication values and/or depth indication values in response to transparency values. Specifically, an improved alignment of transitions of depth indication values and transparency values may provide improved three dimensional foreground image object edge data.

Claims

1. A method of providing three dimensional image data for use in rendering image elements of at least one view of the three dimensional image data, the method comprising:

providing a 2D first image, as a background image

providing a mixed image by overlaying a second image, as a foreground image, on the first image, said overlaying incorporating transparency data for the second image, wherein the mixed image includes background image objects and foreground image objects;

providing a transparency map related to the mixed image, the transparency map comprising transparency values for pixels of the mixed image;

providing a first depth indication map for the mixed image, the first depth indication map comprising depth indication values for pixels of the mixed image;

modifying values of a first set of values in an image region of the mixed image determined between a transition point in the transparency map and a transition point in the first depth indication map based on a second set of values, wherein the first set of values comprising a set of values corresponding to one of;

the transparency values and the depth indication values, and the second set of values comprising a set of values corresponding to one of;

the depth indication values and the transparency values, wherein the modified values of the first set of values represent modification of the first set of values dependent on the second set of values; and

generating the three dimensional image data, said three-dimensional image data comprising the first image, the mixed image, the transparency map and the first depth indication map.

2. The method of claim 1 further comprising generating an image signal comprising the three dimensional image data.

3. The method of claim 1 further comprising:

rendering a view for a different viewing angle than a viewing angle of the first and second image in response to the three dimensional image data.

4. The method of claim 3 further comprising:

rendering image elements corresponding to image elements of multiple views of the three dimensional image data for use in visualizing a three dimensional image in response to the three dimensional image data.

5. The method of claim 1, wherein the image region corresponds to an image object edge.

6. The method of claim 1 further comprising:

modifying a relationship between transparency values of the transparency map and depth indication values of the first depth indication map in the image region.

7. The method of claim 6 wherein modifying the relationship comprises aligning a transparency transition of the transparency map along an image direction and a depth transition in the depth indication values of the first depth indication map along the image direction.

8. The method of claim 1 further comprising:

modifying transparency values of the transparency map in response to depth indication values of the first depth indication map.

9. The method of claim 8 wherein modifying transparency values comprises:

biasing transparency values of the transparency map towards higher transparency in the image region being part of a transparency transition in the transparency map and not being part of a depth transition in the depth indication values in the first depth indication map.

10. The method of claim 8 wherein modifying transparency values comprises modifying transparency values in at least part of the image region comprising a depth transition in the depth indication values and a transparency transition along an image direction by:

determining a first depth transition point in response to the depth indication values of the first depth indication map along the image direction;

biasing transparency values of the transparency map along the image direction towards higher transparency for pixels of an interval along the image direction extending from the first depth transition point in a direction of at least one of a higher transparency and a lower depth.

11. The method of claim 10 further comprising:

determining a first transparency transition point in response to the transparency values of the first transparency map along the image direction and wherein the interval extends from the first transparency point to the first depth transition point.

12. The method of claim 11 further comprising:

determining the first transparency transition point as a point along the image direction having a highest transparency value in an evaluation interval extending from the depth transition point in a direction of at least one of a higher transparency and a lower depth.

13. The method of claim 10 wherein said biasing comprises:

setting transparency values of all transparency values of the interval to a highest transparency value of the interval prior to biasing.

14. The method of claim 10 further comprising:

providing a second depth indication map for the first image, the second depth indication map comprising depth indication values for pixels of the first image; and

wherein determining the first depth transition point comprises further determining the first depth transition point in response to depth indication values of the second depth indication map.

15. The method of claim 14 wherein determining the depth transition point comprises:

determining the depth transition point in response to a comparison of depth indication values of the first depth indication map and depth indication values of the second depth indication map.

16. The method of claim 14 wherein determining the depth transition point comprises:

determining the depth transition point as a point along the image direction for which a difference between a depth indication value of the second depth indication map and a depth indication value of the first depth indication map crosses a threshold.

17. The method of claim 1 further comprising:

modifying depth indication values of the first depth indication map in response to transparency values of the transparency map.

18. The method of claim 17 wherein modifying depth indication values comprises:

biasing depth indication values of the first depth indication map towards foreground depth indication values in the image region corresponding to a transparency transition in the transparency map.

19. The method of claim 17 wherein modifying depth indication values comprises:

modifying depth indication values in at least part of an image region comprising a depth transition and a transparency transition along an image direction by;

determining a first transition point in response to at least one of the transparency values of the transparency map and the depth indication values of the first depth indication map along the image direction;

biasing depth indication values of the first depth indication map along the image direction towards foreground depth indication values for pixels of an interval along the image direction ending at the first transparency transition point and extending from the first transparency point in a direction of lower transparency.

20. The method of claim 19 wherein the first transition point is a first transparency transition point determined in response to the transparency values of the transparency map and further comprising:

determining a second transparency transition point in response to the transparency indication values of the transparency map along the image direction, the second transparency transition point corresponding to a lower transparency than the first transparency point; and

wherein the interval extends from the first depth transition point to the second depth transition point.

21. The method of claim 19 wherein the first transition point is a first transparency transition point determined in response to the transparency values of the transparency map and further comprising:

determining a first depth transition point in response to depth indication values of the first depth indication map along the image direction;

determining a transparency search interval in response to the first depth transition point; and

determining the first transparency point comprises determining the first transparency point in response to transparency values in the transparency search interval.

22. The method of claim 19 wherein biasing depth indication values comprises setting all depth indication values of the interval to a depth indication value corresponding to a furthest foreground depth indication value for the interval prior to biasing.

23. A computer program stored on a non-transitory medium, said computer program containing code enabling a processor to carry out the method of claim 1.

24. A method of generating an image signal, the method comprising:

providing a 2D first image, as a background image;

providing a mixed image by overlaying a second image, as a foreground image, on the first image, the overlaying incorporating transparency value for the second image, wherein the mixed image includes background image objects and foreground image objects;

providing a transparency map related to the mixed image, the transparency map comprising transparency values for pixels of the mixed image;

providing a first depth indication map for the mixed image, the first depth indication map comprising depth indication values for pixels of the mixed image;

modifying values of a first set of values in response to values of a second set of values in an image region of the mixed image determined based on a transition of one of the first set of values and the second set of values, wherein the first set of values corresponding to one of;

said transparency values and said depth indication values and the second set of values corresponding to one of;

said transparency values and said depth indication values, wherein the first set of values are modified from a transition point of the first set of values to a transition point of the second set of values; and

generating the image signal representing the first image, the mixed image, the transparency map and the first depth indication map.

25. The method of claim 24, wherein the first set of values comprises transparency values of the transparency map and the second set of values comprises depth indication values of the first depth indication map.

26. The method of claim 24, wherein the first set of values comprises depth indication values of the first depth indication map and the second set of values comprises transparency values of the transparency map.

27. A method of rendering an image, the method comprising:

providing a first image;

overlaying a second image onto a section the first image to render a mixed image as a combination of the first image and a second image, the second image being a foreground image relative to the first image and the combination incorporating a transparency value for the second image, wherein the mixed image includes first image objects and second image objects;

providing a transparency map related to the mixed image, the transparency map comprising transparency values for pixels of the mixed image;

providing a first depth indication map for the mixed image, the first depth indication map comprising depth indication values for pixels of the mixed image;

setting values of a first set of values within in an image region of the mixed image representative of a transition point of the first set of values to a transition point of a second set of values, the first set of values corresponding to one of;

said transparency values and said depth indication values and the second set of values corresponding to the other of;

said transparency values and said depth indication values; and

rendering the image, wherein the image comprises the first image, the mixed image, the transparency map and the first depth indication map.

28. An image signal encoder comprising:

a data processor, accessing software, which configures the data processor to;

receive a first image;

render a mixed image by overlaying a second image onto the first image, the second image being a foreground image relative to the first image and the overlaying incorporating a transparency value for the second image, wherein the mixed image includes image objects that occur in the first image and in the second image;

generate a transparency map related to the mixed image, the transparency map comprising transparency values for pixels of the mixed image;

generate a first depth indication map for the mixed image, the first depth indication map comprising depth indication values for pixels of the mixed image;

modify values of a first set of values in response to values of a second set of values in an image region of the mixed image determined between a transition of the first set of values and a transition of the second set of values, wherein the first set of values corresponding to one of;

said transparency values of the transparency map and said depth indication values of the first depth indication map and the second set of values corresponding to the other of said transparency values of the transparency map and said depth indication values of the first depth indication map; and

generate a signal comprising the first image, the mixed image, the transparency map and the first depth indication map; and

encode said image to generate an image signal.

29. An image rendering unit comprising:

hardware configure to;

receive, as a background image, a first image;

render a mixed image as a combination of the first image and a second image by overlying the second image onto a section of the first image, the second image being a foreground image relative to the first image and the combination incorporating a transparency value for the second image, wherein the mixed image includes background image objects and foreground image objects;

provide a transparency map related to the mixed image, the transparency map comprising transparency values for pixels of the mixed image;

provide a first depth indication map for the mixed image, the first depth indication map comprising depth indication values for pixels of the mixed image;

determine a transition point in each of the transparency map and the first depth indication map;

modify values of a first set of values in an image region represented between the transition point in the transparency map and the first depth indication map, the first set of values comprising one of;

said transparency values and said depth indication values of the first depth indication map, and said second set of values comprising of the other of;

said transparency values and said depth indication values of the first depth indication map; and

generate a composite image, said composite image comprising the first image, the mixed image, the transparency map and the first depth indication map; and

render the composite image.

30. An apparatus for generating an image signal, said apparatus comprising:

a hardware platform configured to execute software, the software configuring the hardware to;

process data representing a first image, said first image representing a background image;

process data representing a mixed image by overlaying a second image onto the first image, the second image being a foreground image relative to the first image and the overlaying incorporating a transparency value for the second image, wherein the mixed image includes background image objects and foreground image objects;

process data representing a transparency map related to the mixed image, the transparency map comprising transparency values for pixels of the mixed image;

process data representing a first depth indication map for the mixed image, the first depth indication map comprising depth indication values for pixels of the mixed image;

modify values of a first set of values in response to values of a second set of values in an image region of the mixed image, wherein the image region representing an edge region between the first image and the second image and is presented between a transition point in the transparent map and a transition point in the depth indication map, the first set of values corresponding to one of;

transparency values of the transparency map and depth indication values of the first depth indication map and the second set of values corresponding to one of;

said transparency values of the transparency map and said depth indication values of the first depth indication map; and

generate the image signal, said image signal comprising the first image, the mixed image, the transparency map and the first depth indication map.