Efficient coding of depth transitions in 3D (video)

• US 10,097,843 B2
• Filed: 11/08/2010
• Issued: 10/09/2018
• Est. Priority Date: 11/13/2009
• Status: Active Grant

• Current Assignee: Koninklijke Philips Electronics N.V. (Koninklijke Philips N.V.)
• Original Assignee: Koninklijke Philips Electronics N.V. (Koninklijke Philips N.V.)
• Inventors: Bruls, Wilhelmus Hendrikus Alfonsus, Varekamp, Christiaan, Klein Gunnewiek, Reinier Bernardus Maria
• Primary Examiner(s): Philippe, Gims S
• Assistant Examiner(s): Kir, Albert
• Application Number:
  US13/508,739
• Publication Number:
  US 20120229603A1
• Time in Patent Office: 2,892 Days
• Field of Search: 345420, 345427, 345646, 382173, 382260, 348159, 348 43, 358 301, 37524012
• CPC Class Codes: H04N 13/161 Encoding, multiplexing or demultiplexing different image signal components for multi-view video sequence encoding H04N19/597
• CPC Class Codes: H04N 19/21 with binary alpha-plane coding for video objects, e.g. context-based arithmetic encoding [CAE]
• CPC Class Codes: H04N 19/46 Embedding additional information in the video signal during the compression process H04N19/517, H04N19/68, H04N19/70 take precedence
• CPC Class Codes: H04N 19/597 specially adapted for multi-view video sequence encoding
• CPC Class Codes: H04N 2213/003 Aspects relating to the "2D+depth" image format

Abstract

A method of encoding a video data signal (15) is provided, together with a method for decoding. The encoding comprises providing color information (51) for pixels in an image, providing a depth map with depth information (52) for the pixels, providing transition information (56, 57, 60, 70, 71) being representative of a width (63, 73) of a transition region (61, 72) in the image, the transition region (61, 72) comprising a depth transition (62) and blended pixels in which colors of a foreground object and a background object are blended, and generating (24) the video data signal (15) comprising encoded data representing the color information (51), the depth map (52) and the transition information (56, 57, 60, 70, 71). The decoding comprises using the transition information (56, 57, 60, 70, 71) for determining the width (63, 73) of the transition regions (61, 72) and for determining alpha values (53) for pixels inside the transition regions (61, 72). The determined alpha values (53) are used for determining the color of a blended pixel at the transition of a foreground object and a background object.

Claims

1. A method, comprising:

receiving at an input of an encoder circuit a first input video data signal and a second input video data signal, wherein the first input video data signal is produced from a first camera and the second input video data signal is produced from a second camera, wherein the first camera and the second camera view a scene from two different angles;

the encoder circuit generating, from the first input video data signal and the second input video data signal, at least one base two-dimensional image of a scene, wherein the base two-dimensional image comprises color information for pixels in the base two-dimensional image;

the encoder circuit generating from the received first input video data signal and the second input video data signal, depth information for the base two-dimensional image;

the encoder circuit generating from the received first input video data signal and second input video data signal transition information, wherein the transition information is representative of a width of a transition region in the base two-dimensional image, the transition region comprising a depth transition and blended pixels, wherein the blended pixels comprise blended colors of a foreground object and a background, wherein the transition information comprises a width map wherein pixels in the base two-dimensional image along a line in the transition region each have an associated non-zero value in the width map, the non-zero value defining for the depth transition a corresponding width parameter indicating a number of blended pixels in the transition region, and all other pixels in the transition region have associated zero values in the width map,

the encoder circuit generating an encoded three-dimensional video data signal comprising encoded data representing the base two-dimensional image, the depth map, and the transition information; and

the encoder circuit transmitting the encoded three-dimensional video data signal from an output of the encoder circuit.

8. A non-transitory computer medium for encoding a video data signal, which medium is operative to cause a processor circuit to perform the method as claimed in claim 1.

11. The method of claim 1, wherein the encoded three-dimensional video signal further includes at least one parameter describing at least one of a lighting condition, a position, a numerical aperture, and a focal length setting for at least one of the first and second cameras at a time when it produced the corresponding first or second input video data signal.

2. A method, comprising:

receiving at an input of an encoder circuit at least one input video data signal;

the encoder circuit generating from the at least one input video data signal at least one base two-dimensional image of a scene, wherein the base two-dimensional image comprises color information for pixels in the base two-dimensional image;

the encoder circuit generating from the at least one input video data signal, depth information for the base two-dimensional image;

the encoder circuit generating from the at least one input video data signal transition information, wherein the transition information is representative of a width of a transition region in the base two-dimensional image, the transition region comprising a depth transition and blended pixels, wherein the blended pixels comprise blended colors of a foreground object and a background object, wherein the transition information comprises a binary map indicating with a single bit for each of a plurality of sections of one or more pixels in the base two-dimensional image whether the section comprises a transition region or not, and

the encoder circuit generating an encoded three-dimensional video data signal comprising encoded data representing the base two-dimensional image, the binary map, and the transition information.

3. The method of claim 2, wherein the binary map has a lower resolution than the base two-dimensional image.

9. The method of claim 2, wherein each section comprises a plurality of pixels.

10. The method of claim 2, wherein each section comprises exactly one pixel.

4. A method, comprising:

receiving at an input of an encoder circuit at least one input video data signal;

the encoder circuit generating from the at least one input video data signal at least one base two-dimensional image of a scene, wherein the base two-dimensional image comprises color information for pixels in the base two-dimensional image;

the encoder circuit generating from the at least one input video data signal, depth information for the base two-dimensional image;

the encoder circuit generating from the at least one input video data signal transition information, wherein the transition information is representative of a width of a transition region in the base two-dimensional image, wherein the transition region comprises a depth transition and blended pixels, wherein the blended pixels comprise blended colors of a foreground object and a background object, wherein the transition information comprises a table of width values, in units of pixels, for a corresponding plurality of different transition depths, and

the encoder circuit generating an encoded three-dimensional video data signal comprising encoded data representing the base two-dimensional image, the depth map, and the transition information.

5. The method of claim 4, wherein the transition depths are defined as a depth of the foreground object at the depth transition.

6. The method of claim 4, wherein the transition depths are defined as a combination of the depths of the foreground object and the background object at the depth transition.

7. An encoder circuit, comprising:

an input configured to receive at least one input video data signal;

a processer circuit configured to;

generate from the at least one input video data signal at least one base two-dimensional image of a scene including color information for pixels in the base two-dimensional image;

generate from the at least one input video data signal depth information for the base two-dimensional image;

generate from the at least one input video data signal transition information, wherein the transition information is representative of a width of a transition region in the base two-dimensional image, wherein the transition region comprises a depth transition and blended pixels, wherein the blended pixels comprise blended colors of a foreground object and a background object, wherein the transition information comprises one of;

a width map wherein pixels in the base two-dimensional image along a line in the transition region each have an associated non-zero value in the width map, the non-zero value defining for the depth transition a corresponding width parameter indicating a number of blended pixels in the transition region, and all other pixels in the transition region have associated zero values in the width map,

a binary map indicating with a single bit for each of a plurality of sections of one or more pixels in the base two-dimensional image whether the section comprises a transition region or not, and

a table of width values, in units of pixels, for a corresponding plurality of different transition depths; and

generate an encoded three-dimensional video data signal comprising encoded data representing the base two-dimensional image, the depth map, and the transition information; and

an output configured to transmit the encoded three-dimensional video data signal.