Depth Coding as an Additional Channel to Video Sequence

• US 20110122225A1
• Filed: 11/23/2010
• Published: 05/26/2011
• Est. Priority Date: 11/23/2009
• Status: Abandoned Application

• Current Assignee: Google Technology Holdings LLC (Alphabet Inc.)
• Original Assignee: General Instrument Corporation (CommScope Holding Company, Inc.)
• Inventors: Kim, Jae Hoon, Wang, Limin
• Application Number:
  US12/952,781
• Publication Number:
  US 20110122225A1
• Time in Patent Office: Days
• US Class Current: 348/42
• CPC Class Codes: H04N 19/597 specially adapted for multi-view video sequence encoding
• CPC Class Codes: H04N 2213/003 Aspects relating to the "2D+depth" image format

Abstract

A 3D video coding apparatus and method which selectively codes video data from a plurality of video sources to include depth information. Coding may be performed by combining depth information with view information, such as RGB, YCrCb, or YUV, and coded together with the view information as, RGBD, YCrCbD or YUVD. An apparatus may selectively code the depth information based on a depth format flag to include no depth information (e.g. a 2D format) or include depth information as a chroma channel. The depth information may be coded separately or together with YCrCb based on a coding cost or rate distortion estimate to encode the video information to obtain the highest quality.

Claims

1. An apparatus for encoding video data comprising:

an encoder configured to encode said video data by encoding a combined set of view data and depth data.

2. The apparatus of claim 1, wherein said combined set of view data and depth data includes one of:

RGBD, YUVD, or YCbCrD.

3. The apparatus of claim 2, wherein said combined set of view data and depth data is contained in at least one of:

a group of pictures, a picture, a slice, a group of blocks, a macroblock, or a sub-macroblock.

4. The apparatus of claim 1, further comprising a depth format unit configured to identify a depth format of said video data.

5. The apparatus of claim 4, wherein said encoder selects to encode said video data as a plurality of two dimensional images without including depth data when said depth format is set to 0.

6. The apparatus of claim 4, wherein said encoder selects to encode said video data as said combined set of view data and depth data when said depth format is set to a predetermined level.

7. The apparatus of claim 1, wherein said encoder further includes a coding cost calculator which determines coding costs of joint encoding of said combined set of view data and depth data and separate encoding of said combined set of view data and depth data, and determines an encoding mode between joint encoding and separate encoding based on said coding costs.

8. The apparatus of claim 7, wherein said encoder encodes said video data as a joint encoding of view data and depth data when said encoding cost is less than an encoding cost of separately encoding said view data and depth data.

9. The apparatus of claim 1, wherein said video data is one of a:

multiview with depth, multiview without depth, single view with depth, single view without depth.

10. A method of encoding video data comprising:

encoding said video data by encoding a combined set of view data and depth data at an encoder.

11. The method of claim 10, wherein said combined set of view data and depth data includes one of:

RGBD, YUVD, or YCbCrD.

12. The method of claim 11, wherein said combined set of view data and depth data is contained in at least one of:

a group of pictures, a picture, a slice, a group of blocks, a macroblock, or a sub-macroblock.

13. The method of claim 10, further comprising identifying a depth format of said video data.

14. The method of claim 13, wherein said video data is encoded as a plurality of two dimensional images without including depth data when said depth format is set to 0.

15. The method of claim 13, wherein said combined set of view data and depth data is encoded when said depth format is set to a predetermined level.

16. The method of claim 10, further including determining a coding cost of joint encoding said combined set of view data and depth data and separate encoding of said combined set of view data and depth data, and determining an encoding mode between joint encoding and separate encoding based on said coding costs.

17. The method of claim 16, wherein said video data is encoded as a joint encoding of view data and depth data when said encoding cost is less than an encoding cost of separately encoding said view data and depth data.

18. The method of claim 10, wherein said video data is one of a:

multiview with depth, multiview without depth, single view with depth, single view without depth.

19. A non-transitory computer readable medium carrying instructions for an encoder to encode video data, comprising instruction to perform said steps of:

encoding said video data by encoding a combined set of view data and depth data.

20. The computer readable medium of claim 19, wherein said combined set of view data and depth data includes one of:

RGBD, YUVD, or YCbCrD.

21. The computer readable medium of claim 20, wherein said combined set of view data and depth data is contained in at least one of:

a group of pictures, a picture, a slice, a group of blocks, a macroblock, or a sub-macroblock.

22. The computer readable medium of claim 19, further comprising identifying a depth format of said video data.

23. The computer readable medium of claim 22, wherein said video data is encoded as a plurality of two dimensional images without including depth data when said depth format is set to 0.

24. The computer readable medium of claim 22, wherein said combined set of view data and depth data is encoded jointly when said depth format is set to a predetermined level.

25. The computer readable medium of claim 19, further including determining a coding cost of joint encoding said combined set of view data and depth data and separate encoding of said combined set of view data and depth data, and determining an encoding mode between joint encoding and separate encoding based on said coding cost.

26. The computer readable medium of claim 25, wherein said video data is encoded as a joint encoding of view data and depth data when said encoding cost is less than an encoding cost of separately encoding said view data and depth data.

27. The computer readable medium of claim 19, wherein said video data is one of a:

multiview with depth, multiview without depth, single view with depth, single view without depth.

28. An apparatus for decoding video data comprising:

a decoder configured to decode said video data by decoding a combined set of view data and depth data.

29. The apparatus of claim 28, wherein said combined set of view data and depth data includes one of:

RGBD, YUVD, or YCbCrD.

30. The apparatus of claim 29, wherein said combined set of view data and depth data is contained in at least one of:

a group of pictures, a picture, a slice, a group of blocks, a macroblock, or a sub-macroblock.

31. The apparatus of claim 28, further comprising a depth format unit configured to identify a depth format of said video data.

32. The apparatus of claim 31, wherein said decoder selects to decode said video data as a plurality of two dimensional images without including depth data when said depth format is set to 0.

33. The apparatus of claim 31, wherein said decoder selects to decode said video data as said combined set of view data and depth data when said depth format is set to a predetermined level.

34. The apparatus of claim 28, wherein said decoder selectively jointly decodes said combined set of view data and depth data when said combined set was jointly encoded or decodes said combined set of view data and depth data when said combined set was separately encoded.

35. The apparatus of claim 28, wherein said video data is one of a:

multiview with depth, multiview without depth, single view with depth, single view without depth.

36. A method of decoding video data comprising:

decoding said video data by decoding a combined set of view data and depth data at a decoder.

37. The method of claim 36, wherein said combined set of view data and depth data includes one of:

RGBD, YUVD, or YCbCrD.

38. The method of claim 37, wherein said combined set of view data and depth data is contained in at least one of:

a group of pictures, a picture, a slice, a group of blocks, a macroblock, or a sub-macroblock.

39. The method of claim 36, further comprising identifying a depth format of said video data.

40. The method of claim 39, wherein said video data is decoded as a plurality of two dimensional images without including depth data when said depth format is set to 0.

41. The method of claim 39, wherein said combined set of view data and depth data is decoded jointly when said depth format is set to a predetermined level.

42. The method of claim 36, further including selectively jointly decoding said combined set of view data and depth data when said combined set was jointly encoded or decoding said combined set of view data and depth data when said combined set was separately encoded.

43. The method of claim 36, wherein said video data is one of a:

multiview with depth, multiview without depth, single view with depth, single view without depth.

44. A non-transitory computer readable medium carrying instructions for a decoder to decode video data, comprising instruction to perform said steps of:

decoding said video data by encoding a combined set of view data and depth data.

45. The computer readable medium of claim 44, wherein said combined set of view data and depth data includes one of:

RGBD, YUVD, or YCbCrD.

46. The computer readable medium of claim 45, wherein said combined set of view data and depth data is contained in at least one of:

a group of pictures, a picture, a slice, a group of blocks, a macroblock, or a sub-macroblock.

47. The computer readable medium of claim 44, further comprising identifying a depth format of said video data.

48. The computer readable medium of claim 47, wherein said video data is decoded as a plurality of two dimensional images without including depth data when said depth format is set to 0.

49. The computer readable medium of claim 47, wherein said combined set of view data and depth data is decoded jointly when said depth format is set to a predetermined level.

50. The computer readable medium of claim 44, further including selectively jointly decoding said combined set of view data and depth data when said combined set was jointly encoded or decoding said combined set of view data and depth data when said combined set was separately encoded.

51. The computer readable medium of claim 44, wherein said video data is one of a:

multiview with depth, multiview without depth, single view with depth, single view without depth.