Tiling in video decoding and encoding

• US 8,780,998 B2
• Filed: 04/11/2008
• Issued: 07/15/2014
• Est. Priority Date: 04/12/2007
• Status: Active Grant

• Current Assignee: Dolby Laboratories Licensing Corporation (Dolby Laboratories Incorporated)
• Original Assignee: Thomson Licensing (Vantiva SA)
• Inventors: Pandit, Purvin Bibhas, Yin, Peng, Tian, Dong
• Primary Examiner(s): Perungavoor, Sath V
• Assistant Examiner(s): WILLIAMS, JEFFERY A
• Application Number:
  US12/450,829
• Publication Number:
  US 20100046635A1
• Time in Patent Office: 2,286 Days
• Field of Search: 382/296, 375/240.25
• US Class Current: 375/240.25
• CPC Class Codes: H04N 13/111 Transformation of image signals corresponding to virtual viewpoints, e.g. spatial image interpolation
• CPC Class Codes: H04N 13/161 Encoding, multiplexing or demultiplexing different image signal components for multi-view video sequence encoding H04N19/597
• CPC Class Codes: H04N 13/194 Transmission of image signals
• CPC Class Codes: H04N 19/172 the region being a picture, frame or field
• CPC Class Codes: H04N 19/182 the unit being a pixel
• CPC Class Codes: H04N 19/46 Embedding additional information in the video signal during the compression process H04N19/517, H04N19/68, H04N19/70 take precedence
• CPC Class Codes: H04N 19/597 specially adapted for multi-view video sequence encoding
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 19/70 characterised by syntax aspects related to video coding, e.g. related to compression standards
• CPC Class Codes: H04N 2213/003 Aspects relating to the "2D+depth" image format

Abstract

Implementations are provided that relate, for example, to view tiling in video encoding and decoding. A particular method includes accessing a video picture that includes multiple pictures combined into a single picture, accessing information indicating how the multiple pictures in the accessed video picture are combined, decoding the video picture to provide a decoded representation of at least one of the multiple pictures, and providing the accessed information and the decoded video picture as output. Some other implementations format or process the information that indicates how multiple pictures included in a single video picture are combined into the single video picture, and format or process an encoded representation of the combined multiple pictures.

Claims

1. A method, comprising:

accessing a video picture that includes multiple pictures combined into a single picture, the multiple pictures including a picture of a first view and a picture of a second view in which a common object or region appears shifted from one view to the other, the video picture being part of a received video stream;

accessing information indicating how the multiple pictures in the accessed video picture are combined, wherein the accessed information indicates whether at least one of the multiple pictures is individually flipped in one or more of a horizontal direction or a vertical direction such that the common object or region from the first view picture is juxtaposed with the common object or region from the second view picture, and wherein the accessed information is part of the received video stream;

decoding the video picture to provide a decoded representation of at least one of the multiple pictures; and

post-processing the decoded representation using the accessed information.

2. The method of claim 1, wherein the multiple pictures includes the picture of the first view, and a second picture including depth information for the picture of the first view.

3. The method of claim 1, wherein the accessing of the video picture, the accessing of the information, and the decoding of the video picture are performed at a decoder.

4. The method of claim 1, wherein at least one of the multiple pictures represents depth information.

19. The method of claim 1, further comprising:

receiving user input selecting at least one of the multiple pictures for display; and

providing a decoded output of the at least one selected picture, the decoded output being provided based on the accessed information, the decoded representation, and the user input.

21. The method of claim 1 wherein the accessed information indicates that the multiple pictures are combined into the single picture in a manner that provides increased spatial correlation between corresponding portions of the multiple pictures, thereby allowing coding gains in coding the single picture.

22. The method of claim 1 wherein post-processing the decoded representation comprises flipping the decoded representation, if the accessed information indicates that the at least one of the multiple pictures is flipped.

23. The method of claim 1 wherein:

at least one of the multiple pictures is flipped in the horizontal direction in the single picture, and

the accessed information indicates that the flipping is in the horizontal direction.

24. The method of claim 1 wherein:

at least one of the multiple pictures is flipped in the vertical direction in the single picture, and

the accessed information indicates that the flipping is in the vertical direction.

5. An apparatus, comprising:

a decoder for

(i) accessing a video picture that includes multiple pictures combined into a single picture, the multiple pictures including a picture of a first view and a picture of a second view in which a common object or region appears shifted from one view to the other, the video picture being part of a received video stream,

(ii) accessing information indicating how the multiple pictures in the accessed video picture are combined, wherein the accessed information indicates whether at least one of the multiple pictures is individually flipped in one or more of a horizontal direction or a vertical direction such that the common object or region from the first view picture is juxtaposed with the common object or region from the second view picture, and wherein the accessed information is part of the received video stream, and

(iii) decoding the video picture to provide a decoded representation of at least one of the multiple pictures; and

means for post-processing the decoded representation using the accessed information.

6. The apparatus of claim 5, wherein the multiple pictures includes a first picture from a first view, and a second picture including depth information for the first picture.

7. The apparatus of claim 5, wherein at least one of the multiple pictures represents depth information.

20. The apparatus of claim 5, further comprising:

an interface to receive user input selecting at least one of the multiple pictures for display; and

means for providing a decoded output of the at least one selected picture, the decoded output being provided based on the accessed information, the decoded representation, and the user input.

25. The apparatus of claim 5 wherein the accessed information indicates that the multiple pictures are combined into the single picture in a manner that provides increased spatial correlation between corresponding portions of the multiple pictures, thereby allowing coding gains in coding the single picture.

26. The apparatus of claim 5 wherein post-processing the decoded representation comprises flipping the decoded representation, if the accessed information indicates that the at least one of the multiple pictures is flipped.

27. The apparatus of claim 5 wherein:

at least one of the multiple pictures is flipped in the horizontal direction in the single picture, and

the accessed information indicates that the flipping is in the horizontal direction.

28. The apparatus of claim 5 wherein:

at least one of the multiple pictures is flipped in the vertical direction in the single picture, and

the accessed information indicates that the flipping is in the vertical direction.

8. A method, comprising:

generating a video picture that includes multiple pictures combined into a single picture, the multiple pictures including a picture of a first view and a picture of a second view in which a common object or region appears shifted from one view to the other;

generating information indicating how the multiple pictures in the generated video picture are combined, wherein the generated information indicates that at least one of the multiple pictures is individually flipped in one or more of a horizontal direction or a vertical direction such that the common object or region from the first view picture is juxtaposed with the common object or region from the second view picture;

encoding the generated video picture and the generated information; and

providing a video stream that includes the encoded video picture and the encoded information.

9. The method of claim 8, wherein the generating of the video picture, the generating of the information, the encoding, and the providing of the video stream are performed at an encoder.

10. The method of claim 8, wherein at least one of the multiple pictures represents depth information.

33. The method of claim 8 wherein the generated information indicates that the multiple pictures are combined into the single picture in a manner that provides increased spatial correlation between corresponding portions of the multiple pictures, thereby allowing coding gains in coding the single picture.

34. The method of claim 8 wherein:

at least one of the multiple pictures is flipped in the horizontal direction in the single picture, and

the generated information indicates that the flipping is in the horizontal direction.

35. The method of claim 8 wherein:

at least one of the multiple pictures is flipped in the vertical direction in the single picture, and

the generated information indicates that the flipping is in the vertical direction.

11. An apparatus, comprising an encoder for:

generating a video picture that includes multiple pictures combined into a single picture, the multiple pictures including a picture of a first view and a picture of a second view in which a common object or region appears shifted from one view to the other,

generating information indicating how the multiple pictures in the generated video picture are combined, wherein the generated information indicates that at least one of the multiple pictures is individually flipped in one or more of a horizontal direction or a vertical direction such that the common object or region from the first view picture is juxtaposed with the common object or region from the second view picture,

encoding the generated video picture and the generated information, and

providing a video stream that includes the encoded video picture and the encoded information.

36. The apparatus of claim 11 wherein the generated information indicates that the multiple pictures are combined into the single picture in a manner that provides increased spatial correlation between corresponding portions of the multiple pictures, thereby allowing coding gains in coding the single picture.

37. The apparatus of claim 11 wherein:

at least one of the multiple pictures is flipped in the horizontal direction in the single picture, and

the generated information indicates that the flipping is in the horizontal direction.

38. The apparatus of claim 11 wherein:

at least one of the multiple pictures is flipped in the vertical direction in the single picture, and

the generated information indicates that the flipping is in the vertical direction.

12. An apparatus, comprising:

means for accessing a video picture that includes multiple pictures combined into a single picture, the multiple pictures including a picture of a first view and a picture of a second view in which a common object or region appears shifted from one view to the other, the video picture being part of a received video stream;

means for accessing information indicating how the multiple pictures in the accessed video picture are combined, wherein the accessed information indicates whether at least one of the multiple pictures is individually flipped in one or more of a horizontal direction or a vertical direction such that the common object or region from the first view picture is juxtaposed with the common object or region from the second view picture, and wherein the accessed information is part of the received video stream;

means for decoding the video picture to provide a decoded representation of at least one of the multiple pictures; and

means for post-processing the decoded representation using the accessed information.

13. An apparatus, comprising:

means for generating a video picture that includes multiple pictures combined into a single picture, the multiple pictures including a picture of a first view and a picture of a second view in which a common object or region appears shifted from one view to the other;

means for generating information indicating how the multiple pictures in the generated video picture are combined, wherein the generated information indicates that at least one of the multiple pictures is individually flipped in one or more of a horizontal direction or a vertical direction such that the common object or region from the first view picture is juxtaposed with the common object or region from the second view picture;

means for encoding the generated video picture and the generated information; and

means for providing a video stream that includes the encoded video picture and the encoded information.

14. A non-transitory computer readable medium having a video signal structure comprising:

an encoded representation of a video picture that includes multiple pictures combined into a single picture, the multiple pictures including a picture of a first view and a picture of a second view in which a common object or region appears shifted from one view to the other; and

an encoded representation of information indicating how the multiple pictures included in the single video picture are combined, wherein the generated information indicates that at least one of the multiple pictures is individually flipped in one or more of a horizontal direction or a vertical direction such that the common object or region from the first view picture is juxtaposed with the common object or region from the second view picture.

15. A non-transitory computer readable medium having computer readable program code embodied thereon, the computer readable program code comprising:

program code for accessing a video picture that includes multiple pictures combined into a single picture, the multiple pictures including a picture of a first view and a picture of a second view in which a common object or region appears shifted from one view to the other, the video picture being part of a received video stream;

program code for accessing information indicating how the multiple pictures in the accessed video picture are combined, wherein the accessed information indicates whether at least one of the multiple pictures is individually flipped in one or more of a horizontal direction or a vertical direction such that the common object or region from the first view picture is juxtaposed with the common object or region from the second view picture, and wherein the accessed information is part of the received video stream;

program code for decoding the video picture to provide a decoded representation of at least one of the multiple pictures; and

program code for post-processing the decoded representation using the accessed information.

29. The medium of claim 15 wherein the accessed information indicates that the multiple pictures are combined into the single picture in a manner that provides increased spatial correlation between corresponding portions of the multiple pictures, thereby allowing coding gains in coding the single picture.

30. The medium of claim 15 wherein post-processing the decoded representation comprises flipping the decoded representation, if the accessed information indicates that the at least one of the multiple pictures is flipped.

31. The medium of claim 15 wherein:

at least one of the multiple pictures is flipped in the horizontal direction in the single picture, and

the accessed information indicates that the flipping is in the horizontal direction.

32. The medium of claim 15 wherein:

at least one of the multiple pictures is flipped in the vertical direction in the single picture, and

the accessed information indicates that the flipping is in the vertical direction.

16. A non-transitory computer readable medium having computer readable program code embodied thereon, the computer readable program code comprising:

program code for generating a video picture that includes multiple pictures combined into a single picture, the multiple pictures including a picture of a first view and a picture of a second view in which a common object or region appears shifted from one view to the other;

program code for generating information indicating how the multiple pictures in the generated video picture are combined, wherein the generated information indicates that at least one of the multiple pictures is individually flipped in one or more of a horizontal direction or a vertical direction such that the common object or region from the first view picture is juxtaposed with the common object or region from the second view picture;

program code for encoding the generated video picture and the generated information; and

program code for providing a video stream that includes the encoded video picture and the encoded information.

39. The medium of claim 16 wherein the generated information indicates that the multiple pictures are combined into the single picture in a manner that provides increased spatial correlation between corresponding portions of the multiple pictures, thereby allowing coding gains in coding the single picture.

40. The medium of claim 16 wherein:

at least one of the multiple pictures is flipped in the horizontal direction in the single picture, and

the generated information indicates that the flipping is in the horizontal direction.

41. The medium of claim 16 wherein:

at least one of the multiple pictures is flipped in the vertical direction in the single picture, and

the generated information indicates that the flipping is in the vertical direction.

17. An apparatus comprising a processor configured to perform at least the following:

accessing a video picture that includes multiple pictures combined into a single picture, the multiple pictures including a picture of a first view and a picture of a second view in which a common object or region appears shifted from one view to the other, the video picture being part of a received video stream;

accessing information indicating how the multiple pictures in the accessed video picture are combined, wherein the accessed information indicates whether at least one of the multiple pictures is individually flipped in one or more of a horizontal direction or a vertical direction such that the common object or region from the first view picture is juxtaposed with the common object or region from the second view picture, and wherein the accessed information is part of the received video stream;

decoding the video picture to provide a decoded representation of at least one of the multiple pictures; and

post-processing the decoded representation using the accessed information.

18. An apparatus comprising a processor configured to perform at least the following:

generating a video picture that includes multiple pictures combined into a single picture, the multiple pictures including a picture of a first view and a picture of a second view in which a common object or region appears shifted from one view to the other;

generating information indicating how the multiple pictures in the generated video picture are combined, wherein the generated information indicates that at least one of the multiple pictures is individually flipped in one or more of a horizontal direction or a vertical direction such that the common object or region from the first view picture is juxtaposed with the common object or region from the second view picture;

encoding the generated video picture and the generated information; and

providing a video stream that includes the encoded video picture and the encoded information.