Method and apparatus for generating and encoding projection-based frame with 360-degree content represented in projection faces packed in segmented sphere projection layout

• US 10,593,012 B2
• Filed: 03/20/2018
• Issued: 03/17/2020
• Est. Priority Date: 03/22/2017
• Status: Active Grant

• Current Assignee: MediaTek, Inc.
• Original Assignee: MediaTek, Inc.
• Inventors: Lee, Ya-Hsuan, Lin, Hung-Chih, Lin, Jian-Liang, Chang, Shen-Kai
• Primary Examiner(s): Wang, Yi
• Application Number:
  US15/927,028
• Publication Number:
  US 20180276788A1
• Time in Patent Office: 728 Days
• Field of Search: None
• CPC Class Codes: G06T 2219/2016 Rotation, translation, scaling
• CPC Class Codes: G06T 3/047 Fisheye or wide-angle transformations
• CPC Class Codes: G06T 3/12 Panospheric to cylindrical image transformations

Abstract

A video processing method includes receiving an omnidirectional content corresponding to a sphere, generating a projection-based frame according to at least the omnidirectional content and a segmented sphere projection (SSP) format, and encoding, by a video encoder, the projection-based frame to generate a part of a bitstream. The projection-based frame has a 360-degree content represented by a first circular projection face, a second circular projection face, and at least one rectangular projection face packed in an SSP layout. A north polar region of the sphere is mapped onto the first circular projection face. A south polar region of the sphere is mapped onto the second circular projection face. At least one non-polar ring-shaped segment between the north polar region and the south polar region of the sphere is mapped onto said at least one rectangular projection face.

Claims

1. A video processing method comprising:

receiving an omnidirectional content corresponding to a sphere;

generating at least one padding region;

generating a projection-based frame according to at least the omnidirectional content and a segmented sphere projection (SSP) format, wherein the projection-based frame has a 360-degree content represented by a first circular projection face, a second circular projection face, and at least one rectangular projection face packed in an SSP layout, a north polar region of the sphere is mapped onto the first circular projection face, a south polar region of the sphere is mapped onto the second circular projection face, and at least one non-polar ring-shaped segment between the north polar region and the south polar region of the sphere is mapped onto said at least one rectangular projection face; and

encoding, by a video encoder, the projection-based frame to generate a part of a bitstream;

wherein the SSP layout further comprises said at least one padding region, said at least one padding region comprises a first padding region, the first padding region connects with one boundary of a first projection face that is one of the first circular projection face, the second circular projection face and said at least one rectangular projection face; and

wherein generating said at least one padding region comprises;

deriving pixel values of pixels included in the first padding region from duplication of pixel values of pixels included in the first projection face.

2. The video processing method of claim 1, wherein a lowest latitude of the north polar region is +45°

, and a highest latitude of the south polar region is −

45°

.

3. The video processing method of claim 1, wherein the north polar region of the sphere is mapped onto the first circular projection face via equal-angular projection, the south polar region of the sphere is mapped onto the second circular projection face via equal-angular projection, and said at least one non-polar ring-shaped segment of the sphere is mapped onto said at least one rectangular projection face via equal-area projection.

4. The video processing method of claim 1, further comprising:

applying content rotation to the omnidirectional content of the sphere;

wherein generating the projection-based frame according to the omnidirectional content and the SSP format is performed after the omnidirectional content of the sphere is rotated by the content rotation.

5. The video processing method of claim 1, wherein when the first circular projection face and the second circular projection face are packed in the SSP layout, a space is left directly between the first circular projection face and the second circular projection face, such that the first circular projection face does not connect with the second circular projection face.

6. The video processing method of claim 1, wherein when said at least one rectangular projection face and one of the first circular projection face and the second circular projection face are packed in the SSP layout, a space is left directly between said at least one rectangular projection face and said one of the first circular projection face and the second circular projection face, such that said at least one rectangular projection face does not connect with said one of the first circular projection face and the second circular projection face.

7. The video processing method of claim 1, wherein the first projection face is one of said at least one rectangular projection face, the first padding region does not connect with another side of the first projection face, said one side of the first projection face and said another side of the first projection face are opposite sides of the first projection face, the first projection face comprises a partial region, the partial region comprises boundary pixels at said another side of the first projection face, the first padding region is generated by duplicating the partial region, and there is an image content continuity boundary between the first padding region and the first projection face packed in the SSP layout.

8. The video processing method of claim 1, wherein said at least one padding region further comprises a second padding region, the second padding region connects with one boundary of a second projection face that is another of the first circular projection face, the second circular projection face and said at least one rectangular projection face, and the first padding region and the second padding region are packed in the SSP layout without overlapping.

9. The video processing method of claim 1, wherein said at least one padding region further comprises a second padding region, the second padding region connects with one boundary of a second projection face that is another of the first circular projection face, the second circular projection face and said at least one rectangular projection face, and the first padding region and the second padding region are packed in the SSP layout with overlapping.

10. The video processing method of claim 9, wherein a pixel value of a pixel included in an overlapping region resulting from packing of the first padding region and the second padding region is directly set by a pixel value of a padding pixel included in a padding region selected from the first padding region and the second padding region.

11. The video processing method of claim 10, wherein when a pixel included in the overlapping region is nearer to a non-overlapping region of the first padding region than a non-overlapping region of the second padding region, the pixel value of the pixel included in the overlapping region is directly set by a pixel value of a co-located padding pixel included in the first padding region;

and when the pixel included in the overlapping region is nearer to the non-overlapping region of the second padding region than the non-overlapping region of the first padding region, the pixel value of the pixel included in the overlapping region is directly set by a pixel value of a co-located padding pixel included in the second padding region.

12. The video processing method of claim 10, wherein pixel values of all pixels included in the overlapping region are set by pixel values of co-located padding pixels included in a same padding region.

13. The video processing method of claim 9, wherein a pixel value of a pixel included in an overlapping region resulting from packing of the first padding region and the second padding region is set by blending pixel values of different pixels.

14. The video processing method of claim 13, wherein the different pixels comprise a pixel of one of the first projection face and the second projection face.

15. The video processing method of claim 13, wherein the different pixels comprise a pixel of one of the first padding region and the second padding region.

16. The video processing method of claim 1, wherein said at least one rectangular projection face comprises a first rectangular projection face;

and the first circular projection face and the second circular projection face are both located on a same side of the first rectangular projection face.

17. The video processing method of claim 1, wherein said at least one rectangular projection face comprises a first rectangular projection face;

and the first circular projection face and the second circular projection face are located on opposite sides of the first rectangular projection face, respectively.

18. A video processing apparatus comprising:

a conversion circuit, arranged to receive an omnidirectional content corresponding to a sphere, and generate a projection-based frame according to the omnidirectional content and a segmented sphere projection (SSP) format, wherein the projection-based frame has a 360-degree content represented by a first circular projection face, a second circular projection face, and at least one rectangular projection face packed in an SSP layout, a north polar region of the sphere is mapped onto the first circular projection face, a south polar region of the sphere is mapped onto the second circular projection face, at least one non-polar ring-shaped segment between the north polar region and the south polar region of the sphere is mapped onto said at least one rectangular projection face, and the conversion circuit comprises;

a padding circuit, arranged to generate at least one padding region; and

a video encoder, arranged to encode the projection-based frame to generate a part of a bitstream;

wherein the SSP layout further comprises said at least one padding region, said at least one padding region comprises a first padding region, the first padding region connects with one boundary of a first projection face that is one of the first circular projection face, the second circular projection face and said at least one rectangular projection face; and

wherein the padding circuit is arranged to derive pixel values of pixels included in the first padding region from duplication of pixel values of pixels included in the first projection face.