Method and apparatus for generating projection-based frame with 360-degree image content represented by triangular projection faces assembled in octahedron projection layout

• US 10,643,370 B2
• Filed: 09/30/2017
• Issued: 05/05/2020
• Est. Priority Date: 10/07/2016
• Status: Active Grant

• Current Assignee: MediaTek, Inc.
• Original Assignee: MediaTek, Inc.
• Inventors: Lin, Jian-Liang, Lin, Hung-Chih, Li, Chia-Ying, Chang, Shen-Kai, Ju, Chi-Cheng, Huang, Chao-Chih, Ouyang, Hui
• Primary Examiner(s): Yang, Andrew G
• Application Number:
  US15/769,750
• Publication Number:
  US 20190088001A1
• Time in Patent Office: 948 Days
• Field of Search: None
• CPC Class Codes: G06T 15/04 Texture mapping
• CPC Class Codes: G06T 15/08 Volume rendering
• CPC Class Codes: G06T 3/06 Topological mapping of higher dimensional structures onto lower dimensional surfaces
• CPC Class Codes: G06T 3/18 Image warping, e.g. rearranging pixels individually
• CPC Class Codes: H04N 13/00 Stereoscopic video systems; Multi-view video systems; Details thereof

Abstract

A projection-based frame is generated according to an omnidirectional video frame and an octahedron projection layout. The projection-based frame has a 360-degree image content represented by triangular projection faces assembled in the octahedron projection layout. A 360-degree image content of a viewing sphere is mapped onto the triangular projection faces via an octahedron projection of the viewing sphere. One side of a first triangular projection face has contact with one side of a second triangular projection face, one side of a third triangular projection face has contact with another side of the second triangular projection face. One image content continuity boundary exists between one side of the first triangular projection face and one side of the second triangular projection face, and another image content continuity boundary exists between one side of the third triangular projection face and another side of the second triangular projection face.

Claims

1. A method for generating a projection-based frame, comprising:

receiving an omnidirectional video frame corresponding to a viewing sphere; and

generating, by a conversion circuit, the projection-based frame according to the omnidirectional video frame and an octahedron projection layout, wherein the projection-based frame has a 360-degree image content represented by triangular projection faces assembled in the octahedron projection layout, and a 360-degree image content of the viewing sphere is mapped onto the triangular projection faces via an octahedron projection of the viewing sphere;

wherein the triangular projection faces assembled in the octahedron projection layout comprise a first triangular projection face, a second triangular projection face and a third triangular projection face, one side of the first triangular projection face has contact with one side of the second triangular projection face, one side of the third triangular projection face has contact with another side of the second triangular projection face, there is an image content continuity boundary between said one side of the first triangular projection face and said one side of the second triangular projection face, and there is an image content continuity boundary between said one side of the third triangular projection face and said another side of the second triangular projection face;

wherein the triangular projection faces of the octahedron projection layout are obtained from the octahedron projection of the viewing sphere according to an octahedron;

a boundary between one side of a first face of the octahedron and one side of a second face of the octahedron corresponds to the image content continuity boundary between said one side of the first triangular projection face and said one side of the second triangular projection face, where said one side of the first face of the octahedron connects with said one side of the second face of the octahedron; and

a boundary between one side of a third face of the octahedron and another side of the second face of the octahedron corresponds to the image content continuity boundary between said one side of the third triangular projection face and said another side of the second triangular projection face, where said one side of the third face of the octahedron connects with said another side of the second face of the octahedron.

2. The method of claim 1, wherein the triangular projection faces assembled in the octahedron projection layout further comprise a fourth triangular projection face, one side of the fourth triangular projection face has contact with yet another side of the second triangular projection face, and there is an image content continuity boundary between said one side of the fourth triangular projection face and said yet another side of the second triangular projection face.

3. The method of claim 1, wherein a shape of each of the triangular projection faces is an equilateral triangle.

4. The method of claim 3, wherein each of the triangular projection faces has a first side, a second side and a third side, and an equator of the viewing sphere is mapped along first sides of the triangular projection faces.

5. The method of claim 3, wherein an equator of the viewing sphere is not mapped along any side of each of the triangular projection faces.

6. The method of claim 3, wherein the viewing sphere is composed of a first hemisphere and a second hemisphere;

and the first triangular projection face, the second triangular projection face and the third triangular projection face are all derived from the first hemisphere via the octahedron projection of the viewing sphere.

7. The method of claim 6, wherein the triangular projection faces assembled in the octahedron projection layout further comprise a fourth projection face derived from the first hemisphere via the octahedron projection of the viewing sphere, one side of the fourth triangular projection face has contact with another side of the third triangular projection face, and there is an image content discontinuity boundary between said one side of the fourth triangular projection face and said another side of the third triangular projection face.

8. The method of claim 6, wherein a shape of the octahedron projection layout is a rectangle, the triangular projection faces assembled in the octahedron projection layout further comprise a fourth projection face derived from the first hemisphere via the octahedron projection of the viewing sphere, the fourth projection face is split into a first right-triangle-shaped part and a second right-triangle-shaped part, the first right-triangle-shaped part has a first side being one side of the fourth projection face and a second side being a first part of another side of the fourth projection face, the second right-triangle-shaped part has a first side being yet another side of the fourth projection face and a second side being a second part of said another side of the fourth projection face, the first side of the second right-triangle-shaped part has contact with another side of the first triangular projection face, the first side of the first right-triangle-shaped part has contact with another side of the third triangular projection face, there is an image content discontinuity boundary between the first side of the second right-triangle-shaped part and said another side of the first triangular projection face, and there is an image content discontinuity boundary between the first side of the first right-triangle-shaped part and said another side of the third triangular projection face.

9. The method of claim 6, wherein the triangular projection faces assembled in the octahedron projection layout further comprise a fourth projection face derived from the first hemisphere via the octahedron projection of the viewing sphere, a first padding area extended from one side of the fourth triangular projection face has contact with a second padding area extended from another side of the third triangular projection face, the first padding area comprises padding pixels derived from boundary pixels of said one side of the fourth triangular projection face, the second padding area comprises padding pixels derived from boundary pixels of said another side of the third triangular projection face, and there is an image content discontinuity boundary between the first padding area and the second padding area.

10. The method of claim 6, wherein a shape of the octahedron projection layout is a rectangle, the triangular projection faces assembled in the octahedron projection layout further comprise a fourth projection face derived from the first hemisphere via the octahedron projection of the viewing sphere, the fourth projection face is split into a first right-triangle-shaped part and a second right-triangle-shaped part, the first right-triangle-shaped part has a first side being one side of the fourth projection face and a second side being a first part of another side of the fourth projection face, the second right-triangle-shaped part has a first side being yet another side of the fourth projection face and a second side being a second part of said another side of the fourth projection face, a first padding area is extended from the first side of the second right-triangle-shaped part has contact with a second padding area that is extended from another side of the first triangular projection face, a third padding area is extended from the first side of the first right-triangle-shaped part and has contact with a fourth padding area that is extended from another side of the third triangular projection face, the first padding area comprises padding pixels derived from boundary pixels of the first side of the second right-triangle-shaped part, the second padding area comprises padding pixels derived from boundary pixels of said another side of the first triangular projection face, the third padding area comprises padding pixels derived from boundary pixels of the first side of the first right-triangle-shaped part, the fourth padding area comprises padding pixels derived from boundary pixels of said another side of the third triangular projection face, there is an image content discontinuity boundary between the first padding area and the second padding area, and there is an image content discontinuity boundary between the third padding area and the fourth padding area.

11. The method of claim 3, wherein the viewing sphere is composed of a first hemisphere and a second hemisphere;

the first triangular projection face and the second triangular projection face are both derived from the first hemisphere via the octahedron projection of the viewing sphere, and the third triangular projection face is derived from the second hemisphere via the octahedron projection of the viewing sphere.

12. The method of claim 1, wherein a shape of each of the triangular projection faces is an isosceles right triangle.

13. The method of claim 12, wherein each of the triangular projection faces has a first side, a second side and a third side, and an equator of the viewing sphere is mapped along first sides of the triangular projection faces.

14. The method of claim 12, wherein an equator of the viewing sphere is not mapped along any side of each of the triangular projection faces.

15. The method of claim 12, wherein the viewing sphere is composed of a first hemisphere and a second hemisphere;

and the first triangular projection face, the second triangular projection face and the third triangular projection face are all derived from the first hemisphere via the octahedron projection of the viewing sphere.

16. The method of claim 15, wherein a shape of the octahedron projection layout is a rectangle, the first triangular projection face is split into a first right-triangle-shaped part and a second right-triangle-shaped part that are assembled in the octahedron projection layout with different orientations respectively, the first right-triangle-shaped part has a first side being said one side of the first projection face and a second side being a first part of another side of the first projection face, the second right-triangle-shaped part has a first side being yet another side of the first projection face and a second side being a second part of said another side of the first projection face, the third triangular projection face is split into a third right-triangle-shaped part and a fourth right-triangle-shaped part that are assembled in the octahedron projection layout with different orientations respectively, the third right-triangle-shaped part has a first side being said one side of the third projection face and a second side being a first part of another side of the third projection face, and the fourth right-triangle-shaped part has a first side being yet another side of the third projection face and a second side being a second part of said another side of the third projection face.

17. The method of claim 15, wherein a shape of the octahedron projection layout is a rectangle, the triangular projection faces assembled in the octahedron projection layout further comprise a fourth triangular projection face derived from the first hemisphere via the octahedron projection of the viewing sphere, one side of the fourth triangular projection face has contact with another side of the first triangular projection face, another side of the fourth triangular projection face has contact with another side of the third triangular projection face, there is an image content continuity boundary between said one side of the fourth triangular projection face and said another side of the first triangular projection face, and there is an image content continuity boundary between said another side of the fourth triangular projection face and said another side of the third triangular projection face.

18. The method of claim 1, wherein at least one picture boundary resulting from filling of dummy areas in the projection-based frame or at least one image content discontinuity boundary resulting from assembling of the triangular projection faces in the octahedron projection layout has a jagged shape with each jag being even-pixel wide.

19. The method of claim 12, wherein the viewing sphere is composed of a first hemisphere and a second hemisphere;

the first triangular projection face and the second triangular projection face are both derived from the first hemisphere via the octahedron projection of the viewing sphere, and the third triangular projection face is derived from the second hemisphere via the octahedron projection of the viewing sphere.

20. A processing circuit for generating a projection-based frame, comprising:

an input interface, arranged to receive an omnidirectional video frame corresponding to a viewing sphere; and

a conversion circuit, arranged to generate the projection-based frame according to the omnidirectional video frame and an octahedron projection layout, wherein the projection-based frame has a 360-degree image content represented by triangular projection faces assembled in the octahedron projection layout, and a 360-degree image content of the viewing sphere is mapped onto the triangular projection faces via an octahedron projection of the viewing sphere;

wherein the triangular projection faces assembled in the octahedron projection layout comprise a first triangular projection face, a second triangular projection face and a third triangular projection face, one side of the first triangular projection face has contact with one side of the second triangular projection face, one side of the third triangular projection face has contact with another side of the second triangular projection face, there is an image content continuity boundary between said one side of the first triangular projection face and said one side of the second triangular projection face, and there is an image content continuity boundary between said one side of the third triangular projection face and said another side of the second triangular projection face;

wherein the triangular projection faces of the octahedron projection layout are obtained from the octahedron projection of the viewing sphere according to an octahedron;

a boundary between one side of a first face of the octahedron and one side of a second face of the octahedron corresponds to the image content continuity boundary between said one side of the first triangular projection face and said one side of the second triangular projection face, where said one side of the first face of the octahedron connects with said one side of the second face of the octahedron; and

a boundary between one side of a third face of the octahedron and another side of the second face of the octahedron corresponds to the image content continuity boundary between said one side of the third triangular projection face and said another side of the second triangular projection face, where said one side of the third face of the octahedron connects with said another side of the second face of the octahedron.