Method of providing fixed region information or offset region information for subtitle in virtual reality system and device for controlling the same

• US 10,375,375 B2
• Filed: 08/18/2017
• Issued: 08/06/2019
• Est. Priority Date: 05/15/2017
• Status: Active Grant

• Current Assignee: LG Electronics, Inc. (LG Corporation)
• Original Assignee: LG Electronics, Inc. (LG Corporation)
• Inventors: Hwang, Soojin, Oh, Sejin, Lee, Tung Chin
• Primary Examiner(s): Elahi, Shan E
• Application Number:
  US15/680,901
• Publication Number:
  US 20180332265A1
• Time in Patent Office: 718 Days
• Field of Search: 348 36
• CPC Class Codes: H04N 13/128 Adjusting depth or disparity
• CPC Class Codes: H04N 13/156 Mixing image signals
• CPC Class Codes: H04N 13/161 Encoding, multiplexing or demultiplexing different image signal components for multi-view video sequence encoding H04N19/597
• CPC Class Codes: H04N 13/178 Metadata, e.g. disparity information
• CPC Class Codes: H04N 13/183 On-screen display [OSD] information, e.g. subtitles or menus
• CPC Class Codes: H04N 13/194 Transmission of image signals
• CPC Class Codes: H04N 19/70 characterised by syntax aspects related to video coding, e.g. related to compression standards
• CPC Class Codes: H04N 19/88 involving rearrangement of data among different coding units, e.g. shuffling, interleaving, scrambling or permutation of pixel data or permutation of transform coefficient data among different blocks
• CPC Class Codes: H04N 21/4884 for displaying subtitles
• CPC Class Codes: H04N 21/816 involving special video data, e.g 3D video
• CPC Class Codes: H04N 21/8451 using Advanced Video Coding [AVC]

Abstract

A method and apparatus of/for providing subtitles for a 360-degree content, the method including generating 360-degree video data captured by at least one camera; stitching the 360-degree video data; projecting the 360-degree video data to a 2D image; encoding the 2D image into a video stream; generating a 360-degree subtitle Supplemental Enhancement Information (SEI) message for signaling a subtitle for the 360-degree content; inserting the 360-degree subtitle SEI message into the video stream; and generating a signal including the video stream, and transmitting the signal.

Claims

1. A method of providing subtitles for 360-degree content, the method comprising:

generating 360-degree video data captured by at least one camera;

stitching the 360-degree video data;

projecting the 360-degree video data to a 2D image;

encoding the 2D image into a video stream;

generating a 360-degree subtitle Supplemental Enhancement Information (SEI) message for signaling a subtitle for the 360-degree content;

inserting the 360-degree subtitle SEI message into the video stream, the 360-degree subtitle SEI message including a subtitle ID for identifying the subtitle, a region ID for identifying a subtitle region where the subtitle is overlaid in a 360-degree space, and subtitle region information being fixed region information for specifying the subtitle region fixed in the 360-degree space or offset region information for specifying the subtitle region changing in the 360-degree space based on a viewport; and

generating a signal including the video stream, and transmitting the signal.

2. The method of claim 1, wherein the fixed region information includes first information indicating a top-left point of the subtitle region, and second information indicating a bottom-right point of the subtitle region in the 360-degree space.

3. The method of claim 1, wherein the fixed region information includes first information indicating a start point being an angular point of the subtitle region, second information indicating an end point which is a different angular point of the subtitle region diagonally located from the angular point, and third information indicating a middle point for specifying a location of the subtitle region in the 360-degree space.

4. The method of claim 1, wherein the fixed region information includes center information indicating yaw, pitch and roll values of a center point of the subtitle region, and range information indicating horizontal and vertical ranges of the subtitle region from the center point.

5. The method of claim 1, wherein the offset region information includes offset center information indicating yaw, pitch and roll offsets for a center point of the subtitle region, range information indicating horizontal and vertical ranges of the subtitle region from the center point, and type information for indicating a type of the offset center information.

6. The method of claim 5, when the type information indicates a first type, the offset center information indicates yaw, pitch and roll offsets between the center point of the subtitle region and a center point of the viewport currently being used.

7. The method of claim 5, when the type information indicates a second type, the offset center information indicates yaw, pitch and roll offsets between the center point of the subtitle region and a center point of a previous subtitle region specified by a previous 360-degree subtitle SEI message.

8. The method of claim 1, wherein the 360-degree subtitle SEI message further includes number information for indicating the number of multiple subtitle regions where the same subtitle identified by the subtitle ID is overlayed, and multiple sets of region IDs and subtitle region information for identifying and specifying the multiple subtitle regions in the 360-degree space.

9. An apparatus for providing subtitles for 360-degree content, the apparatus comprising:

a processor configured to generate 360-degree video data captured by at least one camera;

a stitcher configured to stitch the 360-degree video data;

a projection processor configured to project the 360-degree video data to a 2D image;

a data encoder configured to encode the 2D image into a video stream;

a metadata processor configured to generate a 360-degree subtitle Supplemental Enhancement Information (SEI) message for signaling a subtitle for the 360-degree content, wherein the data encoder is further configured to insert the 360-degree subtitle SEI message into the video stream, the 360-degree subtitle SEI message including a subtitle ID for identifying the subtitle, a region ID for identifying a subtitle region where the subtitle is overlaid in a 360-degree space, and subtitle region information being fixed region information for specifying the subtitle region fixed in the 360-degree space or offset region information for specifying the subtitle region changing in the 360-degree space based on a viewport;

a transport processor configured to generate a signal including the video stream; and

a transmitter configured to transmit the signal.

10. The apparatus of claim 9, wherein the fixed region information includes first information indicating a top-left point of the subtitle region, and second information indicating a bottom-right point of the subtitle region in the 360-degree space.

11. The apparatus of claim 9, wherein the fixed region information includes first information indicating a start point being an angular point of the subtitle region, second information indicating an end point which is a different angular point of the subtitle region diagonally located from the angular point, and third information indicating a middle point for specifying a location of the subtitle region in the 360-degree space.

12. The apparatus of claim 9, wherein the fixed region information includes center information indicating yaw, pitch and roll values of a center point of the subtitle region, and range information indicating horizontal and vertical ranges of the subtitle region from the center point.

13. The apparatus of claim 9, wherein the offset region information includes offset center information indicating yaw, pitch and roll offsets for a center point of the subtitle region, range information indicating horizontal and vertical ranges of the subtitle region from the center point, and type information for indicating a type of the offset center information.

14. The apparatus of claim 13, when the type information indicates a first type, the offset center information indicates yaw, pitch and roll offsets between the center point of the subtitle region and a center point of the viewport currently being used.

15. The apparatus of claim 13, when the type information indicates a second type, the offset center information indicates yaw, pitch and roll offsets between the center point of the subtitle region and a center point of a previous subtitle region specified by a previous 360-degree subtitle SEI message.

16. The apparatus of claim 9, wherein the 360-degree subtitle SEI message further includes number information for indicating the number of multiple subtitle regions where the same subtitle identified by the subtitle ID is overlayed, and multiple sets of region IDs and subtitle region information for identifying and specifying the multiple subtitle regions in the 360-degree space.