Subtitle displaying method and apparatus

• US 10,645,332 B2
• Filed: 06/14/2019
• Issued: 05/05/2020
• Est. Priority Date: 06/20/2018
• Status: Active Grant

• Current Assignee: Alibaba Group Holding Ltd.
• Original Assignee: Alibaba Group Holding Ltd.
• Inventors: Zhang, Lei
• Primary Examiner(s): Natnael, Paulos M
• Application Number:
  US16/442,329
• Publication Number:
  US 20190394419A1
• Time in Patent Office: 326 Days
• Field of Search: 348468
• CPC Class Codes: G06V 10/25 Determination of region of interest [ROI] or a volume of interest [VOI]
• CPC Class Codes: G06V 20/40 in video content extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44
• CPC Class Codes: G06V 20/46 Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: H04N 21/4316 for displaying supplemental content in a region of the screen, e.g. an advertisement in a separate window
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/47 End-user applications
• CPC Class Codes: H04N 21/4884 for displaying subtitles
• CPC Class Codes: H04N 5/445 for displaying additional information H04N5/50 takes precedence

Abstract

A subtitle displaying method including determining subtitle information of a video frame in a video when a play request for the video is received; identifying a key region in the video frame; determining a subtitle display region in a region in the video frame other than the key region; and displaying subtitle content in the subtitle information in the subtitle display region during playing the video. In the example embodiments of the present disclosure, the subtitle display region is determined to be placed in the region other than the key region, such that display content in the key region is prevented from being blocked by subtitles, thus improving viewing experience of viewers.

Claims

1. A method comprising:

determining subtitle information of a video frame in a video, the subtitle information including a subtitle content;

identifying a key region in the video frame, wherein identifying the key region in the video frame comprises;

traversing the video frame with a window, an area of the window being less than that of the video frame;

determining a region where the window is located as a candidate region;

extracting features in the candidate region;

determining whether the candidate region comprises a target object according to the extracted features; and

determining the candidate region as the key region in response to determining that the candidate region comprises the target object;

determining a subtitle display region in a region in the video frame other than the key region; and

displaying the subtitle content in the subtitle display region.

2. The method of claim 1, wherein the identifying the key region in the video frame further comprises:

identifying the key region in the video frame by using a deep learning algorithm.

3. The method of claim 1, wherein the identifying the key region in the video frame further comprises determining the target object in the video frame by using a deep learning algorithm.

4. The method of claim 1, wherein the identifying the key region in the video frame further comprises:

determining a key part of the target object by using a deep learning algorithm; and

determining a display region where the key part is located as the key region.

5. The method of claim 4, wherein the target object comprises a face.

6. The method of claim 5, wherein the determining the key part of the target object in the video frame by using the deep learning algorithm comprises:

detecting the video frame by using a face recognition algorithm to obtain the face.

7. The method of claim 1, wherein the identifying the key region in the video frame further comprises:

determining that the video frame is a close-shot picture of a video frame previous or preceding to the video frame; and

identifying the key region in the video frame.

8. The method of claim 1, wherein the determining the subtitle display region in the region in the video frame other than the key region comprises:

determining the subtitle display region in the region in the video frame other than the key region according to the subtitle information.

9. The method of claim 8, wherein the subtitle information comprises an original subtitle display region in the video frame.

10. The method of claim 9, wherein the determining the subtitle display region in the region in the video frame other than the key region according to the subtitle information comprises:

determining that the original subtitle display region overlaps the key region; and

adjusting the original subtitle display region to the region in the video frame other than the key region.

11. The method of claim 10, wherein the adjusting the original subtitle display region to the region in the video frame other than the key region comprises:

dividing the original subtitle display region into at least two display sub-regions; and

adjusting the at least two display sub-regions to the region in the video frame other than the key region.

12. The method of claim 8, wherein the determining the subtitle display region in the region in the video frame other than the key region according to the subtitle information comprises:

determining the subtitle display region in the region in the video frame other than the key region according to the subtitle content in the subtitle information.

13. An apparatus comprising:

one or more processors; and

one or more memories stored thereon computer readable instructions that, when executed by the one or more processors, cause the one or more processors to perform acts comprising;

determining subtitle information of a video frame in a video, the subtitle information including a subtitle content;

identifying a key region in the video frame, wherein identifying the key region in the video frame comprises;

traversing the video frame with a window, an area of the window being less than that of the video frame;

determining a region where the window is located as a candidate region;

extracting features in the candidate region;

determining whether the candidate region comprises a target object according to the extracted features; and

determining the candidate region as the key region in response to determining that the candidate region comprises the target object

determining a subtitle display region in a region in the video frame other than the key region; and

displaying the subtitle content in the subtitle display region.

14. The apparatus of claim 13, wherein the determining the subtitle display region in the region in the video frame other than the key region comprises:

determining the subtitle display region in the region in the video frame other than the key region according to the subtitle information.

15. The apparatus of claim 14, wherein:

the subtitle information comprises an original subtitle display region in the video frame; and

the determining the subtitle display region in the region in the video frame other than the key region according to the subtitle information comprises;

determining that the original subtitle display region overlaps the key region; and

adjusting the original subtitle display region to the region in the video frame other than the key region.

16. The apparatus of claim 15, wherein the adjusting the original subtitle display region to the region in the video frame other than the key region comprises:

dividing the original subtitle display region into at least two display sub-regions; and

adjusting the at least two display sub-regions to the region in the video frame other than the key region.

17. One or more memories stored thereon computer readable instructions that, when executed by one or more processors, cause the one or more processors to perform acts comprising:

determining subtitle information of a video frame in a video, the subtitle information including a subtitle content and an original subtitle display region in the video frame;

identifying a key region in the video frame, wherein identifying the key region in the video frame comprises;

traversing the video frame with a window, an area of the window being less than that of the video frame;

determining a region where the window is located as a candidate region;

extracting features in the candidate region;

determining whether the candidate region comprises a target object according to the extracted features; and

determining the candidate region as the key region in response to determining that the candidate region comprises the target object;

determining that the original subtitle display region overlaps the key region; and

adjusting the original subtitle display region to the region in the video frame other than the key region; and

displaying the subtitle content in the subtitle display region.

18. The one or more memories of claim 17, wherein the determining the subtitle display region in the region in the video frame other than the key region comprises:

determining the subtitle display region in the region in the video frame other than the key region according to the subtitle information.

19. The one or more memories of claim 17, wherein the subtitle information comprises an original subtitle display region in the video frame.

20. The one or more memories of claim 17, wherein the adjusting the original subtitle display region to the region in the video frame other than the key region comprises:

dividing the original subtitle display region into at least two display sub-regions; and

adjusting the at least two display sub-regions to the region in the video frame other than the key region.