Overlay of visual representations of captions on video

• US 9,307,292 B2
• Filed: 01/03/2014
• Issued: 04/05/2016
• Est. Priority Date: 06/05/2012
• Status: Active Grant

• Current Assignee: Hulu LLC (The Walt Disney Company)
• Original Assignee: Hulu LLC (The Walt Disney Company)
• Inventors: Coudurier, Baptiste, Post, Robert A.
• Primary Examiner(s): Dang, Hung
• Application Number:
  US14/147,418
• Publication Number:
  US 20140123193A1
• Time in Patent Office: 823 Days
• Field of Search: 386239-248
• US Class Current: 1/1
• CPC Class Codes: H04N 21/47 End-user applications
• CPC Class Codes: H04N 21/4722 for requesting additional data associated with the content
• CPC Class Codes: H04N 21/482 End-user interface for program selection broadcast systems using EPGs H04H60/72
• CPC Class Codes: H04N 21/4884 for displaying subtitles
• CPC Class Codes: H04N 21/631 Multimode Transmission, e.g. transmitting basic layers and enhancement layers of the content over different transmission paths or transmitting with different error corrections, different keys or with different transmission protocols
• CPC Class Codes: H04N 21/6581 Reference data, e.g. a movie identifier for ordering a movie or a product identifier in a home shopping application
• CPC Class Codes: H04N 21/6582 Data stored in the client, e.g. viewing habits, hardware capabilities, credit card number arrangements where receivers interact with the broadcast H04H20/38

Abstract

In one embodiment, a method receiving a request for a media program from a user device. The method then determines a set of visual representations of captions for the media program and determines video information for the media program. Visual representations from the set of visual representations of captions are sent with the video information over a network to the user device where text for the captions has been pre-rendered in the sent visual representations before sending of the visual representations to the user device. Also, the user device is configured to directly render and overlay a visual representation of a caption from the visual representations over a portion of the video information without rendering of the text for caption on the portion of the video information at the user device.

Claims

1. A method comprising:

storing, by a computing device, a set of visual representations of captions for a media program and video information for the media program, wherein text for the captions is rendered in the stored set of visual representations;

receiving, by the computing device, a plurality of first requests for segments of the video information for the media program from a user device, wherein the user device uses a video playlist to determine the segments of video information to request;

receiving, by the computing device, a plurality of second requests from visual representations of captions corresponding to the segments of the video information requested from the user device, wherein the user device uses a visual representation playlist to determine the visual representations of captions to request, wherein the video playlist is separate from the visual representation playlist; and

in response to the plurality of first requests and the plurality of second requests, sending, by the computing device, the visual representations of captions from the set of visual representations of captions with the segments of the video information over a network to the user device, wherein text for the captions has been pre-rendered in the sent visual representations of captions before sending of the visual representations of captions to the user device and the user device is configured to directly render and overlay the visual representations of captions over at least a portion of segments of video information without rendering of the text for the captions on the segments of the video information at the user device.

2. The method of claim 1, further comprising generating, by the computing device, the set of visual representations of captions by pre-rendering the text for the captions into the visual representations of captions before sending of the visual representations of captions from the set of visual representations of captions to the user device.

3. The method of claim 1, wherein sending the visual representations of captions from the set of visual representations of captions with the video information comprises sending, by the computing device, the visual representations of captions from the set of visual representations of captions out of band with the segments of video information to the user device.

4. The method of claim 1, wherein the visual representations of captions that are displayed on a plurality of user devices that include renderers that render text differently are the same.

5. The method of claim 1, wherein the segments of video information are not multiplexed with the set of visual representations of captions.

6. The method of claim 1, wherein sending comprises sending, by the computing device, the visual representations of captions from the set of visual representations of captions in a separate channel from the segments of video information.

7. The method of claim 1, further comprising:

receiving, by the computing device, requests for the visual representations of captions and the segments of video information; and

sending, by the computing device, the requested visual representations of captions and the segments of video information when requested.

8. The method of claim 1, wherein:

the set of visual representations of captions are stored consecutively in a first file;

the segments of video information are stored consecutively in a second file; and

sending comprises sending, by the computing device, the visual representations of captions from the set of visual representations of captions in the first file and sending the segments of video information from the second file.

9. The method of claim 1, wherein a visual representation of a caption is rendered at a specified position over a segment of video information by the user device.

10. A non-transitory computer-readable storage medium containing instructions, that when executed, control a computer system to be configured for:

storing a set of visual representations of captions for a media program and video information for the media program, wherein text for the captions is rendered in the stored set of visual representations;

receiving a plurality of first requests for segments of the video information for the media program from a user device, wherein the user device uses a video playlist to determine the segments of video information to request;

receiving a plurality of second requests from visual representations of captions corresponding to the segments of the video information requested from the user device, wherein the user device uses a visual representation playlist to determine the visual representations of captions to request, wherein the video playlist is separate from the visual representation playlist; and

in response to the plurality of first requests and the plurality of second requests, sending the visual representations of captions from the set of visual representations of captions with the segments of the video information over a network to the user device, wherein text for the captions has been pre-rendered in the sent visual representations of captions before sending of the visual representations of captions to the user device and the user device is configured to directly render and overlay the visual representations of captions over at least a portion of corresponding segments of video information without rendering of the text for the captions on the segments of the video information at the user device.

11. The non-transitory computer-readable storage medium of claim 10, further configured for generating the set of visual representations of captions by pre-rendering the text for the captions into the visual representations of captions before sending of the visual representations of captions from the set of visual representations of captions to the user device.

12. The non-transitory computer-readable storage medium of claim 10, wherein sending the visual representations of captions from the set of visual representations of captions with the video information comprises sending the visual representations of captions from the set of visual representations of captions out of band with the segments of video information to the user device.

13. The non-transitory computer-readable storage medium of claim 10, wherein the visual representations of captions that are displayed on a plurality of user devices that include renderers that render text differently are the same.

14. The non-transitory computer-readable storage medium of claim 10, wherein the segments of video information are not multiplexed with the set of visual representations of captions.

15. The non-transitory computer-readable storage medium of claim 10, wherein sending comprises sending the visual representations of captions from the set of visual representations of captions in a separate channel from the segments of video information.

16. The non-transitory computer-readable storage medium of claim 10, further configured for:

receiving requests for the visual representations of captions and the segments of video information; and

sending the requested visual representations of captions and the segments of video information when requested.

17. The non-transitory computer-readable storage medium of claim 10, wherein:

the set of visual representations of captions are stored consecutively in a first file;

the segments of video information are stored consecutively in a second file; and

sending comprises sending the visual representations of captions from the set of visual representations of captions in the first file and sending the segments of video information from the second file.

18. The non-transitory computer-readable storage medium of claim 10, wherein a visual representation of a caption is rendered at a specified position over a segment of video information by the user device.

19. A method comprising:

sending, by a first user device, a request for a media program to a server, the media program associated with a set of visual representations of captions;

receiving, by the first user device, visual representations of captions in the set of visual representations of captions with segments of video information over a network from the server;

determining, by the first user device, a visual representation of a caption to render from the received visual representations using a visual representation playlist;

determining, by the first user device, a segment of video information based on a video playlist, wherein the video playlist is separate from the visual representation playlist; and

directly rendering and overlaying, by the first user device, the visual representation of the caption over at least a portion of the segment of video information without rendering of the text for the caption on the segment of video information at the user device, wherein text for the captions has been pre-rendered in the received visual representations of captions before receiving of the visual representations of captions at the user device.

20. The method of claim 19, further comprising determining a location on the segment of video information to render the visual representation of the caption, wherein the visual representation of the caption is overlaid at the location.

21. The method of claim 19, wherein the visual representation of the caption that is displayed on the first user device and a second user device is the same.