Preserving captioning through video transcoding

• US 9,426,479 B2
• Filed: 08/07/2008
• Issued: 08/23/2016
• Est. Priority Date: 08/09/2007
• Status: Active Grant

• Current Assignee: Triton US VP Acquisition, Co.
• Original Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Inventors: Shaffer, Gary
• Primary Examiner(s): Werner, David N
• Application Number:
  US12/672,434
• Publication Number:
  US 20110164673A1
• Time in Patent Office: 2,938 Days
• US Class Current: 1/1
• CPC Class Codes: H04N 19/40 using video transcoding, i.e. partial or full decoding of a coded input stream followed by re-encoding of the decoded output stream
• CPC Class Codes: H04N 21/233 Processing of audio elementary streams monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/2335 involving reformatting operations of audio signals, e.g. by converting from one coding standard to another details of audio signal transcoding G10L19/173
• CPC Class Codes: H04N 21/234 Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
• CPC Class Codes: H04N 21/2343 involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements video transcoding H04N19/40; media packet handling at the source H04L65/762
• CPC Class Codes: H04N 21/234309 by transcoding between formats or standards, e.g. from MPEG-2 to MPEG-4 or from Quicktime to Realvideo conversion of standards in analog television systems H04N7/01
• CPC Class Codes: H04N 21/235 Processing of additional data, e.g. scrambling of additional data or processing content descriptors
• CPC Class Codes: H04N 21/2355 involving reformatting operations of additional data, e.g. HTML pages optimising the visualization of content for information retrieval from the Internet G06F16/9577; tracking of instant messages H04L51/234; media packet handling at the source H04L65/762
• CPC Class Codes: H04N 21/236 Assembling of a multiplex stream, e.g. transport stream, by combining a video stream with other content or additional data, e.g. inserting a URL [Uniform Resource Locator] into a video stream, multiplexing software data into a video stream; Remultiplexing of multiplex streams; Insertion of stuffing bits into the multiplex stream, e.g. to obtain a constant bit-rate; Assembling of a packetised elementary stream multiplexing of data packets for data networks, e.g. RTP/UDP H04L65/00
• CPC Class Codes: H04N 21/2362 Generation or processing of Service Information [SI]
• CPC Class Codes: H04N 21/242 Synchronization processes, e.g. processing of PCR [Program Clock References] arrangements for synchronising broadcast or distribution via plural systems in broadcast distribution systems H04H20/18
• CPC Class Codes: H04N 21/440218 by transcoding between formats or standards, e.g. from MPEG-2 to MPEG-4 conversion of standards in analogue television systems H04N7/01
• CPC Class Codes: H04N 21/4884 for displaying subtitles
• CPC Class Codes: H04N 5/278 Subtitling
• CPC Class Codes: H04N 7/0885 for the transmission of subtitles

Abstract

Methods of preserving captioning information in an input video stream through transcoding of the input video stream include extracting caption data from the input video stream, translating the caption data into at least one output caption format, packaging the translated caption data into data packets for insertion into a video stream, synchronizing the packaged caption data with a transcoded version of the input video stream, receiving a preliminary output video stream that is a transcoded version of the input video stream, and combining the packaged caption data with the preliminary output video stream to form an output video stream. Related systems and computer program products are also disclosed.

Claims

1. A method, comprising:

parsing an input video stream to determine a presentation order of encoded frames in the input video stream;

if audio data is present, processing the audio data independently of the input video stream and caption data;

reordering the caption data into the presentation order, the caption data extracted from the input video stream;

assigning time stamps to the caption data;

translating the caption data from closed caption data into an output caption format, the output caption format being a different format than a format of the closed caption data;

providing a target bit rate based on an amount of the caption data to be inserted into a preliminary output video stream;

transcoding the input video stream, excluding the caption data, into the preliminary output video stream at the target bit rate, a frame rate of the input video stream being different from a frame rate of the preliminary output video stream;

reordering the caption data in the output caption format into a coded order of encoded frames in the preliminary output video stream, based on the time stamps of the caption data; and

writing the caption data in the output caption format to a same file as the preliminary output video stream.

2. The method of claim 1, further comprising:

generating a caption data file including caption data output to the preliminary output video stream.

3. The method of claim 1, further comprising:

detecting a caption data underflow or overflow in the caption data; and

compensating for the caption data underflow or overflow in the caption data by modifying the caption data.

4. The method of claim 3, wherein the compensating for the caption data underflow or overflow in the caption data comprises adding null data to the caption data in the case of the caption data underflow, and deleting null data in the caption data in the case of the caption data overflow.

5. The method of claim 1, further comprising:

correcting at least one of a text error and a coding error in the caption data, wherein the reordering is performed in response to the correcting.

6. The method of claim 1, further comprising:

converting the caption data into an Advanced Television Systems Committee (ATSC) format before translating the caption data from the ATSC format into at least one output caption format.

7. The method of claim 1, further comprising:

buffering the caption data before packaging the caption data into data packets to be inserted into the preliminary output video stream.

8. The method of claim 1, wherein the parsing the input video stream comprises the assigning the time stamps to the caption data.

9. The method of claim 1, further comprising:

parsing the preliminary output video stream to determine the coded order of the encoded frames in the preliminary output video stream.

10. The method of claim 1, further comprising:

inserting the caption data in the output caption format into spaces in the preliminary output video stream.

11. The method of claim 1, further comprising:

packaging the translated caption data into data packets; and

synchronizing the packaged caption data with the preliminary output video stream.

12. The method of claim 1, wherein, when the caption data is combined with the preliminary output video stream, a total bandwidth occupied by the preliminary output video stream does not exceed a maximum allowable bandwidth.

13. A system, comprising:

a caption input reorder queue configured to reorder caption data into a presentation order of encoded frames in an input video stream, the caption data extracted from the input video stream;

a caption extraction module configured to parse the input video stream to determine the presentation order of the encoded frames and to assign time stamps to the caption data;

a caption processing module configured to translate the caption data from closed caption data into an output caption format, the output caption format being a different format than a format of the closed caption data;

a caption packaging module configured to provide a target bit rate based on an amount of the caption data to be inserted into a preliminary output video stream;

a video processing module configured to transcode the input video stream, excluding the caption data, into the preliminary output video stream at the target bit rate, a frame rate of the input video stream being different from a frame rate of the preliminary output video stream, wherein, if audio data is present, the audio data is processed independently of the input video stream and the caption data, and the caption packaging module is configured to synchronize the caption data in the output caption format with the preliminary output video stream; and

a combining module configured to reorder the caption data in the output caption format into a coding order of encoded frames in the preliminary output video stream, based on the time stamps of the caption data, and to write the caption data in the output caption format to a same file as the preliminary output video stream.

14. The system of claim 13, wherein the caption packaging module is configured to retrieve caption data having one or more different caption formats for packaging.

15. The system of claim 13, wherein the caption extraction module is configured to compute an amount of the caption data to avoid a caption data overflow by discarding a null character of the caption data.

16. The system of claim 13, wherein the caption packaging module is configured to organize the caption data into caption data packets to be inserted into the preliminary output video stream.

17. The system of claim 13, wherein the combining module is configured to insert the caption data in the output caption format into the preliminary output video stream.

18. The system of claim 17, wherein the caption packaging module is configured to obtain frame ordering and/or timing information for the preliminary output video stream from the video processing module.

19. A system, comprising:

a caption extraction module configured to parse an input video stream to determine a presentation order of encoded frames in the input video stream and to assign time stamps to caption data extracted from the input video stream, wherein the caption data is reordered into the presentation order;

a caption processing module configured to translate the caption data from closed caption data into an output caption format, the output caption format being a different format than a format of the closed caption data; and

a caption packaging module configured to provide a target bit rate based on an amount of the caption data to be inserted into an output video stream and to provide the caption data in the output caption format to a video processing module that transcodes the input video stream, excluding the caption data, into the output video stream at the target bit rate, a frame rate of the input video stream being different from a frame rate of the output video stream, wherein, if audio data is present, the audio data is processed independently of the input video stream and caption data, and the caption data in the output caption format is reordered into a coded order of encoded frames in the output video stream, based on the time stamps of the caption data, and the caption data in the output caption format is written to a same file as the output video stream.

20. The system of claim 19, wherein caption data associated with a video frame is inserted into the preliminary output video stream in response to a recognition that the video frame has been discarded.

21. The system of claim 19, wherein the video processing module adjusts a compression of the encoded frames in the output video stream.

22. A non-transitory, computer readable storage medium having computer readable program code embodied in the medium, the program code, when executed, causing a processing unit to perform a method comprising:

parsing an input video stream to determine a presentation order of encoded frames in the input video stream;

if audio data is present, processing the audio data independently of the input video stream and caption data;

reordering the caption data into the presentation order, the caption data extracted from the input video stream;

assigning time stamps to the caption data;

translating the caption data from closed caption data into an output caption format, the output caption format being a different format than a format of the closed caption data;

providing a target bit rate based on an amount of the caption data to be inserted into a preliminary output video stream;

transcoding the input video stream, excluding the caption data, into the preliminary output video stream at the target bit rate, a frame rate of the input video stream being different from a frame rate of the preliminary output video stream;

reordering the caption data in the output caption format into a coded order of encoded frames in the preliminary output video stream, based on the time stamps of the caption data; and

writing the caption data in the output caption format to a same file as the preliminary output video stream.

23. The storage medium of claim 22, the method further comprising:

inserting a control code into caption data to display a caption, if the caption is stored in a non-displayed memory and is overwritten without ever being displayed.

24. The storage medium of claim 22, the method further comprising:

computing an amount of closed caption data to be buffered; and

deleting null data in the caption data, if an overflow is detected in the caption data.

25. The storage medium of claim 22, wherein, when the caption data is combined with the preliminary output video stream, a total bandwidth occupied by the preliminary output video stream does not exceed a maximum allowable bandwidth.