Merging of multiple encoded audio-video streams into one program with source clock frequency locked and encoder clock synchronized

• US 7,735,111 B2
• Filed: 04/29/2005
• Issued: 06/08/2010
• Est. Priority Date: 04/29/2005
• Status: Expired due to Fees

• Current Assignee: The DIRECTV Group, Inc. (AT&T, Inc.)
• Original Assignee: The DIRECTV Group, Inc. (AT&T, Inc.)
• Inventors: Michener, James A., Stranger, Leon J.
• Primary Examiner(s): Koenig; Andrew Y
• Assistant Examiner(s): Featherstone; Mark D
• Application Number:
  US11/118,926
• Publication Number:
  US 20060248559A1
• Time in Patent Office: 1,866 Days
• Field of Search: None
• US Class Current: 725/146
• CPC Class Codes: H04N 21/233 Processing of audio elementary streams monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/2368 Multiplexing of audio and video streams
• CPC Class Codes: H04N 21/242 Synchronization processes, e.g. processing of PCR [Program Clock References] arrangements for synchronising broadcast or distribution via plural systems in broadcast distribution systems H04H20/18
• CPC Class Codes: H04N 21/2665 Gathering content from different sources, e.g. Internet and satellite
• CPC Class Codes: H04N 21/8106 involving special audio data, e.g. different tracks for different languages

Abstract

A system and method for merging of multiple encoded audio and video streams into a single program in the transport stream enables playback of any audio stream with any video stream. This is accomplished by first locking the frequency of audio and video input streams for a plurality of different programs to a source clock, independently encoding each program with its own PCR and PTS into a program stream, multiplexing the program streams into a transport stream, and then synchronizing the PCR and PTS of the program streams to a declared master PCR.

Claims

1. A method, comprising:

frequency locking the audio and video streams from a plurality of programs;

using independent encoder/program multiplexers, each having its own clock reference, to separately encode and multiplex each program'"'"'s audio and video streams into a program stream, each program stream having an independent program clock reference (PCR) and audio and video presentation time stamps (PTS) in the audio and video streams;

multiplexing the program streams into a transport stream; and

synchronizing the programs'"'"' PCR and audio and video PTSs in the transport stream from a declared master PCR while avoiding feedback to any of the encoder/program multiplexers in one of the program streams to merge the programs'"'"' PCR and audio and video PTSs into one program having the master PCR against which the PTSs for all the audio and video streams are referenced, wherein the programs'"'"' PCR and PTS are synchronized by declaring one program and its PCR to be the master PCR and the remaining programs and their PCRs to be slaves, comparing each slave PCR to an interpolated value of the master PCR to compute an offset, setting the slave PCR to the interpolated value, and adding the offset to each occurrence of the slave'"'"'s PTS.

2. The method of claim 1, wherein the audio and video streams are frequency locked by providing a single source clock to capture the streams.

3. The method of claim 1, wherein the audio and video streams are frame synchronized to both frequency and phase lock the streams.

4. The method of claim 1, wherein the programs'"'"' PCR and PTS are synchronized sequentially.

5. The method of claim 1, wherein the offsets for each slave program are time averaged.

6. The method of claim 1, further comprising:

transmitting the transport stream to a customer site, in response to a customer command selecting a video stream from a program stream and an audio stream from a different program stream, extracting the selected audio and video streams, the master PCR and the audio and video PTSs from the transport stream,

decoding the selected audio and video streams as if they came from the same program stream, and

in accordance with the PTSs for the audio and video with respect to the master PCR, playing back the audio and video streams in frequency and time synchronization.

7. The method of claim 5, wherein, if a discontinuity is detected between the time-averaged offset and the current offset, resetting the time averaged offset to the current offset.

8. The method of claim 6, wherein the customer selects a plurality of video streams and/or a plurality of audio streams from different program streams for playback.

9. A method, comprising:

receiving a transport stream comprising multiplexed program streams each of which comprise an encoded video stream, an encoded audio stream, a program clock reference (PCR) and audio and video presentation time stamps (PTS), wherein said video streams are source clock frequency locked and said PCRs are encoder clock synchronized to a master PCR against which the PTSs for all the audio and video streams are referenced, wherein the programs'"'"' PCR and PTS are synchronized by declaring one program and its PCR to be the master PCR and the remaining programs and their PCRs to be slaves, comparing each slave PCR to an interpolated value of the master PCR to compute an offset, setting the slave PCR to the interpolated value, and adding the offset to each occurrence of the slave'"'"'s PTS,

in response to a customer command selecting a video stream from a program stream and an audio stream from a different program stream, extracting the selected audio and video streams, the master PCR, and the audio and video PTSs from the transport stream,

decoding the selected audio and video streams as if they came from a common program stream, and

in accordance with the PTSs for the selected audio and video streams with respect to the master PCR, playing back the selected audio and video streams in frequency and time synchronization.

10. The method of claim 9, wherein the custom selects a plurality of video streams and/or a plurality of audio streams from different program streams for playback.

11. The method of claim 9, wherein the audio and video streams are decoded by,

locking an internal clock of the decoder to the master PCR by loading the next PCR value into its counter and continuing to count,

decoding the selected audio and video streams,

storing the decoded audio and video streams in respective buffers, and

monitoring the master PCR via the counter and the PTSs to output the audio and video streams to the playback device.

12. An apparatus, comprising:

a frequency locker that frequency locks the audio and video streams from a plurality of programs;

a plurality of independent encoders each having its own clock reference, each encoder encoding and multiplexing a different said program'"'"'s audio and video streams into a program stream, each program stream having an independent program clock reference (PCR) and audio and video presentation time stamps (PTS) in the audio and video streams;

a transport multiplexer that multiplexes the program streams into a transport stream; and

an encoder clock synchronizer that, without feedback to any of the encoder/program multiplexers, declares one program and its PCR to be the master and the remaining programs and their PCRs to be slaves, and synchronizes the programs'"'"' PCR and audio and video PTSs in the transport stream from the declared master PCR in one of the program streams while avoiding feedback to any of the encoder/program multiplexers by comparing each slave PCR to an interpolated value of the master PCR to compute an offset, setting the slave PCR to the interpolated value, and adding the offset to each occurrence of the slave'"'"'s PTS, to merge the programs'"'"' PCR and audio and video PTSs into one program having the master PCR against which the PTS for all the audio and video streams are referenced.

13. The apparatus of claim 12, wherein the frequency locker comprises a single source clock used to capture the audio and video streams.

14. The apparatus of claim 12, wherein the frequency locker comprises a plurality of frequency lockers that receive the respective programs and a common reference signal.

15. The apparatus of claim 12, wherein the encoder clock synchronizer synchronizes the programs'"'"' PCR and PTS sequentially.

16. The apparatus of claim 12, wherein the encoder clock synchronizer time averages the offsets for each slave program.

17. The apparatus of claim 12, further comprising:

a communication channel for transmitting the transport stream to a plurality of user sites, each user site comprising,

a receiver for receiving the transport stream,

a user selection device for selecting a video stream from a program stream an audio stream from a different program stream to extract the selected audio and video streams, the master PCR and the audio and video PTSs from the transport stream;

a decoder for decoding the selected audio and video streams as if they came from the same program stream; and

a playback device for playing back the audio and video streams in frequency and time synchronization.

18. The apparatus of claim 17, wherein the decoder executes the following steps to decode the audio and video streams,

locking an internal clock of the decoder to the master PCR by loading the next PCR value into its counter and continuing to count,

decoding the selected audio and video streams,

storing the decoded audio and video streams in respective buffers, and

monitoring the master PCR via the counter and the PTSs to output the audio and video streams to the playback device.

19. The apparatus of claim 16, wherein, if a discontinuity is detected between the time averaged offset and the current offset, the encoder clock synchronizer resets the time averaged offset to the current offset.