System and method for synchronizing playback of audio and video

• US 8,027,560 B2
• Filed: 02/05/2007
• Issued: 09/27/2011
• Est. Priority Date: 02/05/2007
• Status: Active Grant

• Current Assignee: Thales Avionics Incorporated (Thales SA)
• Original Assignee: Thales Avionics Incorporated (Thales SA)
• Inventors: Treneer, George, Norton, Lyle K., Brady, Kenneth A. Jr.
• Primary Examiner(s): TRAN, THAI Q
• Application Number:
  US11/702,251
• Publication Number:
  US 20080187282A1
• Time in Patent Office: 1,695 Days
• Field of Search: 386/200, 386/203, 386/229, 386/201, 386/354, 375/293, 375/256, 375/240.28, 348/513, 348/521
• US Class Current: 386/203
• CPC Class Codes: H04L 65/4015 where at least one of the additional parallel sessions is real time or time sensitive, e.g. white board sharing, collaboration or spawning of a subconference
• CPC Class Codes: H04L 65/80 Responding to QoS
• CPC Class Codes: H04L 67/12 specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
• CPC Class Codes: H04N 21/2146 located in mass transportation means, e.g. aircraft, train or bus flight-deck installations for entertainment or communications B64D11/0015; arrangements specially adapted for transportation systems in broadcast systems H04H20/62; moving wireless networks H04W84/005
• CPC Class Codes: H04N 21/43076 of the same content streams on multiple devices, e.g. when family members are watching the same movie on different devices
• CPC Class Codes: H04N 21/47202 for requesting content on demand, e.g. video on demand
• CPC Class Codes: H04N 21/64322 IP
• CPC Class Codes: H04N 5/073 for mutually locking plural sources of synchronising signals, e.g. studios or relay stations
• CPC Class Codes: H04N 5/775 between a recording apparatus and a television receiver
• CPC Class Codes: H04N 7/52 Systems for transmission of a pulse code modulated video signal with one or more other pulse code modulated signals, e.g. an audio signal or a synchronizing signal assembling of a multiplex stream by combining a video stream with other content or additional data, remultiplexing of multiplex streams, insertion of stuffing bits into the multiplex stream, assembling of a packetised elementary stream at server side H04N21/236; disassembling of a multiplex stream, remultiplexing of multiplex streams, extraction or processing of Service Information, disassembling of packetised elementary stream at client side H04N21/434
• CPC Class Codes: H04N 9/8042 involving data reduction
• CPC Class Codes: H04N 9/8063 using time division multiplex of the PCM audio and PCM video signals

Abstract

A system and method for providing a digital In-Flight Entertainment (IFE) system in a vehicle, such as an aircraft, that is capable of presenting a video program and associated audio in a synchronized manner to a large number of individual video monitors and speakers. The system and method employ processing operations in at least one decoder of the IFE system, to perform operations such as adjusting a local clock based on a master clock, setting a delay time in relation to a master clock, and adjusting video and audio playback based on the delay, to substantially synchronize playback of the audio and video data by the audio and video players, to thus eliminate or at least minimize the negative effects of source and load based jitter, network delays, clock drifts, network errors and decoder buffering differences, on synchronizing video and audio playback.

Claims

1. A method for synchronizing digital video and audio playback by at least two video players and at least two audio players in a system, the method comprising:

providing digital video and audio data over a digital data network to a plurality of decoders associated with the at least two video players and the at least two audio players, each of the plurality of decoders connected to the digital data network at a different location; and

operating each of the plurality of decoders to adjust a local clock of the respective decoder based on a master clock derived from content of the data, set a system delay time in relation to the master clock, the system delay time representing a maximum delay in delivering the digital video and audio data to the plurality of decoders, and adjust video and audio playback based on their respective local clock and the system delay time to substantially synchronize playback of the audio and video data by the at least two audio players and the at least two video players associated with different ones of the plurality of decoders.

2. A method as claimed in claim 1, wherein operating each of the plurality of decoders to adjust the local clock comprises:

initializing the local clock upon receipt of a program clock reference (PCR); and

upon receipt of a subsequent PCR, determining whether to adjust the local clock based on a value of the local clock in relation to a presentation time stamp (PTS).

3. A method as claimed in claim 2, wherein operating each of the plurality of decoders to adjust the local clock further comprises:

adjusting the value of the local clock to substantially match the PTS if the value of the local clock is greater than the PTS.

4. A method as claimed in claim 1, wherein operating each of the plurality of decoders to adjust video playback comprises:

comparing a time value at which a video frame is being presented in relation to a sum of a time of the master clock and the system delay time; and

operating the at least one video player to at least one of restart, replay or skip the video frame based on the comparison.

5. A method as claimed in claim 4, further comprising:

operating one of the at least two video players to restart video presentation when the time value is greater than the sum by a first amount.

6. A method as claimed in claim 5, further comprising:

operating the one of the at least two video players to skip presenting the video frame when the time value is greater than the sum by a second amount that is less than the first amount.

7. A method as claimed in claim 5, further comprising:

operating the one of the at least two video players to repeat presenting the video frame when the time value is less than the sum by a third amount that is less than the first amount.

8. A method as claimed in claim 1, wherein operating each one of the plurality of decoders to adjust audio playback comprises:

comparing a time value at which a moment of audio is being presented to a sum of a time of the master clock and the system delay time; and

operating one of the at least two audio players to at least one of restart, skip ahead in the audio or refrain from presenting audio for a time period based on the comparison.

9. A method as claimed in claim 8, further comprising:

operating the one of the at least two audio players to restart audio presentation when the time value is greater than the sum by a first amount.

10. A method as claimed in claim 9, further comprising:

operating the one of the at least two audio players to refrain from presenting audio for the period, when the time value is greater than the sum by a second amount that is less than the first amount.

11. A method as claimed in claim 9, further comprising:

operating the at one of the at least two audio players to skip ahead in the audio when the time value is less than the sum by a third amount that is less than the first amount.

12. A method as claimed in claim 1, further comprising:

adjusting a frequency of at least one of a first playback clock associated with one the at least two video players or a second playback clock associated with the one of the at least two audio players in relation to a clock associated with a source that provides the digital video and audio data.

13. A method as claimed in claim 1, wherein the method operates so that a person is able to perceive a frame of the digital video being presented on a first of the at least two video players at substantially a same time as the frame is presented on a second of the at least two video players.

14. A decoder, associated with an audio player and a video player, for use in a digital system that provides digital video and audio data, the decoder comprising:

a network interface that receives the digital video and audio data;

a local clock;

a video decoder and an audio decoder that decode the received video and audio data;

a video driver and an audio driver that drive the associated video player and the associated audio player; and

a controller to adjust the local clock based on a master clock derived from content of the data, to set a playback time for the decoded video and audio based on the master clock and a system-delay time to substantially synchronize playback of the audio and video data by the audio player and the video player with playback of the audio and video data at another audio player and another video player associated with a different decoder in the system that also receives the digital video and audio data, wherein the system delay time represents a maximum delay in delivering the digital video and audio data to the decoders, the decoders coupled to the digital system at different locations.

15. A decoder as claimed in claim 14, wherein:

the controller is to adjust the local clock by initializing the local clock upon receipt of a program clock reference (PCR), and upon receipt of a subsequent PCR by the interface, adjust the local clock based on a value of the local clock in relation to a presentation time stamp (PTS).

16. A decoder as claimed in claim 14, wherein:

the controller is to adjust video playback by comparing a time value at which a video frame is being presented by the video player in relation to a sum of a time of the master clock and the system delay time, and to control the video driver to operate the video player to at least one of restart, replay or skip the video frame based on the comparison.

17. A decoder as claimed in claim 14, wherein:

the controller is to adjust audio playback by comparing a time value at which a moment of audio is being presented by the audio player in relation to a sum of a time of the master clock and the system delay time, and to control the audio driver to operate the audio player to at least one of restart, skip ahead in the audio or refrain from presenting audio for a period based on the comparison.

18. A decoder as claimed in claim 14, wherein the controller is to adjust a frequency of at least one of a first playback clock associated with the video player and a second playback clock associated with the audio player in relation to a clock associated with a source that provides the digital video and audio data.