Method, apparatus and computer program product for synchronizing presentation of digital video data with serving of digital video data

• US 6,654,956 B1
• Filed: 04/10/2000
• Issued: 11/25/2003
• Est. Priority Date: 04/10/2000
• Status: Expired due to Fees

• Current Assignee: V-Silicon Semiconductors (HEFEI) Co., Ltd.
• Original Assignee: Sigma Designs Incorporated
• Inventors: Ignaszewski, Michael, Mahe, Jacques, Trinh, Vincent
• Primary Examiner(s): Miller, John
• Assistant Examiner(s): Hoye, Michael W.
• Application Number:
  US09/546,055
• Time in Patent Office: 1,324 Days
• Field of Search: 348/515, 348/512, 348/518, 348/423.1, 348/425.4, 375/240.28, 725/100, 725/91, 725/95, 725/131
• US Class Current: 725/100
• CPC Class Codes: H04N 21/426 Internal components of the client ; Characteristics thereof H04N5/44 takes precedence
• CPC Class Codes: H04N 21/4305 Synchronising client clock from received content stream, e.g. locking decoder clock with encoder clock, extraction of the PCR packets arrangements for synchronising receiver with transmitter by comparing receiver clock with transmitter clock H04L7/0012; arrangements for synchronising receiver with transmitter wherein the receiver takes measures against momentary loss of synchronisation H04L7/0083
• CPC Class Codes: H04N 21/4341 Demultiplexing of audio and video streams
• CPC Class Codes: H04N 21/8547 involving timestamps for synchronizing content
• CPC Class Codes: H04N 5/04 Synchronising for television systems using pulse code modulation H04N7/56
• CPC Class Codes: H04N 5/602 for digital sound signals

Abstract

The present invention provides a method for matching the rate of presentation of digital video data at a receiver/client with the rate the server is serving the data without the need of a voltage control oscillator or other hardware. An embodiment of the invention compares the presentation time at the receiver with the server elapsed time estimated from timestamp values on the served data. When the presentation time and the server elapsed time differ by an unacceptably large amount, an adjustment is made to the audio data stream to re-synchronize the presentation time with the elapsed time, which effectively also synchronizes the overall presentation rate with the overall server rate. The video data stream is then synchronized to the adjusted audio stream. The video data stream and audio data stream are converted to analog signals for presentation.

Claims

1. A method for synchronizing serving of a formatted digital video data stream that is served by a server with decoding of said formatted digital video data stream at a receiver including steps of:

receiving said formatted digital video data stream at said receiver;

demultiplexing said formatted digital video data stream into an audio data stream and a video data stream, said audio data stream having an audio time interval;

checking whether serving of said formatted digital video data stream is synchronized with decoding of said audio data stream;

performing, responsive to the step of checking, the further steps of;

adjusting said audio data stream to synchronize with a server elapsed time, said step of adjusting resulting in an adjusted audio data stream;

synchronizing said video data stream to said adjusted audio data stream.

2. A method as in claim 1 wherein the step of checking is done periodically.

3. A method as in claim 1 wherein the step of checking includes steps of:

monitoring said formatted digital video data stream for a time stamp;

determining, responsive to the step of monitoring, whether a difference between a presentation time and a server elapsed time is acceptable.

4. A method as in claim 3, wherein said difference is acceptable if said difference is the time required to processes on the order of 100 audio samples.

5. A method as in claim 1, wherein said step of synchronizing the video data stream to said adjusted audio data stream is done responsive to a time stamp value.

6. A method as in claim 1, further including steps of:

converting said audio data to an analog audio signal;

converting said video data to an analog video signal;

presenting said analog audio signal; and

presenting said analog video signal.

7. A method as in claim 1 wherein the step of adjusting said audio data stream to synchronize with said server elapsed time includes steps of:

loading a necessary adjustment value into a register;

adjusting said audio stream in accordance with a value in said register.

8. A method as in claim 7, wherein the step of adjusting includes the steps of:

if said register value is negative, dropping a number of samples in said audio data stream in accordance with said register value;

incrementing the value of said register to reflect said number of samples.

9. A method as in claim 7, wherein the step of adjusting includes the steps of:

if said register value is positive, duplicating a second number of samples in said audio data stream in accordance with said register value;

decrementing the value of said register to reflect said number of samples.

25. A method as in claim 1, wherein said steps of adjusting and synchronizing are effective to synchronize serving of said formatted digital video data stream with decoding of said formatted digital video data stream so as to help prevent underflow or overflow of a buffer at said receiver that receives said formatted digital video data stream from said server.

10. An apparatus for synchronizing serving of a formatted digital video data stream that is served by a server with decoding of said formatted digital video data stream at a receiver including:

a memory interface configured to receive said formatted digital video data stream at said receiver;

a synchronization time checker configured to check whether serving of said formatted digital video data stream is synchronized with decoding of said audio data stream, said formatted digital video data stream flowing to said checker from said memory interface;

a time stamp detector coupled to said checker, said time stamp detector configured to detect time stamps in said formatted digital video data stream;

a demultiplexer coupled to said checker, said demultiplexer configured to demultiplex said formatted digital video data stream into an audio data stream and a video data stream, said audio data stream having an audio time interval, said formatted digital video stream flowing to said demultiplexer from said checker;

a comparor/calculator coupled to said time stamp detector and coupled to said demultiplexer, said comparor/calculator configured to compare a presentation time with a server elapsed time and configured to calculate an adjustment value;

an adjustor configured to receive said audio data stream and to adjust said audio data stream to synchronize with said server elapsed time, in accordance with said adjustment value, resulting in an adjusted audio data stream; and

a audio-video synchronizer configured to synchronize the video data stream to said adjusted audio data stream, said audio-video synchronizer coupled to said adjustor and coupled to said demultiplexer, said video data stream flowing to said audio-video synchronizer from said demultiplexer, said adjusted audio stream flowing to said audio-video synchronizer from said adjustor.

11. An apparatus as in claim 10, further including:

an audio digital to analog converter (ADAC) configured to convert said audio data to an analog audio signal, said ADAC coupled to said audio-video synchronizer;

a video digital to analog converter (VDAC) configured to convert said video data to an analog video signal, said VDAC coupled to said audio-video synchronizer;

a first presentation device configured to present said analog audio signal, said presentation device coupled to said ADAC; and

a second presentation device configured to present said analog video signal, said second presentation device coupled to said VDAC.

12. An apparatus as in claim 10 wherein said synchronization time checker is configured to perform periodically.

13. An apparatus as in claim 10, wherein said adjustor includes a register that holds a register value, and wherein said register is configured to change said register value responsive to said adjustment value and responsive to adjustments to said audio data stream made by said adjustor.

14. An apparatus as in claim 13 wherein if said register value is negative, said adjustor is configured to drop a number of samples in said audio data stream in accordance with said register value and increment the value of said register to reflect said number of samples.

15. An apparatus as in claim 13 wherein if said register value is positive, said adjustor is configured to duplicate a second number of samples in said audio data stream in accordance with said register value and to decrement the value of said register to reflect said second number of samples.

26. An apparatus as in claim 10, further comprising a buffer at said receiver that receives said formatted digital video data stream from said server, wherein said adjustor and said audio-video synchronizer are effective to synchronize serving of said formatted digital video data stream with decoding of said formatted digital video data stream so as to help prevent underflow or overflow of said buffer.

16. A computer program product that stores a program, the program including the steps of:

receiving said formatted digital video data stream at said receiver;

demultiplexing said formatted digital video data stream into an audio data stream and a video data stream, said audio data stream having an audio time interval;

checking whether serving of said formatted digital video data stream is synchronized with decoding of said audio data stream;

performing, responsive to the step of checking, the further steps of;

adjusting said audio data stream to synchronize with a server elapsed time, said step of adjusting resulting in an adjusted audio data stream;

synchronizing said video data stream to said adjusted audio data stream.

17. A computer program product as in claim 16 wherein the step of checking is done periodically.

18. A computer program product as in claim 16 wherein the step of checking includes steps of:

monitoring said formatted digital video data stream for a time stamp;

determining, responsive to the step of monitoring, whether a difference between a presentation time and a server elapsed time is acceptable.

19. A computer program product as in claim 18, wherein said difference is acceptable if said difference is the time required to processes on the order of 100 audio samples.

20. A computer program product as in claim 16, wherein said step of synchronizing the video data stream to said adjusted audio data stream is done responsive to a time stamp value.

21. A computer program product as in claim 16, further including steps of:

converting said audio data to an analog audio signal;

converting said video data to an analog video signal;

presenting said analog audio signal; and

presenting said analog video signal.

22. A computer program product as in claim 16 wherein the step of adjusting said audio data stream to synchronize with said server elapsed time includes steps of:

loading a necessary adjustment value into a register;

adjusting said audio stream in accordance with a value in said register.

23. A computer program product as in claim 22, wherein the step of adjusting includes the steps of:

if said register value is negative, dropping a number of samples in said audio data stream in accordance with said register value;

incrementing the value of said register to reflect said number of samples.

24. A computer program product as in claim 22, wherein the step of adjusting includes the steps of:

if said register value is positive, duplicating a second number of samples in said audio data stream in accordance with said register value;

decrementing the value of said register to reflect said number of samples.

27. A computer program product as in claim 16, wherein said steps of adjusting and synchronizing are effective to synchronize serving of said formatted digital video data stream with decoding of said formatted digital video data stream so as to help prevent underflow or overflow of a buffer at said receiver that receives said formatted digital video data stream from said server.