Synchronization of related audio and video streams

• US 6,452,974 B1
• Filed: 11/02/2000
• Issued: 09/17/2002
• Est. Priority Date: 01/02/1998
• Status: Expired due to Term

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Xu, Yueheng, Chu, Sheng-Hui, Menon, Rama R.
• Primary Examiner(s): Harvey, David E.
• Application Number:
  US09/705,464
• Time in Patent Office: 684 Days
• Field of Search: H04/N.591, H04/N.715, H04/N.504, 348/515, 348/425.4, 348/439, 348/15, 348/390, 348/17-19, 345/328, 345/302, 386/96, 375/240.28
• US Class Current: 375/240.28
• CPC Class Codes: H04N 5/04 Synchronising for television systems using pulse code modulation H04N7/56
• CPC Class Codes: H04N 5/91 Television signal processing therefor
• CPC Class Codes: H04N 5/93 Regeneration of the television signal or of selected parts thereof
• CPC Class Codes: H04N 7/147 Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals selecting H04Q
• CPC Class Codes: H04N 7/152 Multipoint control units therefor

Abstract

A method and apparatus for synchronizing audio and video streams in a video conferencing system is provided. During a video conferencing session, audio and video streams are transmitted from one processing system to a remote processing system, where they are recorded. Because the video stream has a variable frame rate during transmission, extra frames are inserted into the recorded video stream in order to maintain a constant, predetermined frame rate. During playback, synchronization information from the audio stream is provided by an audio playback process to a video playback process in order to synchronize the start of playing the audio and video streams, as well as to repeatedly synchronize the audio and video streams during playback.

Claims

1. A method of synchronizing an audio stream with a video stream, the method comprising:

inputting an audio stream and an associated video stream, wherein a rate of one of the audio stream and the video stream is variable during the transmission while a rate of the other is substantially fixed; and

playing the audio stream and the video stream, including using information from the audio stream to adjust the playing of the video stream without using video time stamp information.

2. A method according to claim 1, wherein the associated audio and video streams are received from a remote source and are part of a video conference transmission.

3. A method according to claim 1, wherein said using information from the audio stream to adjust the playing of the video stream comprises using information from the audio stream to synchronize the playing of the video stream with the playing of the audio stream, without using video time stamp information.

4. A method according to claim 3, further comprising recording the audio stream and the video stream after said inputting and prior to said playing, such that said playing is playing the recorded audio stream and the recorded video stream.

5. A method according to claim 1, wherein the associated audio and video streams are received over a computer network.

6. A method according to claim 1, wherein the video stream includes a plurality of video frames, the method further comprising inserting copies of video frames that are already present in the video stream into the video stream to maintain a substantially constant, predetermined video frame rate.

7. A method according to claim 4, wherein using information from the recorded audio stream to adjust the playing of the recorded video stream comprises:

using information from the recorded audio stream to synchronize a start of playing of the recorded audio stream with a start of playing of the recorded video stream; and

repeatedly using information from the recorded audio stream to resynchronize the playing of the recorded audio stream with the playing of the recorded video stream.

8. A method according to claim 1, wherein the video stream includes a plurality of video frames, the method further comprising adding video frames into the video stream to maintain a substantially constant video frame rate.

9. A method according to claim 1, wherein the information from the audio stream comprises information indicative of a degree of progress of the playing of the audio stream.

10. A method according to claim 1, wherein the video stream includes a plurality of video frames, the method further comprising inserting blank video frames into the video stream to maintain a substantially constant, predetermined video frame rate.

11. An apparatus for synchronizing a video stream with a corresponding audio stream, the apparatus comprising:

means for recording the audio stream and the video stream, the video stream including a plurality of video frames, the means for recording including means for inserting extra frames into the video stream to maintain a constant, predetermined video frame rate during subsequent playback of the video stream; and

means for playing back the recorded audio and video streams, including means for adjusting the playback of the video stream based on progress information from the audio stream.

12. An apparatus according to claim 11, wherein the audio and video streams are recorded audio and video streams of a real-time audiovisual data transmission from a remote source.

13. An apparatus according to claim 11, wherein the means for playing back the recorded audio and video streams comprises means for using information from the recorded audio stream to synchronize a start of playing of the recorded audio stream with a start of playing of the recorded video stream.

14. A machine-readable program storage medium tangibly embodying sequences of instructions, the sequences of instructions for execution by at least one processing system to perform a method of synchronizing corresponding audio and video data streams, the method comprising:

recording an audio stream and a corresponding video stream, the video stream including a plurality of video frames and having a variable frame rate, wherein recording the audio and video streams includes adding frames to the video stream based on the variable frame rate to simulate a constant, predetermined video frame rate; and

playing back the recorded audio and video streams, including using information from the audio stream to synchronize the video stream with the audio stream.

15. A machine-readable program storage medium according to claim 14, wherein the audio and video streams recorded during said recording are from a real-time audiovisual data transmission from a remote processing system.

16. A machine-readable program storage medium according to claim 14, wherein playing back of the recorded audio and video streams comprises using information from the recorded audio stream to synchronize a start of playing of the recorded audio stream with a start of playing of the recorded video stream.