System and method for synchronizing data streams

• US 5,794,018 A
• Filed: 03/02/1995
• Issued: 08/11/1998
• Est. Priority Date: 11/24/1993
• Status: Expired due to Fees

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Gutmann, Michael, Tung, Peter, Vrvilo, Benjamin
• Primary Examiner(s): Sheikh, Ayaz R.
• Assistant Examiner(s): DHARIA, RUPAL
• Application Number:
  US08/398,164
• Time in Patent Office: 1,258 Days
• Field of Search: 395/550, 395/551, 395/162, 395/200.34, 395/200.61, 395/200.78, 370/100.1, 370/62, 358/341, 387/97, 348/512, 348/420, 348/15, 379/202
• US Class Current: 713/400
• CPC Class Codes: G06Q 10/10 Office automation; Time management
• CPC Class Codes: G06T 1/00 General purpose image data processing
• CPC Class Codes: G06T 2200/16 involving adaptation to the client's capabilities
• CPC Class Codes: H04L 12/1813 for computer conferences, e.g. chat rooms instant messaging H04L51/04; protocols for multimedia communication H04L65/1101; arrangements for multi-party communication H04L65/403; telephonic conference arrangements H04M3/56; television conference systems H04N7/15
• CPC Class Codes: H04L 12/1818 Conference organisation arrangements, e.g. handling schedules, setting up parameters needed by nodes to attend a conference, booking network resources, notifying involved parties
• CPC Class Codes: H04L 12/1827 Network arrangements for conference optimisation or adaptation
• CPC Class Codes: H04M 3/567 Multimedia conference systems
• CPC Class Codes: H04M 7/006 Networks other than PSTN/ISDN providing telephone service, e.g. Voice over Internet Protocol (VoIP) , including next generation networks with a packet-switched transport layer H04L65/00 takes precedence; aspects not specific to the type of network H04M3/00; special services in those networks H04M3/42
• CPC Class Codes: H04M 9/08 Two-way loud-speaking telephone systems with means for conditioning the signal, e.g. for suppressing echoes for one or both directions of traffic
• CPC Class Codes: H04N 21/4143 embedded in a Personal Computer [PC]
• CPC Class Codes: H04N 21/643 Communication protocols network streaming protocols, e.g. real-time transport protocol [RTP] or real-time control protocol [RTCP], H04L65/65
• CPC Class Codes: H04N 7/148 Interfacing a video terminal to a particular transmission medium, e.g. ISDN
• CPC Class Codes: H04N 7/15 Conference systems
• CPC Class Codes: H04Q 2213/13003 Constructional details of switching devices
• CPC Class Codes: H04Q 2213/13103 Memory
• CPC Class Codes: H04Q 2213/13106 Microprocessor, CPU
• CPC Class Codes: H04Q 2213/13204 Protocols
• CPC Class Codes: H04Q 2213/13209 ISDN
• CPC Class Codes: H04Q 2213/1324 Conference call
• CPC Class Codes: H04Q 2213/13299 Bus
• CPC Class Codes: H04Q 2213/13337 : Picturephone, videotelephony
• CPC Class Codes: H04Q 2213/13389 : LAN, internet

Abstract

Two sequences of signal packets are played. Each signal packet of a first type of signals has been stamped with a sequence number corresponding to the position of the signal packet in the first sequence. Each signal packet of a second type of signals has been stamped with a corresponding sequence number of the first sequence. The playing of the second sequence is synchronized with the playing of the first sequence by comparing the sequence numbers of the second sequence to the sequence numbers of the first sequence. In a preferred embodiment, the two types of signals are audio signals and video signals, and the video signal packets are stamped with sequence numbers for the audio signal packets. Each video signal packet of the video sequence is processed for display based on a comparison of the sequence number for the video signal packet and the sequence number for the current audio signal packet.

Claims

1. A computer-implemented process for synchronizing data streams, comprising the steps of:

(a) playing a first sequence of signal packets of a first type of signals, wherein each signal packet of the first sequence has been stamped with a sequence number corresponding to the position of the signal packet in the first sequence; and

(b) playing a second sequence of signal packets of a second type of signals, wherein;

each signal packet of the second sequence has been stamped with a corresponding sequence number of the first sequence; and

the playing of the second sequence is synchronized with the playing of the first sequence by comparing the sequence numbers of the second sequence to the sequence numbers of the first sequence.

2. The process of claim 1, wherein:

D is the absolute value of the difference between the sequence number of a current signal packet of the first sequence and the sequence number of a current signal packet of the second sequence; and

step (b) comprises the steps of;

(1) playing the current signal packet of the second sequence in a normal mode, if D is less than or equal to a first threshold value;

(2) processing the current signal packet of the second sequence in a hurry-up mode, if D is greater than the first threshold value and D is less than or equal to a second threshold value, wherein the second threshold value is greater than the first threshold value; and

(3) restarting the playing of the second sequence, if D is greater than the second threshold value.

3. The process of claim 2, wherein:

step (b)(1) comprises the steps of decoding and displaying the current signal packet of the second sequence immediately; and

step (b)(2) comprises the step of partially decoding the current signal packet of the second sequence without displaying the partially decoded current signal packet.

4. The process of claim 3, wherein step (b)(2) comprises the step of partially decoding the current signal packet of the second sequence without color converting the partially decoded current signal packet of the second sequence.

5. The process of claim 1, wherein the first sequence of signal packets of the first type of signals comprises an audio sequence of audio signal packets and the second sequence of signal packets of the second type of signals comprises a video sequence of video signal packets.

6. The process of claim 5, wherein:

the audio sequence has been generated by a first node of a conferencing network;

the first node has stamped each audio signal packet of the audio sequence with an audio sequence number corresponding to the position of the audio signal packet in the audio sequence;

the video sequence has been generated by the first node;

the first node has stamped each video signal packet of the video sequence with a corresponding audio sequence number of the audio sequence;

step (a) comprises the steps of;

(1) receiving the audio sequence and the video sequence by a second node of the conferencing network over a communications link of the conferencing network from the first node; and

(2) playing the audio sequence by the second node; and

step (b) comprises the step of playing the video sequence by the second node, wherein the playing of the video sequence is synchronized with the playing of the audio sequence by comparing the audio sequence numbers of the video sequence to the audio sequence numbers of the audio sequence.

7. The process of claim 6, wherein the communications link is an ISDN network.

8. The process of claim 6, wherein each video signal packet of the video sequence has been stamped with the audio sequence number corresponding to the most recent audio signal packet.

9. The process of claim 1, wherein:

the first sequence of signal packets of the first type of signals comprises an audio sequence of audio signal packets;

the second sequence of signal packets of the second type of signals comprises a video sequence of video signal packets;

audio sequence has been generated by a first node of a conferencing network;

the first node has stamped each audio signal packet of the audio sequence with a time stamp;

the video sequence has been generated by the first node;

the first node has stamped each video signal packet of the video sequence with the time stamp of the most recent audio signal packet of the audio sequence;

step (a) comprises the steps of;

(1) receiving the audio sequence and the video sequence by a second node of the conferencing network over a communications link of the conferencing network from the first node; and

(2) playing the audio sequence by the second node;

D is the absolute value of the difference between the time stamp of a current audio signal packet and the time stamp of a current video signal packet;

step (b) comprises the steps of;

(1) decoding and displaying the current video signal packet immediately, if D is less than or equal to a first threshold value;

(2) partially decoding the current video signal packet without color converting and without displaying the partially decoded current video signal packet, if D is greater than the first threshold value and D is less than or equal to a second threshold value, wherein the second threshold value is greater than the first threshold value; and

(3) restarting the playing of the video sequence, if D is greater than the second threshold value.

10. The process of claim 9, wherein the communications link is an ISDN network.

27. The process of claim 1, wherein the sequence numbers are time stamps.

11. A computer system for synchronizing data streams, comprising:

(a) a first subsystem for playing a first sequence of signal packets of a first type of signals; and

(b) a second subsystem for playing a second sequence of signal packets of a second type of signals, wherein;

each signal packet of the first sequence has been stamped with a sequence number corresponding to the position of the signal packet in the first sequence;

each signal packet of the second sequence has been stamped with a corresponding sequence number of the first sequence; and

playing of the second sequence is synchronized with the playing of the first sequence by comparing the sequence numbers of the second sequence to the sequence numbers of the first sequence.

12. The system of claim 11, wherein:

D is the absolute value of the difference between the sequence number of a current signal packet of the first sequence and the sequence number of a current signal packet of the second sequence;

the second subsystem plays the current signal packet of the second sequence in a normal mode, if D is less than or equal to a first threshold value;

the second subsystem processes the current signal packet of the second sequence in a hurry-up mode, if D is greater than the first threshold value and D is less than or equal to a second threshold value, wherein the second threshold value is greater than the first threshold value; and

the second subsystem restarts the playing of the second sequence, if D is greater than the second threshold value.

13. The system of claim 12, wherein:

the second subsystem decodes and displays the current signal packet of the second sequence immediately, if D is less than or equal to the first threshold value; and

the second subsystem partially decodes the current signal packet of the second sequence without displaying the partially decoded current signal packet, if D is greater than the first threshold value and D is less than or equal to the second threshold value.

14. The system of claim 13, wherein the second subsystem partially decodes the current signal packet of the second sequence without color converting the partially decoded current signal packet of the second sequence, if D is greater than the first threshold value and D is less than or equal to the second threshold value.

15. The system of claim 11, wherein the first sequence of signal packets of the first type of signals comprises an audio sequence of audio signal packets and the second sequence of signal packets of the second type of signals comprises a video sequence of video signal packets.

16. The system of claim 15, wherein:

the audio sequence has been generated by a first node of a conferencing network;

the first node has stamped each audio signal packet of the audio sequence with a sequence number corresponding to the position of the audio signal packet in the audio sequence;

video sequence has been generated by the first node;

the first node has stamped each video signal packet of the video sequence with a corresponding sequence number of the audio sequence;

a second node of the conferencing network receives the audio sequence and the video sequence over a communications link of the conferencing network from the first node, wherein;

the second node comprises the first subsystem and the second subsystem;

the first subsystem is an audio subsystem; and

the second subsystem is a video subsystem;

the audio subsystem plays the audio sequence; and

the video subsystem plays the video sequence, wherein the playing of the video sequence is synchronized with the playing of the audio sequence by comparing the sequence numbers of the video sequence to the sequence numbers of the audio sequence.

17. The system of claim 16, wherein the communications link is an ISDN network.

18. The system of claim 16, wherein each video signal packet of the video sequence has been stamped with the sequence number corresponding to the most recent audio signal packet.

19. The system of claim 11, wherein:

the first sequence of signal packets of the first type of signals comprises an audio sequence of audio signal packets;

the second sequence of signal packets of the second type of signals comprises a video sequence of video signal packets;

the audio sequence has been generated by a first node of a conferencing network;

the first node has stamped each audio signal packet of the audio sequence with a time stamp;

the video sequence has been generated by the first node;

the first node has stamped each video signal packet of the video sequence with the time stamp of the most recent audio signal packet of the audio sequence;

a second node of the conferencing network receives the audio sequence and the video sequence over a communications link of the conferencing network from the first node, wherein;

the second node comprises the first subsystem and the second subsystem;

the first subsystem is an audio subsystem; and

the second subsystem is a video subsystem;

the audio subsystem plays the audio sequence;

D is the absolute value of the difference between the time stamp of a current audio signal packet and the time stamp of a current video signal packet;

the video subsystem decodes and displays the current video signal packet immediately, if D is less than or equal to a first threshold value;

the video subsystem partially decodes the current video signal packet without color converting and displaying the partially decoded current video signal packet, if D is greater than the first threshold value and D is less than or equal to a second threshold value, wherein the second threshold value is greater than the first threshold value; and

the video subsystem restarts the playing of the video sequence, if D is greater than the second threshold value.

20. The system of claim 19, wherein the communications link is an ISDN network.

28. The system of claim 11, wherein the sequence numbers are time stamps.

21. A computer-implemented process for synchronizing data streams, comprising the steps of:

(a) generating a first sequence of signal packets of a first type of signals;

(b) stamping each signal packet of the first sequence with a sequence number corresponding to the position of the signal packet in the first sequence;

(c) generating a second sequence of signal packets of a second type of signals; and

(d) stamping each signal packet of the second sequence a corresponding sequence number of the first sequence.

22. The process of claim 21, wherein:

the first sequence of signal packets of the first type of signals comprises an audio sequence of audio signal packets;

the second sequence of signal packets of the second type of signals comprises a video sequence of video signal packets;

step (a) comprises the step of generating the audio sequence;

step (b) comprises the step of stamping each audio signal packet of the audio sequence with a sequence number corresponding to the position of the audio signal packet in the audio sequence;

step (c) comprises the step of generating the video sequence; and

step (d) comprises the step of stamping each video signal packet of the video sequence with a corresponding sequence number of the audio sequence.

23. The process of claim 22, wherein step (d) comprises the step of stamping each video signal packet of the video sequence with the sequence number corresponding to the most recent audio signal packet.

29. The process of claim 21, wherein the sequence numbers are time stamps.

24. A computer system for synchronizing data streams, comprising:

(a) a first subsystem; and

(b) a second subsystem, wherein;

the first subsystem generates a first sequence of signal packets of a first type of signals;

the first subsystem stamps each signal packet of the first sequence with a sequence number corresponding to the position of the signal packet in the first sequence;

the second subsystem generates a second sequence of signal packets of a second type of signals; and

the second subsystem stamps each signal packet of the second sequence with a corresponding sequence number of the first sequence.

25. The system of claim 24, wherein:

the first sequence of signal packets of the first type of signals comprises an audio sequence of audio signal packets;

the second sequence of signal packets of the second type of signals comprises a video sequence of video signal packets;

the first subsystem generates the audio sequence;

the first subsystem stamps each audio signal packet of the audio sequence with a sequence number corresponding to the position of the audio signal packet in the audio sequence;

the second subsystem generates the video sequence; and

the second subsystem stamps each video signal packet of the video sequence with a corresponding sequence number of the audio sequence.

26. The system of claim 25, wherein the second subsystem stamps each video signal packet of the video sequence with the sequence number corresponding to the most recent audio signal packet.

30. The system of claim 24, wherein the sequence numbers are time stamps.