Audio-video packet synchronization at network gateway

• US 7,043,749 B1
• Filed: 02/24/1999
• Issued: 05/09/2006
• Est. Priority Date: 02/27/1998
• Status: Expired due to Term

• Current Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Original Assignee: Tandberg Telecom AS (Cisco Systems, Inc.)
• Inventors: Davies, Steven Gareth
• Primary Examiner(s): Bui, Kieu-Oanh
• Application Number:
  US09/623,058
• Time in Patent Office: 2,631 Days
• Field of Search: 725/120, 725/136, 725/138, 725/62, 725/63, 725/73, 725/86, 725/91, 725/105, 725/114, 725/118, 348/181, 348/189, 348/192, 348/484, 348/515, 348/516, 348/518, 370/352, 370/355, 370/356, 370/389, 370/392, 370/465, 370/503, 715/716, 715/721, 709/224, 709/233, 455/3.01, 455/403, 455/550, 455/553.1
• US Class Current: 725/120
• CPC Class Codes: H04L 12/64 Hybrid switching systems
• CPC Class Codes: H04L 12/66 Arrangements for connecting between networks having differing types of switching systems, e.g. gateways
• CPC Class Codes: H04L 65/103 in the network
• CPC Class Codes: H04L 65/104 in the network
• CPC Class Codes: H04L 65/1101 Session protocols
• CPC Class Codes: H04L 65/613 for the control of the source by the destination control signals issued by the client directed to the server or network components specially adapted for selective content distribution H04N21/637
• CPC Class Codes: H04L 65/756 adapting media to device capabilities
• CPC Class Codes: H04L 65/80 Responding to QoS
• CPC Class Codes: H04L 9/40 Network security protocols
• CPC Class Codes: H04N 21/43072 of multiple content streams on the same device
• CPC Class Codes: H04N 21/64784 Data processing by the network data processing in packet switching systems H04L12/56; flow control in packet networks H04L47/10; intermediate storage or scheduling H04L49/90; provisioning of proxy services in data packet switching networks H04L67/56
• CPC Class Codes: H04N 7/152 Multipoint control units therefor
• CPC Class Codes: H04Q 11/0435 Details
• CPC Class Codes: H04Q 2213/13098 Mobile subscriber
• CPC Class Codes: H04Q 2213/13196 Connection circuit/link/trunk/junction, bridge, router, gateway
• CPC Class Codes: H04Q 2213/13204 Protocols
• CPC Class Codes: H04Q 2213/13209 ISDN
• CPC Class Codes: H04Q 2213/13213 Counting, timing circuits
• CPC Class Codes: H04Q 2213/1322 PBX
• CPC Class Codes: H04Q 2213/13248 Multimedia
• CPC Class Codes: H04Q 2213/1325 : Priority service
• CPC Class Codes: H04Q 2213/13296 : Packet switching, X.25, frame relay
• CPC Class Codes: H04Q 2213/13337 : Picturephone, videotelephony
• CPC Class Codes: H04Q 2213/1336 : Synchronisation
• CPC Class Codes: H04Q 2213/13389 : LAN, internet

Abstract

A method and apparatus for synchronizing sound and images in a real-time multimedia communication, such as an audio-video telephone call, through a network gateway, when the source and/or the destination of the audio signals, and optionally also the video signals, are from and/or to separate audio and video communication devices.

Claims

1. A method of calibrating a network gateway in order to synchronize audible and visible outputs from a multimedia terminal comprising an audio communication device and a separate video communication device, when the signal to the audio device, and optionally also the signal to the video device, is/are routed through a network gateway that imposes an intrinsic gateway processing delay on at least one signal routed through the gateway, comprising the steps of:

i) providing an audio signal and a video signal;

ii) supplying as an input to the gateway the audio signal and optionally also the video signal so that said signal(s) is/are routed through the gateway to the multimedia terminal;

iii) detecting any mismatch in the synchronization between the audible and visible outputs from the audio device and the video device owing at least partly to the intrinsic gateway processing delay(s);

iv) adjusting a variable delay in the gateway on a signal routed through the gateway until the audible and visible outputs from the separate devices are effectively synchronized; and

,

v) storing the variable delay in the gateway for future use.

2. A method of calibrating a network gateway as claimed in claim 1, in which the video device is an integrated audio and video telephony device which forms with the separate audio communication device a combined multimedia terminal.

3. A method of calibrating a network gateway as claimed in claim 1, in which the audio and video signals are effectively synchronized with each other prior to at least the audio signal being supplied as an input to the gateway.

4. A method of calibrating a network gateway as claimed in claim 1, in which the video signal is not routed through the gateway, but a data signal from the video device is routed through the gateway.

5. A method of calibrating a network gateway as claimed in claim 4, in which step i) includes providing a synchronization marker in the audio signal and video signal, and step vii) includes detecting a mismatch between the synchronization markers.

6. A method of calibrating a network gateway as claimed in claim 1, when the signals to and from both the audio device and the video device are routed through the gateway and when signals to said devices may both be looped back to the gateway from said devices, at least one of said devices imposing an intrinsic device transmission delay on a signal looped back to the gateway, comprising the steps after step iv) of:

vi) looping back the signals from the audio device and video device to the gateway;

vii) detecting any mismatch in the synchronization between the looped back signals from the audio device and video device at the gateway owing to the intrinsic device transmission delay(s);

viii) adjusting a variable device transmission delay in the gateway so that the looped back signals at the gateway are effectively synchronized; and

,

ix) storing the variable device transmission delay in the gateway for future use.

7. A method of calibrating a network gateway as claimed in claim 1, when the signals to and from both the audio device and video device are routed through the gateway, at least one of said devices imposing an intrinsic device transmission delay on a signal transmitted to the gateway, comprising the steps after step iv) of:

x) providing from the audio device and video device, respectively, an audio signal and a video signal;

xi) supplying said signals as an input to the gateway so that said signals are routed through the gateway to a second multimedia terminal for which the gateway has already been calibrated according to the method of claim 1, the second multimedia terminal comprising a first-calibrated audio communication device and a first-calibrated video communication device;

xii) detecting any mismatch in the synchronization between the audible and visible outputs from the first-calibrated audio device and first-calibrated video device owing to the intrinsic device transmission delays of the audio device and video device;

xiii) adjusting a variable device transmission delay in the gateway on a signal routed through the gateway until the audible and visible outputs from the first-calibrated audio device and first-calibrated video device are effectively synchronized; and

,

xiv) storing the variable device transmission delay in the gateway for future use.

8. A method of calibrating a network gateway as claimed in claim 7, in which the calibration once achieved is maintained when the multimedia terminal is in audio and video communication with an H.323 compliant multimedia terminal via the gateway, wherein the gateway determines a relative time difference between itself and the H.323 compliant multimedia terminal in order to increase or decrease a device transmission delay of the H.323 complaint multimedia terminal in order to synchronize the audible and visible outputs from the multimedia terminal.

9. A method of calibrating a network gateway as claimed in claim 1, in which the calibration once achieved is maintained by a user entering data into a data entry means associated with either the audio device or the video device, said data being interpreted by the network gateway as a command to increase or decrease a variable delay in the gateway in order to synchronize the audible and visible outputs from the multimedia terminal.

10. A multimedia communication system, comprising a network gateway and one or more multimedia terminal(s), each terminal comprising an audio communication device and a separate video communication device, wherein the gateway has an input for receiving a signal from the audio device, and optionally also an input for receiving a signal from the video device so that said signal(s) may be routed through the gateway, wherein the network gateway has been calibrated according to the method of claim 1 in order to synchronize audible and visible outputs from, respectively, the audio device and video device.

11. The method of claim 1, wherein said gateway receives a signal formatted according to a first standard, and transcodes and repackages the signal to be formatted in a second standard, said second standard different from said first standard.

12. The method of claim 1, wherein said variable delay is dynamically adjusted.

13. A method of calibrating a network gateway in order to synchronize audible and visible outputs from a multimedia terminal comprising an audio communication device and a separate video communication device, when the signal to the audio device and the signal to the video device are routed through a network gateway that imposes an intrinsic gateway processing delay on at least one signal routed through the gateway, comprising the steps of:

a) providing a first test audio signal with a marker;

b) providing from an integrated audio and video communication device a video signal and a first audio signal sourced from the first test audio signal;

c) providing from a separate audio device a second audio signal sourced from the first test audio signal;

d) supplying as an input to the gateway the first audio signal, the second audio signal and the video signal so that said signals are routed through the gateway to the multimedia terminal;

e) using the gateway to detect the relative delay between the first audio signal and the second audio signal owing at least partly to the intrinsic gateway processing delay(s);

f) determining a skew delay at the gateway between the first audio signal and the video signal from the integrated audio and video communication device, owing at least partly to the intrinsic gateway processing delay(s);

g) summing said relative delay and the skew delay to generate a variable device transmission delay between the video signal and the second audio signal; and

,

h) storing the variable device transmission delay in the gateway for future use.

14. A method of calibrating a network gateway as claimed in claim 13, in which following step h), the method comprises the steps of:

i) providing from the gateway a second audio test signal with marker;

j) transmitting the second audio test signal from the gateway to the separate audio device;

k) providing an audio coupling between the separate audio device and the audio device of the integrated audio and video communication device;

l) looping back the second audio test signal from the separate audio device through the integrated audio and video communication device to the gateway;

m) determining a first loop back delay in the second audio test signal between the transmission and reception of said signal at the gateway, including any intrinsic gateway processing delay(s); and

,

n) storing the first loop back delay in the gateway for future use.

15. A method of calibrating a network gateway as claimed in claim 14, in which following step n), the method comprises the steps of:

o) providing from the gateway and a third audio test signal with marker and a video test signal, said signals being effectively synchronized;

p) transmitting the third audio test signal and video test signals from the gateway to the integrated audio and video communication device;

q) providing an audio coupling between the separate audio device and the audio device of the integrated audio and video communication device;

r) looping-back the third audio test signal from the integrated audio and video communication device through the separate audio device to the gateway;

s) determining a second loop back delay in the third audio test signal between the transmission and reception of said signal at the gateway, including any intrinsic gateway processing delay(s); and

,

t) storing the second loop back delay in the gateway for future use.

16. A method of calibrating a network gateway as claimed in claim 15, in which following step t), the method comprises the steps of:

u) calculating a variable sensory output delay from a sum of the variable device transmission delay, first loop back delay and second loop back delay; and

,

v) storing the variable sensory output delay in the gateway for future use.

17. A method of calibrating a network gateway as claimed in claim 14, in which the calibration once achieved is maintained when the multimedia terminal is in audio and video communication with an H.323 compliant multimedia terminal via the gateway, wherein the gateway determines a relative time difference between itself and the H.323 compliant multimedia terminal in order to increase or decrease a device transmission delay of the H.323 complaint multimedia terminal in order to synchronize the audible and visible outputs from the multimedia terminal.

18. A method of calibrating a network gateway as claimed in claim 13, in which the calibration once achieved is maintained by a user entering data into a data entry means associated with either the audio device or the video device, said data being interpreted by the network gateway as a command to increase or decrease a variable delay in the gateway in order to synchronize the audible and visible outputs from the multimedia terminal.

19. A multimedia communication system, comprising a network gateway and one or more multimedia terminal(s), each terminal comprising an audio communication device and a separate video communication device, wherein the gateway has an input for receiving a signal from the audio device, and optionally also an input for receiving a signal from the video device so that said signal(s) may be routed through the gateway, wherein the network gateway has been calibrated according to the method of claim 13 in order to synchronize audible and visible outputs from, respectively, the audio device and video device.