Video conferencing systems

• US 9,088,695 B2
• Filed: 12/16/2013
• Issued: 07/21/2015
• Est. Priority Date: 06/16/2011
• Status: Active Grant

• Current Assignee: Claret European Specialty Lending Company III SA RL
• Original Assignee: Startleaf Limited
• Inventors: Gwilt, David John, Nancekievill, Alexander Edward
• Primary Examiner(s): Anwah, Olisa
• Application Number:
  US14/108,039
• Publication Number:
  US 20140168353A1
• Time in Patent Office: 582 Days
• Field of Search: 379/93.21, 379/158, 379/202.01, 348/14.08
• US Class Current: 1/1
• CPC Class Codes: H04M 2201/22 Synchronisation circuits
• CPC Class Codes: H04M 2201/50 Telephonic communication in combination with video communication
• CPC Class Codes: H04M 3/567 Multimedia conference systems
• CPC Class Codes: H04M 3/568 audio processing specific to telephonic conferencing, e.g. spatial distribution, mixing of participants echo suppression in two-way loud-speaking telephone systems H04M9/02; sound field processing per se H04S7/30
• CPC Class Codes: H04M 7/0027 Collaboration services where a computer is used for data transfer and the telephone is used for telephonic communication
• CPC Class Codes: H04M 7/0039 where the data service is provided by a stream of packets which are rendered in real time by the receiving terminal network streaming of media packets H04L65/60
• CPC Class Codes: H04N 7/147 Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals selecting H04Q
• CPC Class Codes: H04N 7/15 Conference systems

Abstract

We describe a method of synchronizing a teleconference comprising audio carried on a telephone network and a digital data stream carried on a computer network. Audio at a first node is characterized to determine audio characterizing data. A digital data stream for the teleconference is also input at the first node and the audio characterizing data is inserted into the digital data stream and forwarded over the network. The audio and digital data streams are received separately at a second node of the system, and the audio characterizing data is extracted from the digital data stream and processed in conjunction with the received audio to determine a first-second node time offset. The audio and digital data streams are then synchronizing by adjusting, for example, a relative time delay of the received audio and digital streams at the second node responsive to the determined time offset.

Claims

1. A method of synchronising a teleconference comprising audio carried on a telephone network and a digital data stream carried on a computer network, the method comprising:

inputting first audio for said teleconference at a first node of a teleconference system;

processing said first audio to determine first audio characterising data characterising said first audio;

providing said first audio to a telephone network providing an audio teleconference;

inputting at said first node a first digital data stream for said teleconference;

inserting said first audio characterising data into said first digital data stream;

forwarding said first digital data stream over said computer network;

receiving said first audio at a second node of said teleconference system coupled to said telephone network, wherein said second node is a participant in said audio teleconference;

receiving said first digital data stream at said second node of said teleconference system via said computer network;

extracting said first audio characterising data from said digital data stream at said second node;

processing said received first audio at said second node in conjunction with said extracted first audio characterising data to determine a first-second node time offset between said first audio received at said second node and said first digital data stream received at said second node; and

synchronising said received first audio and said received first digital data stream by adjusting one or both of i) a time delay of one of said received first audio and said received first digital data stream with respect to the other at said second node responsive to said determined first-second node time offset; and

ii) a time delay of one said first audio and said first digital data stream with respect to the other at said first node responsive to said determined first-second node time offset.

2. A method as claimed in claim 1 wherein said processing of said received first audio comprises determining received first audio characterising data characterising said received first audio, and correlating said received first audio characterising data and said first audio characterising data to determine said first-second node time offset.

3. A method as claimed in claim 2 wherein said first audio and said received first audio characterising data each comprise data identifying a pattern of sound level variation, wherein said correlating comprises comparing respective said patterns of second level variation, and wherein one or both of the respective said patterns is weighted such that sound levels below a threshold level have a reduced significance in said comparing.

4. A method as claimed in claim 1 wherein said teleconference system further comprises a third node, the method further comprising:

receiving at said second node third audio and a third digital data stream from said third node via said telephone network and said computer network respectively, wherein said third digital data stream includes third audio characterising data characterising said third audio;

processing said received third audio at said second node in conjunction with said extracted third audio characterising data to determine a second-third node time offset between said third audio received at said second node and said third digital data stream received at said second node; and

synchronising said received first audio, said received first digital data stream, said received third audio, and said received third digital data stream, using said first-second node time offset and said second-third node time offset.

5. A method as claimed in claim 4 wherein said synchronising comprises one or both of:

i) adjusting a time delay of one of said first audio and said first digital data stream with respect to the other at said first node responsive to said data dependent on said first-second node time offset communicated to said first node, and ii) adjusting a time delay of one of said third audio and said third digital data stream with respect to the other at said third node responsive to said data dependent on said second-third node time offset communicated to said third node.

6. A method as claimed in claim 4 wherein said synchronising comprises communicating data dependent on said first-second node time offset and said second-third node time offset to said first and third nodes respectively.

7. A method as claimed in claim 6 wherein said synchronising comprises determining, at a said node, from time offset dependent data communicated to said second node from other said nodes, a maximum time offset to apply to a said audio or digital data stream received at the node from a said other node;

and

applying said maximum time offset to synchronise audio from said other nodes at said second node; and

synchronising said digital data streams from said other nodes to said synchronised audio.

8. A method as claimed in claim 4 wherein said synchronising comprises adjusting, at said second node, a delay in one or both of said first and third digital data streams.

9. A method as claimed in claim 1 wherein a said digital data stream comprises a video data stream, and wherein said audio characterising data changes no faster than every half-a-frame duration of video in said video data stream.

10. A physical carrier carrying processor control code for:

inputting first audio for said teleconference at a first node of a teleconference system;

processing said first audio to determine first audio characterising data characterising said first audio;

providing said first audio to a telephone network providing an audio teleconference;

inputting at said first node a first digital data stream for said teleconference;

inserting said first audio characterising data into said first digital data stream;

forwarding said first digital data stream over said computer network;

receiving said first audio at a second node of said teleconference system coupled to said telephone network, wherein said second node is a participant in said audio teleconference;

receiving said first digital data stream at said second node of said teleconference system via said computer network;

extracting said first audio characterising data from said digital data stream at said second node;

processing said received first audio at said second node in conjunction with said extracted first audio characterising data to determine a first-second node time offset between said first audio received at said second node and said first digital data stream received at said second node; and

synchronising said received first audio and said received first digital data stream by adjusting one or both of i) a time delay of one of said received first audio and said received first digital data stream with respect to the other at said second node responsive to said determined first-second node time offset; and

ii) a time delay of one said first audio and said first digital data stream with respect to the other at said first node responsive to said determined first-second node time offset.

11. A system for synchronising a teleconference comprising audio carried on a telephone network and a digital data stream carried on a computer network, the system comprising a plurality of nodes each having a first interface for a phone, a telephone network interface for connecting to a telephone network, and at least one computer network interface for connecting to a computer network, and wherein said nodes are configured to:

input first audio for said teleconference at a first node of a teleconference system;

process said first audio to determine first audio characterising data characterising said first audio;

provide said first audio to a telephone network providing an audio teleconference;

input at said first node a first digital data stream for said teleconference;

insert said first audio characterising data into said first digital data stream;

forward said first digital data stream over said computer network;

receive said first audio at a second node of said teleconference system coupled to said telephone network, wherein said second node is a participant in said audio teleconference;

receive said first digital data stream at said second node of said teleconference system via said computer network;

extract said first audio characterising data from said digital data stream at said second node;

process said received first audio at said second node in conjunction with said extracted first audio characterising data to determine a first-second node time offset between said first audio received at said second node and said first digital data stream received at said second node; and

synchronise said received first audio and said received first digital data stream by adjusting one or both of i) a time delay of one of said received first audio and said received first digital data stream with respect to the other at said second node responsive to said determined first-second node time offset; and

ii) a time delay of one said first audio and said first digital data stream with respect to the other at said first node responsive to said determined first-second node time offset.

12. A node unit for the system of claim 11, for synchronising a teleconference comprising audio carried on a telephone network and a digital data stream carried on a computer network, the node unit comprising a first interface for a phone, a telephone network interface for connecting to a telephone network and at least one computer network interface for connecting to a computer network, a time offset measurement system coupled to said computer network interface and to said telephone network interface, to determine a time offset between audio on said telephone network interface and data characterising said audio in a digital data stream on said computer network interface, and wherein said node unit is configured to perform one or both of i) local control of a relative time offset between audio received from said telephone network interface and said digital data steam, and ii) transmission of data representing said time offset over said computer network to at least one other said node unit.

13. A node unit as claimed in claim 12 further comprising a time delay control system coupled to said computer network interface to control an audio time delay between said first interface and said telephone network interface responsive to time offset control data received over said computer network interface.