Video conferencing over IP networks

• US 20080034104A1
• Filed: 08/06/2007
• Published: 02/07/2008
• Est. Priority Date: 08/07/2006
• Status: Active Grant

• Current Assignee: Krush Technologies, LLC
• Original Assignee: Cymi Communications, Inc.
• Inventors: Gelfer, Vladislav, Pesherov, Sergey, Kariti, Eran
• Granted Patent: US 8,856,371 B2
• Time in Patent Office: Days
• US Class Current: 709/231
• CPC Class Codes: G06Q 10/10 Office automation; Time management
• CPC Class Codes: H04L 41/509 wherein the managed service relates to media content delivery, e.g. audio, video or TV
• CPC Class Codes: H04L 47/2475 for supporting traffic characterised by the type of applications
• CPC Class Codes: H04L 7/0008 Synchronisation information channels, e.g. clock distribution lines
• CPC Class Codes: H04M 1/7243 with interactive means for internal management of messages
• CPC Class Codes: H04M 3/5307 for recording messages comprising any combination of audio and non-audio components
• CPC Class Codes: H04M 3/5315 where the non-audio components are still images or video still image mailbox systems H04N1/324, intermediate storage of video signals for videophones H04N7/147
• CPC Class Codes: H04N 5/265 Mixing
• CPC Class Codes: H04N 7/152 Multipoint control units therefor

Abstract

A method for communication includes establishing communication links over a packet network between a server and plurality of client computers that are to participate in a video teleconference. The server receives from the client computers uplink audio packets and uplink video packets, which respectively contain audio and video data captured by each of the client computers. The server mixes the audio data from the uplink audio packets so as to create respective streams of mixed audio data for transmission to the client computers and transmits to the client computers downlink audio packets containing the respective streams of mixed audio data. The server relays the video data to the client computers in downlink video packets. The client computers receive and synchronize the video data with the mixed audio data.

Claims

1. A method for communication, comprising:

establishing communication links over a packet network between a server and plurality of client computers that are to participate in a video teleconference;

receiving at the server from the client computers uplink audio packets and uplink video packets, which respectively contain audio and video data captured by each of the client computers;

mixing the audio data from the uplink audio packets at the server so as to create respective streams of mixed audio data for transmission to the client computers;

transmitting from the server to the client computers downlink audio packets containing the respective streams of mixed audio data;

relaying the video data from the server to the client computers in downlink video packets;

receiving and synchronizing the video data with the mixed audio data at the client computers; and

outputting the synchronized video and mixed audio data to a respective user of each of the client computers.

2. The method according to claim 1, wherein establishing the communication links comprises establishing respective first and second communication links between first and second client computers and a server over the packet network using different, respective first and second transport layer protocols.

3. The method according to claim 1, wherein receiving the uplink video packets comprises controlling a quality of the video data conveyed to the server by the client computers by transmitting instructions from the server to the client computers.

4. The method according to claim 3, wherein transmitting the instructions comprises receiving messages from the client computers that are indicative of downlink bandwidth availability for transmission from the server to the client computers, and determining the quality of the video data responsively to the downlink bandwidth availability.

5. The method according to claim 4, wherein receiving the messages comprises detecting, at one of the client computers, a delay in receiving one or more of the downlink audio and video packets, and informing the server of the delay.

6. The method according to claim 5, wherein transmitting the instructions comprises instructing the clients to reduce the quality of the video data transmitted in the uplink video packets responsively to detecting the delay at the one of the clients.

7. The method according to claim 3, wherein controlling the quality comprises instructing the client computers to increase or decrease at least one quality parameter selected from a group of quality parameters consisting of an image resolution, a degree of image compression, a frame rate and a bandwidth.

8. The method according to claim 1, wherein receiving the uplink packets comprises detecting, at the server, a delay in the audio data, and eliminating an interval of silent audio data in order to compensate for the delay.

9. The method according to claim 8, wherein eliminating the interval comprises marking, at one or more of the client computers, at least one block of the audio data as a silent block, and eliminating the silent block from the mixed audio data.

10. The method according to claim 1, wherein each of the downlink video packets contains the video data captured by a respective one of the client computers.

11. The method according to claim 10, wherein outputting the synchronized video and mixed audio data comprises displaying the video data captured by the respective one of the client computers in a respective window among multiple windows displayed by each of the client computers.

12. The method according to claim 11, wherein synchronizing the video data comprises controlling, at the client computer, the multiple windows so that the video data conveyed from each of the client computers are synchronized with the mixed audio data.

13. The method according to claim 10, wherein relaying the video data comprises passing the video data from the uplink video packets to the downlink video packets without transcoding of the video data at the server.

14. The method according to claim 1, wherein receiving the uplink audio and video packets comprises receiving at the server synchronization data from each of the client computers, and comprising generating synchronization information at the server based on the synchronization data, and transmitting the synchronization information from the server to the client computers for use in synchronizing the video data with the mixed audio data.

15. The method according to claim 1, wherein the plurality of client computers comprises at least three client computers that participate in the video teleconference.

16-21. -21. (canceled)

22. Communication apparatus, comprising:

a plurality of client computers, which are connected to communicate over a packet network and are configured to capture audio and video data and to transmit over the packet network uplink audio packets and uplink video packets, which respectively contain the audio and video data; and

a conference server, which is coupled to establish communication links over the packet network with the client computers that are to participate in a video teleconference and to receive the uplink audio packets and uplink video packets over the communication links, and which is configured to mix the audio data from the uplink audio packets so as to create respective streams of mixed audio data for transmission to the client computers, and to transmit to the client computers downlink audio packets containing the respective streams of mixed audio data while relaying the video data from the uplink video packets to the client computers in downlink video packets,

wherein the client computers are configured to synchronize the video data with the mixed audio data, and to output the synchronized video and mixed audio data to a respective user of each of the client computers.

23. The apparatus according to claim 22, wherein the conference server is configured to establish respective first and second communication links with first and second client computers over the packet network using different, respective first and second transport layer protocols.

24. The apparatus according to claim 22, wherein the conference server is configured to control a quality of the video data conveyed to the server by the client computers by transmitting instructions to the client computers.

25. The apparatus according to claim 24, wherein the conference server is coupled to receive messages from the client computers that are indicative of downlink bandwidth availability for transmission from the server to the client computers, and to determine the quality of the video data responsively to the downlink bandwidth availability.

26. The apparatus according to claim 25, wherein the client computers are configured to determine the downlink bandwidth availability by detecting a delay in receiving one or more of the downlink audio and video packets, and to inform the server of the delay.

27. The apparatus according to claim 26, wherein the conference server is configured to instruct the clients to reduce the quality of the video data transmitted in the uplink video packets responsively to detecting the delay at one or more of the clients.

28. The apparatus according to claim 24, wherein the client computers are configured to control the quality by increasing or decreasing at least one quality parameter selected from a group of quality parameters consisting of an image resolution, a degree of image compression, a frame rate and a bandwidth.

29. The apparatus according to claim 22, wherein the conference server is configured to detect a delay in the audio data, and to eliminate an interval of silent audio data in order to compensate for the delay.

30. The apparatus according to claim 29, wherein the client computers are configured to mark at least one block of the audio data as a silent block, and wherein the conference is configured to eliminate the silent block from the mixed audio data.

31. The apparatus according to claim 22, wherein each of the downlink video packets contains the video data captured by a respective one of the client computers.

32. The apparatus according to claim 31, wherein the client computers are configured to display the video data captured by the respective one of the client computers in a respective window among multiple windows displayed by each of the client computers.

33. The apparatus according to claim 32, wherein the client computers are configured to control the multiple windows so that the video data conveyed from each of the client computers are synchronized with the mixed audio data.

34. The apparatus according to claim 31, wherein the conference server is configured to pass the video data from the uplink video packets to the downlink video packets without transcoding of the video data at the server.

35. The apparatus according to claim 22, the client computers are configured to transmit synchronization data to the conference server, and wherein the conference server is configured to generate synchronization information on the synchronization data, and to transmit the synchronization information to the client computers for use in synchronizing the video data with the mixed audio data.

36. The apparatus according to claim 22, wherein the plurality of client computers comprises at least three client computers that participate in the video teleconference.

37. A conference server, comprising:

a network interface, which is coupled to establish communication links over a packet network with a plurality of client computers that are to participate in a video teleconference, and to receive from the client computers uplink audio packets and uplink video packets, which respectively contain audio and video data captured by each of the client computers; and

a processor, which is configured to mix the audio data from the uplink audio packets so as to create respective streams of mixed audio data for transmission to the client computers, and to transmit to the client computers via the network interface downlink audio packets containing the respective streams of mixed audio data while relaying the video data from the uplink video packets to the client computers in downlink video packets, for synchronization by the client computers with the mixed audio data.

38-43. -43. (canceled)

44. A computer software product, comprising a computer-readable medium in which program instructions are stored, which instructions, when read by a server, cause the server to establish communication links over a packet network with a plurality of client computers that are to participate in a video teleconference, and to receive from the client computers uplink audio packets and uplink video packets, which respectively contain audio and video data captured by each of the client computers,

wherein the instructions cause the server to mix the audio data from the uplink audio packets so as to create respective streams of mixed audio data for transmission to the client computers, and to transmit to the client computers via the network interface downlink audio packets containing the respective streams of mixed audio data while relaying the video data from the uplink video packets to the client computers in downlink video packets, for synchronization by the client computers with the mixed audio data.

45. (canceled)

46. A client computer, comprising:

a user interface; and

a processor, which is configured to establish a communication link over a packet network with a conference server so as to participate in a video teleconference, and to transmit uplink audio packets and uplink video packets, which respectively contain audio and video data captured by the client computer,

wherein the processor is configured to receive from the server downlink audio packets containing the a stream of mixed audio data generated by the server and to receive downlink video packets containing the video data transmitted by other client computers in the video teleconference, and to synchronize the video data with the mixed audio data for output via the user interface.

47. (canceled)

48. (canceled)