Video conferencing over IP networks

• US 9,185,347 B2
• Filed: 10/06/2014
• Issued: 11/10/2015
• Est. Priority Date: 08/07/2006
• Status: Expired due to Fees

• Current Assignee: Krush Technologies, LLC
• Original Assignee: ooVoo LLC (Oovoo Holdings LLC)
• Inventors: Kariti, Eran, Pesherov, Sergey, Gelfer, Vladislav
• Primary Examiner(s): Eskandarnia, Arvin
• Application Number:
  US14/507,405
• Publication Number:
  US 20150092014A1
• Time in Patent Office: 400 Days
• Field of Search: 709/231
• US Class Current: 1/1
• CPC Class Codes: G06Q 10/10 Office automation; Time management
• CPC Class Codes: H04L 41/509 wherein the managed service relates to media content delivery, e.g. audio, video or TV
• CPC Class Codes: H04L 47/2475 for supporting traffic characterised by the type of applications
• CPC Class Codes: H04L 7/0008 Synchronisation information channels, e.g. clock distribution lines
• CPC Class Codes: H04M 1/7243 with interactive means for internal management of messages
• CPC Class Codes: H04M 3/5307 for recording messages comprising any combination of audio and non-audio components
• CPC Class Codes: H04M 3/5315 where the non-audio components are still images or video still image mailbox systems H04N1/324, intermediate storage of video signals for videophones H04N7/147
• CPC Class Codes: H04N 5/265 Mixing
• CPC Class Codes: H04N 7/152 Multipoint control units therefor

Abstract

A method for communication includes establishing communication links over a packet network between a server and plurality of client computers that are to participate in a video teleconference. The server receives from the client computers uplink audio packets and uplink video packets, which respectively contain audio and video data captured by each of the client computers. The server mixes the audio data from the uplink audio packets so as to create respective streams of mixed audio data for transmission to the client computers and transmits to the client computers downlink audio packets containing the respective streams of mixed audio data. The server relays the video data to the client computers in downlink video packets. The client computers receive and synchronize the video data with the mixed audio data.

Claims

1. A method for communication, comprising:

establishing communication links over a packet network;

receiving uplink audio packets and uplink video packets, which respectively contain audio and video data;

mixing the audio data from the uplink audio packets to create respective streams of mixed audio data for transmission;

transmitting downlink audio packets containing the respective streams of mixed audio data;

relaying the video data in downlink video packets;

analyzing relative time differences between the mixed audio data and each relayed video data stream;

generating at least one corresponding synchronization packet based on the analyzed relative time differences between mixed audio data and each relayed video data stream; and

,

transmitting the at least one corresponding synchronization packet containing synchronization information for synchronizing the relayed video data in the downlink video packets with the downlink audio packets containing the respective streams of mixed audio data in addition to transmitting the downlink video packets and the downlink audio packets.

2. The method according to claim 1, wherein receiving the uplink video packets comprises controlling a quality of the video data by transmitting instructions.

3. The method according to claim 2, wherein transmitting the instructions comprises receiving messages that are indicative of downlink bandwidth availability for transmission, and determining the quality of the video data responsively to the downlink bandwidth availability.

4. The method according to claim 3, wherein receiving the messages comprises detecting a delay in receiving one or more of the downlink audio and video packets.

5. The method according to claim 4, wherein transmitting the instructions comprises instructions to reduce the quality of the video data transmitted in the uplink video packets responsively to detecting the delay in receiving one or more of the downlink audio and video packets.

6. The method according to claim 2, wherein controlling the quality comprises instructing an increase or decrease of at least one quality parameter selected from a group of quality parameters consisting of an image resolution, a degree of image compression, a frame rate and a bandwidth.

7. The method according to claim 1, wherein receiving the uplink packets comprises detecting a delay in the audio data and eliminating an interval of silent audio data in order to compensate for the delay.

8. The method according to claim 7, wherein eliminating the interval comprises marking at least one block of the audio data as a silent block and eliminating the silent block from the mixed audio data.

9. The method according to claim 1, wherein relaying the video data comprises passing the video data from the uplink video packets to the downlink video packets without transcoding of the video data at the server.

10. The method according to claim 1, wherein receiving the uplink audio and video packets comprises receiving synchronization data, generating synchronization information on the synchronization data, and transmitting the synchronization information for use in synchronizing the video data with the mixed audio data.

11. A communication apparatus, comprising:

a server, which is coupled to establish communication links over a packet network with remote computers that are to participate in a video teleconference and to receive uplink audio packets and uplink video packets, which respectively comprise audio and video data, over the communication links;

wherein the server is configured to mix the audio data from the uplink audio packets so as to create respective streams of mixed audio data for transmission to the remote computers, and to generate at least one synchronization packet containing synchronization information, and to transmit to the remote computers downlink audio packets containing the respective streams of mixed audio data in addition to the at least one synchronization packet while relaying the video data from the uplink video packets to the client computers in downlink video packets.

12. The apparatus according to claim 11, wherein the server is configured to establish respective first and second communication links with first and second remote computers over the packet network using different, respective first and second transport layer protocols.

13. The apparatus according to claim 11, wherein the server is configured to control a quality of the video data conveyed to the server by transmitting instructions to the remote computers.

14. The apparatus according to claim 13, wherein the server is coupled to receive messages that are indicative of downlink bandwidth availability for transmission from the server, and to determine the quality of the video data responsively to the downlink bandwidth availability.

15. The apparatus according to claim 14, wherein the server is coupled to receive messages that are indicative of a delay in receiving one or more of the downlink audio and video packets by the remote computers.

16. The apparatus according to claim 15, wherein the server is configured to instruct the remote computers to reduce the quality of the video data transmitted in the uplink video packets responsively to detecting the delay at one or more of the clients.

17. The apparatus according to claim 11, wherein the server is configured to detect a delay in the audio data, and to eliminate an interval of silent audio data in order to compensate for the delay.

18. The apparatus according to claim 11, wherein each of the downlink video packets contains the video data captured by a respective one of the remote computers.

19. The apparatus according to claim 18, wherein the server is configured to pass the video data from the uplink video packets to the downlink video packets without transcoding of the video data at the server.

20. The apparatus according to claim 11, wherein the server is configured to receive synchronization data from the remote computers, generate the synchronization information based on the synchronization data, and transmit the synchronization information to the remote computers.

21. A server, comprising:

a network interface, which is coupled to establish communication links over a packet network with a plurality of remote computers that are to participate in a video teleconference, and to receive from the remote computers uplink audio packets and uplink video packets, which respectively contain audio and video data captured by each of the remote computers; and

a processor, which is configured to mix the audio data from the uplink audio packets so as to create respective streams of mixed audio data for transmission to the remote computers and to generate at least one synchronization packet containing synchronization information, and to transmit to the remote computers via the network interface downlink audio packets containing the respective streams of mixed audio data in addition to the at least one synchronization packet while relaying the video data from the uplink video packets to the remote computers in downlink video packets.

22. A computer program product, comprising a computer usable medium having a computer readable program code embodied therein, said computer readable program code adapted to be executed to implement a method for implementing a method for communication, said method comprising:

establishing communication links over a packet network;

receiving uplink audio packets and uplink video packets, which respectively contain audio and video data;

mixing the audio data from the uplink audio packets to create respective streams of mixed audio data for transmission;

transmitting downlink audio packets containing the respective streams of mixed audio data;

relaying the video data in downlink video packets;

analyzing relative time differences between the mixed audio data and each relayed video data stream;

generating at least one corresponding synchronization packet based on the analyzed relative time differences between mixed audio data and each relayed video data stream; and

,

transmitting the at least one corresponding synchronization packet containing synchronization information for synchronizing the relayed video data in the downlink video packets with the downlink audio packets containing the respective streams of mixed audio data in addition to transmitting the downlink video packets and the downlink audio packets.

23. The computer program product according to claim 22, wherein receiving the uplink video packets comprises controlling a quality of the video data by transmitting instructions.

24. The computer program product according to claim 23, wherein transmitting the instructions comprises receiving messages that are indicative of downlink bandwidth availability for transmission, and determining the quality of the video data responsively to the downlink bandwidth availability.

25. The computer program product according to claim 24, wherein receiving the messages comprises detecting a delay in receiving one or more of the downlink audio and video packets.

26. The computer program product according to claim 25, wherein transmitting the instructions comprises instructions to reduce the quality of the video data transmitted in the uplink video packets responsively to detecting the delay in receiving one or more of the downlink audio and video packets.

27. The computer program product according to claim 23, wherein controlling the quality comprises instructing an increase or decrease of at least one quality parameter selected from a group of quality parameters consisting of an image resolution, a degree of image compression, a frame rate and a bandwidth.

28. The computer program product according to claim 22, wherein receiving the uplink packets comprises detecting a delay in the audio data and eliminating an interval of silent audio data in order to compensate for the delay.

29. The computer program product according to claim 28, wherein eliminating the interval comprises marking at least one block of the audio data as a silent block and eliminating the silent block from the mixed audio data.

30. The computer program product according to claim 22, wherein relaying the video data comprises passing the video data from the uplink video packets to the downlink video packets without transcoding of the video data at the server.

31. The computer program product according to claim 22, wherein receiving the uplink audio and video packets comprises receiving synchronization data, generating synchronization information on the synchronization data, and transmitting the synchronization information for use in synchronizing the video data with the mixed audio data.

32. A method for communication, comprising:

establishing communication links over a packet network;

transmitting uplink audio packets and uplink video packets, which respectively contain audio and video data;

receiving downlink audio packets containing respective streams of mixed audio data, wherein the downlink audio packets contain mixed audio data from at least one remote computer;

receiving downlink video packets, which comprise video data from the at least one remote computer; and

,

receiving at least one corresponding synchronization packet comprising synchronization information based on relative time differences between the mixed audio data and the video data from the at least one remote computer for synchronizing the video data in the downlink video packets with the downlink audio packets containing the respective streams of mixed audio data in addition to receiving the downlink video packets and the downlink audio packets.

33. The method according to claim 32, wherein receiving at least one corresponding synchronization packet comprises receiving messages that are indicative of downlink bandwidth availability for transmission.

34. The method according to claim 33, wherein receiving the messages comprises detecting a delay in receiving one or more of the downlink audio and video packets, and transmitting the delay.

35. The method according to claim 34, wherein synchronizing the video data comprises controlling the multiple windows so that the video data conveyed from each of the remote computers are synchronized with the mixed audio data.

36. The method according to claim 35, wherein controlling the quality comprises increasing or decreasing at least one quality parameter selected from a group of quality parameters consisting of an image resolution, a degree of image compression, a frame rate and a bandwidth.

37. The method according to claim 36, wherein controlling the quality comprises eliminating an interval of silent audio data in order to compensate for the delay by marking at least one block of the audio data as a silent block, and eliminating the silent block from the mixed audio data.

38. The method according to claim 37, wherein each of the downlink video packets contains the video data from each respective at least one remote computer.

39. The method according to claim 32 further comprising the steps of:

receiving and synchronizing the video data with the mixed audio data based on synchronization data from the received synchronization packet; and

outputting the synchronized video and mixed audio data to a respective user of each of the remote computers wherein outputting the synchronized video and mixed audio data comprises displaying the video data captured by the respective one of the remote computers in a respective window among multiple windows displayed by each of the remote computers.

40. The method according to claim 39, wherein synchronizing the video data comprises controlling the multiple windows so that the video data conveyed from each of the remote computers are synchronized with the mixed audio data.

41. A computer program product, comprising a computer usable medium having a computer readable program code embodied therein, said computer readable program code adapted to be executed to implement a method for implementing a method for communication, said method comprising:

establishing communication links over a packet network;

transmitting uplink audio packets and uplink video packets, which respectively contain audio and video data;

receiving downlink audio packets containing respective streams of mixed audio data, wherein the downlink audio packets contain mixed audio data from at least one remote computer;

receiving downlink video packets, which comprise video data from the at least one remote computer; and

,

receiving at least one corresponding synchronization packet comprising synchronization information based on relative time differences between the mixed audio data and the video data from the at least one remote computer for synchronizing the video data in the downlink video packets with the downlink audio packets containing the respective streams of mixed audio data in addition to receiving the downlink video packets and the downlink audio packets.

42. The computer program product according to claim 41, wherein receiving at least one corresponding synchronization packet comprises receiving messages that are indicative of downlink bandwidth availability for transmission.

43. The computer program product according to claim 42, wherein receiving the messages comprises detecting a delay in receiving one or more of the downlink audio and video packets, and transmitting the delay.

44. The computer program product according to claim 43, wherein receiving the messages comprises receiving instructions for controlling quality by reducing the quality of the video data transmitted in the uplink video packets responsively to detecting the delay.

45. The computer program product according to claim 44, wherein controlling the quality comprises increasing or decreasing at least one quality parameter selected from a group of quality parameters consisting of an image resolution, a degree of image compression, a frame rate and a bandwidth.

46. The computer program product according to claim 45, wherein controlling the quality comprises eliminating an interval of silent audio data in order to compensate for the delay by marking at least one block of the audio data as a silent block, and eliminating the silent block from the mixed audio data.

47. The computer program product according to claim 46, wherein each of the downlink video packets contains the video data from each respective at least one remote computer.

48. The computer program product according to claim 47 further comprising the steps of:

receiving and synchronizing the video data with the mixed audio data based on synchronization data from the received synchronization packet; and

outputting the synchronized video and mixed audio data to a respective user of each of the remote computers wherein outputting the synchronized video and mixed audio data comprises displaying the video data captured by the respective one of the remote computers in a respective window among multiple windows displayed by each of the remote computers.

49. The computer program product according to claim 48, wherein synchronizing the video data comprises controlling the multiple windows so that the video data conveyed from each of the remote computers are synchronized with the mixed audio data.

50. A client computer, comprising:

a user interface; and

a processor, which is configured to establish a communication link over a packet network so as to participate in a video teleconference, and to transmit uplink audio packets and uplink video packets, which respectively contain audio and video data,

wherein the processor is configured to receive downlink audio packets containing the a stream of mixed audio data, to receive synchronization packets containing synchronization information generated, and to receive downlink video packets containing the video data transmitted by other remote computers in the video teleconference, and to synchronize the video data with the mixed audio data based on the synchronization information for output via the user interface.