Video conferencing system and method

• US 20040119814A1
• Filed: 12/20/2002
• Published: 06/24/2004
• Est. Priority Date: 12/20/2002
• Status: Abandoned Application

• Current Assignee: World Visual Web Incorporated
• Original Assignee: World Visual Web Incorporated
• Inventors: Kumar, Praveen, Clisham, Allister B., Erfani, Mehran, Luo, Dexiang Edward, Lakshmanan, Balaji, Lam, Han T.
• Application Number:
  US10/327,038
• Publication Number:
  US 20040119814A1
• Time in Patent Office: Days
• US Class Current: 348/14.08
• CPC Class Codes: H04N 21/2543 Billing , e.g. for subscription services payment schemes, architectures or protocols G06Q20/00; e-commerce G06Q30/00; arrangements for billing for the use of broadcast information or broadcast-related information H04H60/21
• CPC Class Codes: H04N 21/25875 involving end-user authentication restricting access to computer systems by authenticating users using a predetermined code G06F21/33; arrangements for secret or secure communication including means for verifying the identity or authority of a user of the system H04L9/32; networks authentication protocols H04L63/08; authentication in wireless network security H04W12/06
• CPC Class Codes: H04N 21/41407 embedded in a portable device, e.g. video client on a mobile phone, PDA, laptop constructional details of equipment or arrangements specially adapted for portable computer application G06F1/1626; arrangements specially adapted for mobile receivers in broadcast systems H04H20/57
• CPC Class Codes: H04N 21/440263 by altering the spatial resolution, e.g. for displaying on a connected PDA
• CPC Class Codes: H04N 21/440281 by altering the temporal resolution, e.g. by frame skipping television signal recording using magnetic recording on tape for reproducing at a rate different from the recording rate H04N5/783
• CPC Class Codes: H04N 21/4788 communicating with other users, e.g. chatting arrangements for providing for computer conferences, e.g. chat rooms, to substation in data switching networks H04L12/1813; distributed application using peer-to-peer [P2P] networks H04L67/104
• CPC Class Codes: H04N 21/643 Communication protocols network streaming protocols, e.g. real-time transport protocol [RTP] or real-time control protocol [RTCP], H04L65/65
• CPC Class Codes: H04N 21/64792 Controlling the complexity of the content stream, e.g. by dropping packets intermediate media network packet handling H04L65/765; proxy provisioning conversion or adaptation for reducing the amount or size of exchanged application data H04L67/5651; negotiation of resources in wireless networks H04W28/16
• CPC Class Codes: H04N 7/141 between two video terminals, e.g. videophone telephonic communication systems combined with television receiver for reception of entertainment or information matter H04M11/085
• CPC Class Codes: H04N 7/15 Conference systems

Abstract

A high quality video conference system enables remote video conferencing over wireless communication links. A portable client device is configured to capture video and audio streams, encode them, and transmit them to a remote client device using a wireless communication link. Multiple video coding algorithms can be implemented in the client device and one of the algorithms chosen for a particular video conference session. One or more destination devices can simultaneously video conference with a client device. Multiple servers manage the high speed communication network. The servers authenticate users and manage video conference sessions, including the initiation and termination of sessions.

Claims

1. A video conference system configured to provide video conferencing over wireless communication links for portable devices, the system comprising:

a network having wireless access points;

a plurality of client devices in communication with the network, at least one of the plurality of client devices in communication with the network using a wireless communication link to one of the wireless access points, the at least one of the plurality of client devices configured to process a video and audio stream to generate a packetized video and audio stream, the at least one of the plurality of client devices wirelessly transmitting the packetized video and audio stream as video conference session data to the network, the at least one of the plurality of client devices further configured to receive a packetized remote video and audio stream from the network;

a billing system management application (BSMA) server connected to the network and in communication with the at least one of the plurality of client devices, the BSMA server configured to authenticate the plurality of client devices, communicate call requests from a source device to a destination device, initiate a video conference session between the source device and the destination device, and determine a bill for the video conference session between the source device and the destination device.

2. The video conference system of claim 1, wherein the BSMA server comprises an authentication server coupled to the network and configured to receive from the network an authentication request message transmitted by the source client, the authentication server authenticating the source client with the video conference system.

3. The video conference system of claim 1, wherein the BSMA server comprises a session manager configured to manage the video conference session between the source device and the destination device in part, by tracking a duration of an active video conference session between the source device and the destination device.

4. The video conference system of claim 1, wherein the BSMA server comprises a session manager configured to receive an authentication/initialization request from the source device requesting a video conference session with the destination device.

5. The video conference system of claim 4, wherein the authentication/initialization request includes the MAC and IP address of the source client device, and a user ID associated with the destination device and the session manager accesses a database to retrieve an IP address of the destination device, and the session manager transmits the IP address of the destination device to the source device.

6. The video conference system of claim 4, wherein the session manager is configured to generate a random session ID in response to the authentication/initialization request, and the session manager is further configured to transmit the session ID to the source device.

7. The video conference system of claim 4, wherein the session manager is configured to receive a start message from the source device and update a session entry to indicate a session start time.

8. The video conference system of claim 4, wherein the session manager is configured to receive an abort message from the source device and update a session entry to indicate a session abort time.

9. The video conference system of claim 4, wherein the BSMA server further comprises a billing manager, and wherein the session manager is configured to receive a termination message from one of the source device or the destination device, the session manager further configured to update a session entry to indicate a session end time, and transmit a billing request message to the billing manager.

10. The video conference system of claim 1, wherein the BSMA server further comprises a background process configured to periodically access a session entry to determine if a heartbeat signal has been received from the source device within a predetermined period of time.

11. The video conference system of claim 1, wherein the source device is configured to transmit a heartbeat signal during an active video conference session and the BSMA server comprises a heartbeat manager configured to update a heartbeat time stored in a database and transmit a heartbeat acknowledgement message to the source device in response to the heartbeat signal.

12. The video conference system of claim 1, wherein the network is configured to prioritize video conference session data.

13. A video conferencing device for use in a wireless video conferencing system where the video conferencing device communicates with a network over a wireless communication link, the device comprising:

a video encoder configured to receive a video stream and generate an encoded video stream by selectively encoding the video stream using one of a plurality of video encoding algorithms;

a video packetizer coupled to the video encoder and configured to packetize the encoded video stream to generate a packetized video stream;

an audio encoder configured to receive an audio stream and generate an encoded audio stream by encoding the audio stream using an audio encoding algorithm;

an audio packetizer coupled to the audio encoder and configured to packetize the encoded audio stream to generate a packetized audio stream;

a transmit stream controller coupled to the video packetizer and the audio packetizer, the transmit stream controller configured to wirelessly transmit the packetized video stream and packetized audio stream to the network at a rate greater than 2 Mbps for delivery to a destination device in wireless communication with the network.

14. The video conferencing device of claim 13, further comprising a camera coupled to the video encoder, the camera configured to generate the video stream.

15. The video conferencing device of claim 13, further comprising a microphone coupled to the audio encoder, the microphone configured to generate the audio stream.

16. The video conferencing device of claim 13, wherein the plurality of video encoding algorithms comprises H.263+, MPEG-2, MPEG-4, Motion-JPEG, and Motion-JPEG2000.

17. The videoconferencing device of claim 13, wherein the encoder is further configured to encode the video stream with a first encoding algorithm during a first period of time and encode the video stream with a second encoding algorithm during a second period of time.

18. The video conferencing device of claim 13, wherein the transmit stream controller is configured to transmit the packetized video stream and packetized audio stream at a rate greater than 3 Mbps.

19. The video conferencing device of claim 13, wherein the transmit stream controller is configured to transmit the packetized video stream and packetized audio stream at a rate greater than 4 Mbps.

20. The video conferencing device of claim 13, wherein the transmit stream controller is configured to transmit the packetized video stream and packetized audio stream at a rate greater than 5 Mbps.

21. The video conference device of claim 13, wherein the video packetizer is configured to packetize the encoded video stream according to a Real Time Protocol (RTP).

22. The video conference device of claim 13, wherein the video packetizer is configured to packetize the encoded video stream according to a Real Time Protocol (RTP) over a User Datagram Protocol (UDP).

23. The video conference device of claim 13, wherein the video packetizer is configured to packetize the encoded video stream according to a Real Time Protocol (RTP) over a Transmission Control Protocol (TCP).

24. The video conference device of claim 13, wherein the transmit stream controller is configured to wirelessly transmit the packetized video stream and packetized audio stream, via a communication network, to a first destination device.

25. The video conference device of claim 24, wherein the transmit stream controller is further configured to wirelessly transmit the packetized video stream and packetized audio stream, via the communication network, to a second destination device.

26. The video conferencing device of claim 24, further comprising:

a receive stream controller coupled to the transmit stream controller, the receive stream controller configured to receive the packetized video stream and packetized audio stream from the transmit stream controller, the receive stream controller further configured to wirelessly receive, at a rate greater than 2 Mbps, a remote packetized video stream and a remote packetized audio stream;

a video depacketizer connected to the receive stream controller, the video depacketizer configured to depacketize the remote packetized video stream to generate a remote encoded video stream;

a video decoder connected to the video depacketizer, the video decoder configured to decode the remote encoded video stream to generate a decoded remote video stream;

an audio depacketizer connected to the receive stream controller, the audio depacketizer configured to depacketize the remote packetized audio stream to generate a remote encoded audio stream;

an audio decoder connected to the audio depacketizer, the audio decoder configured to decode the remote encoded audio stream to generate a decoded remote audio stream; and

a resynchronizer coupled to the video decoder and audio decoder, the resynchronizer configured to resynchronize the remote decoded video stream to the remote decoded audio stream.

27. The video conferencing device of claim 26, further comprising a display connected to the resynchronizer, the display configured to display the remote decoded video stream.

28. The video conferencing device of claim 26, further comprising a speaker connected to the resynchronizer, the speaker configured to output the remote decoded audio stream.

29. The video conferencing device of claim 26, wherein the video depacketizer is further configured to depacketize the packetized video stream to generate a loopback encoded video stream.

30. The video conference device of claim 26, further comprising a session controller, and wherein the video depacketizer is further configured to determine a rate of dropped packets and communicate the rate of dropped packets to the session controller, the session controller transmitting the rate of dropped packets in a feedback message to a remote destination.

31. A video conference server for managing video conference sessions in a wireless video conference system where each of a plurality of client devices communicates with the video conference server over wireless communication links, the video conference server comprising:

a session manager configured to control initialization and termination of a video conference session between a first client device in wireless communication with a network and a second client device in wireless communication with the network;

a billing manager in communication with the session manager, the billing server configured to determine a bill for the video conference session, following termination of the video conference session between the first client device and the second client device, in response to a billing request message from the session manager;

a database in communication with the session manager and the billing manager, the database configured to store user account data and video conference session entries;

a heartbeat manager in communication with the database, the heartbeat manager configured to receive a heartbeat signal transmitted by a source device engaged in an active video conference session and further configured to update a heartbeat entry in the database in response to receiving the heartbeat signal; and

a session monitor daemon in communication with the session manager, the session monitor daemon configured to periodically determine a status of the heartbeat signal transmitted by the source device.

32. The video conference server of claim 31, wherein the session manager is further configured to receive an initialization request from the source device requesting the video conference session with a destination device, the session manager accesses a database to retrieve an address of the destination device, and transmits the address of the destination device to the source device in response to the initialization request.

33. The video conference server of claim 32, wherein the session manager is further configured to generate a random session ID in response to the initialization request, and the session manager is further configured to transmit the session ID to the source device.

34. The video conference server of claim 31, wherein the session manager is further configured to receive a start message from the source device and update a session entry to indicate a session start time.

35. The video conference server of claim 31, wherein the session manager is further configured to receive an abort message from the source device and update a session entry to indicate a session abort time.

36. The video conference server of claim 31, wherein the session manager is further configured to receive a termination message from the source device and update a session entry to indicate a session end time, and wherein the session manager transmits the billing request message in response to the termination request.

37. The video conference server of claim 31, wherein the billing server determines the bill for the video conference session, in part, on a usage rate associated with the source device.

38. The video conference server of claim 31, wherein the billing server determines the bill for the video conference session, in part, on a duration of the video conference session.

39. The video conference server of claim 31, wherein the billing server determines if a user associated with the source client has enabled a prepayment account, and wherein the billing server deducts a price of the video conference from a balance indicated in the prepayment account.

41. The method of claim 32, further comprising:

decoding the local video stream to produce a loopback video stream; and

displaying the loopback video stream.

42. The method of claim 32, further comprising, prior to receiving the remote packetized stream:

transmitting an authentication/initialization request to a session manager, the authentication/initialization request including a MAC address and IP address of the source client device, and a destination user ID; and

receiving an initialization response from the session manager, the initialization response including a destination user address.

43. The method of claim 34, further comprising:

transmitting a call request to a destination device; and

receiving an acceptance message from the destination device.

44. The method of claim 32, further comprising:

periodically transmitting a heartbeat signal to a heartbeat manager; and

receiving a heartbeat acknowledgement message from the heartbeat manager.

45. The method of claim 36, further comprising ending a video conference session if the heartbeat acknowledgement message is not received within a predetermined period of time.

46. The method of claim 32, wherein the act of capturing a local video stream comprises capturing the local video stream at a rate of 30 frames per second (fps), the local video stream having a resolution of at least 640×

480.

40. A method of videoconferencing using wireless communication links, the method comprising:

capturing a local video stream;

encoding the local video stream according to a first encoding algorithm;

capturing a local audio stream;

encoding the local audio stream;

generating a local packetized stream comprising the encoded local video stream and encoded local audio stream, both configured according to a Real Time Protocol (RTP);

wirelessly transmitting the local packetized stream;

wirelessly receiving a remote packetized stream comprising an encoded remote video stream and an encoded remote audio stream;

decoding the encoded remote video stream to produce a remote video stream;

decoding the remote audio stream to produce a remote audio stream;

resynchronizing the remote video stream to the remote audio stream;

displaying the remote video stream; and

outputting the remote audio stream.

47. A method of video conferencing using wireless communication links, the method comprising:

wirelessly receiving from a network access point a call request message transmitted by a remote device;

wirelessly transmitting an acceptance message;

initiating a lecture mode, wherein initiating the lecture mode comprises;

receiving a remote video stream transmitted by the remote device;

receiving a remote audio stream transmitted by the remote device;

displaying the remote video stream;

outputting the remote audio stream; and

inhibiting transmission of a local video stream and a local audio stream.

48. The method of claim 47, further comprising displaying, in a preview window, the remote video stream transmitted by the remote device prior to transmitting the acceptance message.

49. The method of claim 47, further comprising:

inhibiting the lecture mode; and

enabling a conference mode, wherein enabling the conference mode comprises;

receiving the remote video stream transmitted by the remote device;

receiving the remote audio stream transmitted by the remote device;

displaying the remote video stream;

outputting the remote audio stream;

wirelessly transmitting the local video stream; and

wirelessly transmitting the local audio stream.

50. The method of claim 49, further comprising:

receiving a feedback signal from the remote device; and

adjusting a resolution of the local video stream based, at least in part, on the feedback signal.

51. The method of claim 49, wherein wirelessly transmitting the local video stream comprises:

receiving a video stream from a video source;

selectively encoding the video stream with an encoding algorithm from a plurality of encoding algorithms to generate an encoded video stream;

packetizing the encoded video stream to generate a packetized video stream; and

wirelessly transmitting the packetized video stream to the network access point.

52. The method of claim 51, wherein receiving the video stream from the video source comprises receiving the video stream from a camera.

53. The method of claim 51, wherein receiving the video stream from the video source comprises receiving a prerecorded video stream from a playback device.

54. The method of claim 51, wherein receiving the video stream from the video source comprises:

receiving a first video stream from a first video source;

receiving a second video stream from a second video source; and

multiplexing the first video stream with the second video stream.

55. The method of claim 51, wherein receiving the video stream from the video source comprises:

receiving a video signal;

receiving a text signal; and

multiplexing the video signal with the text signal.

56. The method of claim 55, wherein receiving the text signal comprises:

receiving an audio voice signal; and

converting the audio voice signal to the text signal using voice recognition in combination with speech to text conversion.

57. The method of claim 51, wherein selectively encoding the video stream comprises:

encoding the video stream with a first encoding algorithm;

terminating the encoding of the video stream with the first encoding algorithm; and

encoding the video stream with a second encoding algorithm.

58. The method of claim 51, wherein packetizing the encoded video stream comprises packetizing the encoded video stream according to Real Time Protocol (RTP) over User Datagram Protocol (UDP).

59. The method of claim 51, wherein packetizing the encoded video stream comprises packetizing the encoded video stream according to Real Time Protocol (RTP) over Transmission Control Protocol (TCP).

60. The method of claim 49, wherein wirelessly transmitting the local audio stream comprises:

selectively encoding the audio stream with an audio encoding algorithm from a plurality of audio encoding algorithms to generate an encoded audio stream;

packetizing the encoded audio stream to generate a packetized audio stream; and

wireIessly transmitting the packetized audio stream to the network access point.

61. The method of claim 47, further comprising:

determining a signal quality of the remote video stream; and

transmitting a feedback signal, based in part on the signal quality, to the remote device.

62. The method of claim 47, further comprising storing a portion of the received video stream and received audio stream in memory.

63. The method of claim 47, wherein wirelessly transmitting the acceptance message comprises:

determining an address of the remote device;

automatically generating the call acceptance message if the address of the remote device corresponds to a predetermined address; and

wirelessly transmitting the acceptance message.

64. The method of claim 47, further comprising:

receiving an additional video stream from an additional remote device;

receiving an additional audio stream from the additional audio device;

displaying the video stream in a first window; and

displaying the additional video stream in a second window, wherein the second window partially overlaps the first window.

65. The method of claim 64, further comprising:

detecting an audio magnitude of the audio stream;

detecting an audio magnitude of the additional audio stream;

displaying the second window over the first window if the audio magnitude of the additional signal is greater than the audio magnitude of the audio stream.

66. The method of claim 64, further comprising:

assigning a first priority to the audio stream;

assigning a second priority to the additional audio stream; and

displaying the first window and the second window based, in part, on the first and second priority.

67. A method of video conferencing using wireless communication links, the method comprising:

wirelessly receiving from a network access point a call request message transmitted by a remote device;

wirelessly transmitting a rejection message; and

storing in memory a video stream transmitted by the remote device in response to the rejection message.

68. A method of multiple concurrent video conferencing using wireless communication links to a network, the method comprising:

initiating a first video conference session between a first client device, in wireless communication with a first access point of the network, and a second client device in wireless communication with a second access point of the network;

wirelessly transmitting a first packetized video stream from the first client device to the network including an address for delivery to the second client device;

wirelessly receiving, at the first client device, a second packetized video stream from the network, the second packetized video stream transmitted by the second client device;

initiating a second video conference session between the first client device and a third client device in wireless communication with a third access point of the network;

wirelessly transmitting the first packetized video stream from the first client device to the network including an address for delivery to the third client device;

wirelessly receiving a third packetized video stream from the network, the third packetized video stream transmitted by the third client device;

initiating a third video conference session between the second client device and the third client device;

wirelessly transmitting, from the second client device, the second packetized video stream from the second client device to the network including an address for delivery to the third client device;

wirelessly receiving, at the second client device, the third packetized video stream from the network, the third packetized video stream transmitted by the third client device;