Video teleconferencing for networked workstations

• US 5,546,324 A
• Filed: 11/22/1994
• Issued: 08/13/1996
• Est. Priority Date: 06/03/1992
• Status: Expired due to Term

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Digital Equipment Corporation (HP Inc.)
• Inventors: Palmer, Larry G., Palmer, Ricky S.
• Primary Examiner(s): Voeltz, Emanuel T.
• Assistant Examiner(s): Peeso, Thomas
• Application Number:
  US08/343,657
• Time in Patent Office: 630 Days
• Field of Search: 364/514, 364/514 R, 370/62, 370/60, 379/96, 379/94, 379/202, 348/15, 348/13, 348/19, 348/212
• US Class Current: 348/14.1
• CPC Class Codes: H04N 21/4135 external recorder interface circuits between an apparatus for recording television signals and a television receiver H04N5/775
• CPC Class Codes: H04N 21/4143 embedded in a Personal Computer [PC]
• CPC Class Codes: H04N 21/42203 sound input device, e.g. microphone
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/43072 of multiple content streams on the same device
• CPC Class Codes: H04N 21/4621 Controlling the complexity of the content stream or additional data, e.g. lowering the resolution or bit-rate of the video stream for a mobile client with a small screen arrangements for using the results of monitoring on user's side in broadcast systems H04H60/65; flow control in packet networks H04L47/10
• CPC Class Codes: H04N 21/4788 communicating with other users, e.g. chatting arrangements for providing for computer conferences, e.g. chat rooms, to substation in data switching networks H04L12/1813; distributed application using peer-to-peer [P2P] networks H04L67/104
• CPC Class Codes: H04N 21/485 End-user interface for client configuration
• CPC Class Codes: H04N 21/6118 involving cable transmission, e.g. using a cable modem
• CPC Class Codes: H04N 21/6168 involving cable transmission, e.g. using a cable modem
• CPC Class Codes: H04N 21/643 Communication protocols network streaming protocols, e.g. real-time transport protocol [RTP] or real-time control protocol [RTCP], H04L65/65
• CPC Class Codes: H04N 21/64322 IP
• CPC Class Codes: H04N 7/147 Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals selecting H04Q
• CPC Class Codes: H04N 7/15 Conference systems

Abstract

A video teleconferencing method and apparatus for computer workstations connected by a digital data network includes a transmission source portion for a local workstation to send audio and video teleconference data across the network to one or more remote workstations, and, a receiver for the local workstation to receive audio and video teleconference data back from the remote workstations. The local workstation sends teleconference data to each of the remote workstations over a variable bandwidth digital data connection, and each of the remote workstations returns teleconference data back to the local workstation over another variable bandwidth digital data connection. The transmission source portion includes a master software process executing on the local workstation, and the receiver includes a slave software process executing the remote workstation. The master process of a local workstation causes execution of a slave process on a remote workstation for receiving video teleconference data from the local workstation. An audio data transmitter for sends an audio data stream to the remote workstation such that the audio data can be reconstructed into a continuous audio signal. A video transmitter sends video data so that each frame of video data to be sent is inserted into the audio data stream without affecting the continuity of the reconstructed audio signal at the remote workstation.

Claims

1. Video teleconferencing apparatus for a distributed data processing system having a plurality of computer workstations connected by a digital data local area network, the computer workstations comprising:

a) source means for a local workstation to send audio and video data across the digital data local area network connecting the computer workstations as digital data packets to a remote workstation;

b) receiver means for the local workstation to receive audio and video data from across the digital data local area network as digital data packets sent from source means of the remote workstation;

wherein the source means of the local workstation sends audio and video data to the receiver means of the remote workstation over one variable bandwidth digital data connection, and the source means of the remote workstation sends audio and video data to the receiver means of the local workstation over another variable bandwidth digital data connection.

2. The video teleconferencing apparatus of claim 1, wherein

a) the source means further comprises means for a local workstation to send audio and video data to a plurality of remote workstations;

and

b) the receiver means further comprises means for the local workstation to receive audio and video data from the source means of the plurality of remote workstations;

wherein the source means of the local workstation sends audio and video data to the receiver means of the plurality of remote workstations over a like plurality of variable bandwidth digital data connections, and the source means of each of the plurality of remote workstations sends audio and video data to the receiver means of the local workstation over another like plurality of variable bandwidth digital data connections.

3. The video teleconferencing apparatus of claim 1 wherein the digital data connections comprise a DECnet protocol data socket.

4. The video teleconferencing apparatus of claim 1 wherein the digital data connections are established across an FDDI data network.

5. The video teleconferencing apparatus of claim 1 wherein the digital data connections are established across an ethernet data network.

6. Video teleconferencing apparatus for a distributed data processing system having a plurality of computer workstations connected by a digital data network, the computer workstations comprising:

a) source means for a local workstation to send audio and video data across the digital data network as digital data packets to a remote workstation, including audio transmission means for sending an audio data stream from the local workstation to the remote workstation via the digital data network such that the audio data can be reconstructed by the remote workstation into a continuous audio signal, and video transmission means for sending video data from the local workstation to the remote workstation via the digital data network such that each frame of video data sent to the remote workstation is inserted into the audio data stream by the local workstation without affecting the continuity of the reconstructed audio signal at the remote workstation;

b) receiver means for the local workstation to receive audio and video data from across the digital data network as digital data packets sent from source means of the remote workstation;

wherein the source means of the local workstation sends audio and video data to the receiver means of the remote workstation over one variable bandwidth digital data connection, and the source means of the remote workstation sends audio and video data to the receiver means of the local workstation over another variable bandwidth digital data connection.

7. The video teleconferencing apparatus of claim 6 wherein

a) the video transmission means further comprises means for precluding a frame of video data from being sent to the remote workstation in response to a system overload condition;

and

b) the audio transmission means further comprises means for sending the audio data stream corresponding to the precluded frame of video data to the remote workstation to prevent loss of continuity of the audio signal reconstructed by the remote workstation during a system overload condition.

8. The video teleconferencing apparatus of claim 7 wherein

a) the receiver means comprises acknowledgement transmission means for issuing a frame acknowledge message to the local workstation for each frame of video data successfully received from the local workstation by the remote workstation;

and

b) the source means comprises acknowledgement receiver means for receiving the acknowledgement message from the remote workstation and for deferring the sending of another frame of video data from the local workstation to the remote workstation until an acknowledgement message for the most recently sent frame of video data is received from the remote workstation.

9. The video teleconferencing apparatus of claim 8 wherein the system overload condition is determined by the elapsed time between sending a frame of video data to the remote workstation and receiving an acknowledgement message for that frame of video data from the remote workstation.

10. The video teleconferencing apparatus of claim 9 wherein

a) the audio transmission means further comprises means for sending any audio data accumulated by the local workstation since the last video frame was sent to the remote workstation immediately upon receipt of the acknowledgement message from the remote workstation for the last video frame if an overload condition exists;

and

b) the video transmission means further comprises means for sending the most recently available frame of video data from the local workstation to the remote workstation and dropping any other intervening frames of video data which became available during the elapsed time of the overload condition; and

c) the audio transmission means comprises means for sending to the remote workstation any audio data accumulated by the local workstation, since the last video frame was sent to the remote workstation, when another video frame becomes available at the local workstation for sending to the remote workstation if no overload condition exists; and

d) the video transmission means comprises means for sending the most recently available frame of video data from the local workstation to the remote workstation if no overload condition exists.

11. The video teleconferencing apparatus of claim 6 wherein

a) the video transmission means further comprises means for precluding a frame of video data from being sent to the remote workstation in response to a system failure condition;

b) the audio transmission means further comprises

1) accumulating means for accumulating audio data at the local workstation corresponding to a predetermined time interval;

2) means for sending the accumulated audio data to the remote workstation immediately after the system failure condition is corrected; and

c) the video transmission means also further comprises means for sending the most recently available frame of video data to the remote workstation after sending the accumulated audio data.

12. The video teleconferencing apparatus of claim 6 wherein

a) the source means further comprises means for attaching timing information to each frame of video data sent to the remote workstation by the video transmission means, the timing information indicating a point in the continuous audio data stream which corresponds in time to the frame of video data;

and

b) the receiver means further comprises synchronization means for displaying a received frame of video on the display of the remote workstation when the point in the audio stream corresponding to the timing information of the received video frame is audibly reproduced at the remote workstation.

13. The video teleconferencing apparatus of claim 12 wherein

a) the receiver means further comprises timing means for generating timing information by counting the amount of audio data received from the local workstation in the continuous audio stream;

and

b) the synchronization means further comprises comparison means for comparing the timing information generated by the timing means with the timing information sent along with the most recently received video frame to determine when to display the most recently received frame of video data on the display of the remote workstation.

14. The video teleconferencing apparatus of claim 6 wherein the digital data connections comprise a TCP/IP protocol data socket.

15. The video teleconferencing apparatus of claim 6 wherein the digital data connections are established across an ISDN data network.

16. A method for sending audio and video data between a plurality of video teleconferencing workstations connected by a digital data network, comprising the steps of

a) sending an audio data stream from a local workstation to a remote workstation via the digital data network such that the audio data stream can be reconstructed by the remote workstation into a continuous audio signal, and

b) sending video data from the local workstation to the remote workstation via the digital data network such that each frame of video data sent to the remote workstation is inserted into the audio data stream by the local workstation without affecting the continuity of the reconstructed audio signal at the remote workstation.

17. The method of claim 16 further comprising the steps of

a) precluding a frame of video data from being sent to the remote workstation in response to a system overload condition;

and

b) sending the audio data stream corresponding to the precluded frame of video data to the remote workstation to prevent loss of continuity of the audio signal reconstructed by the remote workstation during a system overload condition.

18. The method of claim 17 further comprising the steps of

a) the remote workstation issuing a frame acknowledge message to the local workstation for each frame of video data successfully received from the local workstation by the remote workstation;

and

b) the local workstation deferring the sending of another frame of video data to the remote workstation until an acknowledgement message for the most recently sent frame of video data is received from the remote workstation.

19. The method of claim 18 wherein the system overload condition is determined by the elapsed time between sending a frame of video data to the remote workstation and receiving an acknowledgement message for that frame of video data from the remote workstation.

20. The method of claim 19 further comprising the steps of

a) if a system overload condition exists, then:

sending any audio data accumulated by the local workstation, since the last video frame was sent to the remote workstation, immediately upon receipt of the acknowledgement message from the remote workstation for the last video frame; and

b) sending the most recently available frame of video data from the local workstation to the remote workstation and dropping any other intervening frames of video data which became available during the elapsed time of the overload condition; and

if a system overload condition does not exist, then;

a) sending to the remote workstation any audio data accumulated by the local workstation, since the last video frame was sent to the remote workstation, when another video frame becomes available at the local workstation for sending to the remote workstation; and

b) sending the most recently available frame of video data from the local workstation to the remote workstation.

21. The method of claim 16 further comprising the steps of

a) preventing a frame of video data from being sent to the remote workstation in response to a system failure condition;

and

b) accumulating audio data at the local workstation corresponding to a predetermined time interval;

c) sending the accumulated audio data to the remote workstation immediately after the system failure condition is corrected; and

d) sending the most recently available frame of video data to the remote workstation after sending the accumulated audio data.

22. The method of claim 16 further comprising the steps of

a) attaching timing information to each frame of video data sent to the remote workstation indicating a point in the continuous audio data stream which corresponds in time to the frame of video data;

and

b) displaying a received fame of video on display of the remote workstation when the point in the audio stream corresponding to the timing information of the received video frame is audibly reproduced at the remote workstation.

23. The method of claim 22 further comprising the steps of

a) the remote workstation internally generating timing information by counting the amount of audio data received from the local workstation in the continuous audio stream;

and

b) the remote workstation comparing the internally generated timing information with the timing information sent along with the most recently received video frame to determine when to display the most recently received frame of video data on the display of the remote workstation.

34. A video teleconferencing computer workstation as claimed in claim 21, wherein the digital data network comprises an Ethernet network.

24. A video teleconferencing computer workstation, comprising:

a video frame grabber for generating local video data by digitizing analog video signals received from a video source, and for accumulating the local video data to provide frames of the local video data;

a video buffer for receiving and temporarily storing the frames of local video data;

an audio controller for generating local audio data by digitizing a live audio signal from an audio source; and

a network controller for transmitting the local audio data and the frames of local video data, temporarily stored by the video buffer, to a remote computer workstation via a first one-way variable bandwidth digital data connection providing a first virtual circuit connection across a digital data network.

25. The workstation of claim 24, wherein the video source is a video camera providing sequential frames of live analog video, and the video frame grabber captures, digitizes, and stores each sequential frame of live analog video signals to generate the frames of the local video data.

26. The workstation of claim 24 wherein the audio source comprises a microphone for transducing live audio into the live audio signal.

27. The workstation of claim 24 wherein the audio controller applies mu-law compression in digitizing the live audio signal.

28. The workstation of claim 24 wherein the digital data network comprises a fiber distributed data interface digital data network.

29. The workstation of claim 24 wherein the digital data network comprises an ISDN digital data network.

30. The workstation of claim 24 wherein the digital data network comprises an interface to an Ethernet digital data network.

31. The workstation of claim 24 wherein the digital data network comprises a TCP/IP socket.

32. A video teleconferencing computer workstation as claimed in claim 24, wherein the digital data network is a local area network.

33. A video teleconferencing computer workstation as claimed in claim 24, wherein the digital data network comprises a fiber distributed data interface.

35. A method for maintaining a video teleconference in a distributed computer system including at least three computers and a network linking the computers for transporting data packets between the computers, the method comprising:

maintaining two variable bandwidth digital data connections, one in each direction between every pair of the computers;

each one of the computers generating audio and video teleconferencing data; and

each one of the computers transmitting the audio and video teleconferencing data to other ones of the computers via the variable bandwidth digital connections.

36. A method as claimed in claim 35, wherein generating the audio and video teleconferencing data comprises digitizing frames of live video images and digitizing live audio sounds.

37. A method as claimed in claim 36, wherein transmitting the audio and video teleconferencing data comprises packetizing the audio and video teleconferencing data in accordance with a protocol of the digital data network and transmitting the packets through the variable bandwidth digital data connections.

38. A method as claimed in claim 37, each one of the computers receives the audio and video teleconferencing data and acknowledges receipt of the digitized frames of the live video images.

39. A method as claimed in claim 38, wherein the computers prioritize the transmission of audio teleconferencing data by transmitting the audio teleconferencing data prior to the video teleconferencing data in response to detected delays in the transmission of the audio and video teleconferencing data.

40. A method as claimed in claim 39, wherein the computers detect delays in the transmission of the audio and video teleconferencing data by detecting an elapsed time between the transmission of a frame of the live video images and receipt of a signal acknowledging receipt of that frame by another one of the computers.

41. A method as claimed in claim 40, wherein the digital data network is a local area network.

42. A method as claimed in claim 35, wherein the digital data network is an Ethernet network.

43. A method as claimed in claim 35 wherein the digital data network is a fiber distributed data interface.