System for providing a directory of AV devices and capabilities and call processing such that each participant participates to the extent of capabilities available

• US 5,884,039 A
• Filed: 06/07/1996
• Issued: 03/16/1999
• Est. Priority Date: 10/01/1993
• Status: Expired due to Term

• Current Assignee: Pragmatus AV LLC (Pragmatus, LLC)
• Original Assignee: Collaboration Properties Incorporated
• Inventors: Lantz, Keith A., Lauwers, J. Chris, Burns, Emmett R., Burnett, Gerald J., Ludwig, Lester F.
• Primary Examiner(s): Lee, Thomas C.
• Assistant Examiner(s): Chen, Anderson I.
• Application Number:
  US08/660,418
• Time in Patent Office: 1,012 Days
• Field of Search: 379/211, 379/202, 379/205, 395/200, 395/200.57, 395/200.77, 370/261
• US Class Current: 709/227
• CPC Class Codes: G06Q 10/10 Office automation; Time management
• CPC Class Codes: H04L 12/1813 for computer conferences, e.g. chat rooms instant messaging H04L51/04; protocols for multimedia communication H04L65/1101; arrangements for multi-party communication H04L65/403; telephonic conference arrangements H04M3/56; television conference systems H04N7/15
• CPC Class Codes: H04L 12/1818 Conference organisation arrangements, e.g. handling schedules, setting up parameters needed by nodes to attend a conference, booking network resources, notifying involved parties
• CPC Class Codes: H04L 12/1822 Conducting the conference, e.g. admission, detection, selection or grouping of participants, correlating users to one or more conference sessions, prioritising transmission
• CPC Class Codes: H04L 12/1827 Network arrangements for conference optimisation or adaptation
• CPC Class Codes: H04L 12/1831 Tracking arrangements for later retrieval, e.g. recording contents, participants activities or behavior, network status
• CPC Class Codes: H04L 12/1836 with heterogeneous network architecture
• CPC Class Codes: H04L 12/185 with management of multicast group membership
• CPC Class Codes: H04L 51/00 User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail
• CPC Class Codes: H04L 51/04 Real-time or near real-time messaging, e.g. instant messaging [IM]
• CPC Class Codes: H04L 51/10 Multimedia information
• CPC Class Codes: H04L 65/1101 Session protocols
• CPC Class Codes: H04L 65/4038 with floor control
• CPC Class Codes: H04L 65/80 Responding to QoS
• CPC Class Codes: H04L 67/14 Session management for real-time applications in data packet communications networks H04L65/1066
• CPC Class Codes: H04L 67/75 Indicating network or usage conditions on the user display
• CPC Class Codes: H04L 9/40 Network security protocols
• CPC Class Codes: H04M 2201/38 Displays
• CPC Class Codes: H04M 2203/5018 Initiating a conference during a two-party conversation, i.e. three-party-service or three-way-call computer conference organisation arrangements H04L12/1818
• CPC Class Codes: H04M 2203/5027 Dropping a party from a conference conducting a computer conference, e.g. admission detection H04L12/1822
• CPC Class Codes: H04M 3/42221 : Conversation recording systems at the subscriber's set H04M1/656
• CPC Class Codes: H04M 3/42323 : PBX's with CTI arrangements
• CPC Class Codes: H04M 3/4285 : Notifying, informing or entertaining a held party while on hold, e.g. Music On Hold
• CPC Class Codes: H04M 3/5307 : for recording messages comprising any combination of audio and non-audio components
• CPC Class Codes: H04M 3/562 : where the conference facilities are distributed
• CPC Class Codes: H04M 3/563 : User guidance or feature selection
• CPC Class Codes: H04M 3/564 : whereby the feature is a sub-conference
• CPC Class Codes: H04M 3/567 : Multimedia conference systems
• CPC Class Codes: H04M 7/006 : Networks other than PSTN/ISDN providing telephone service, e.g. Voice over Internet Protocol (VoIP) , including next generation networks with a packet-switched transport layer H04L65/00 takes precedence; aspects not specific to the type of network H04M3/00; special services in those networks H04M3/42
• CPC Class Codes: H04M 7/12 : for working between exchanges having different types of switching equipment, e.g. power-driven and step by step or decimal and non-decimal
• CPC Class Codes: H04N 7/147 : Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals selecting H04Q
• CPC Class Codes: H04N 7/148 : Interfacing a video terminal to a particular transmission medium, e.g. ISDN
• CPC Class Codes: H04N 7/15 : Conference systems
• CPC Class Codes: H04N 7/152 : Multipoint control units therefor
• CPC Class Codes: H04Q 2213/1305 : Software aspects
• CPC Class Codes: H04Q 2213/13093 : Personal computer, PC
• CPC Class Codes: H04Q 2213/13175 : Graphical user interface [GUI], WWW interface, visual indication
• CPC Class Codes: H04Q 2213/1319 : Amplifier, attenuation circuit, echo suppressor
• CPC Class Codes: H04Q 2213/1322 : PBX
• CPC Class Codes: H04Q 2213/1324 : Conference call
• CPC Class Codes: H04Q 2213/1326 : Consultation call, broker's call, call hold, toggling
• CPC Class Codes: H04Q 2213/1332 : Logic circuits
• CPC Class Codes: H04Q 2213/13337 : Picturephone, videotelephony
• CPC Class Codes: H04Q 2213/13389 : LAN, internet

Abstract

A multimedia collaboration system that integrates separate real-time and asynchronous networks--the former for real-time audio and video, and the latter for control signals and textual, graphical and other data--in a manner that is interoperable across different computer and network operating system platforms and which closely approximates the experience of face-to-face collaboration, while liberating the participants from the limitations of time and distance. These capabilities are achieved by exploiting a variety of hardware, software and networking technologies in a manner that preserves the quality and integrity of audio/video/data and other multimedia information, even after wide area transmission, and at a significantly reduced networking cost as compared to what would be required by presently known approaches. The system architecture is readily scalable to the largest enterprise network environments. It accommodates differing levels of collaborative capabilities available to individual users and permits high-quality audio and video capabilities to be readily superimposed onto existing personal computers and workstations and their interconnecting LANs and WANs. In a particular preferred embodiment, a plurality of geographically dispersed multimedia LANs are interconnected by a WAN. The demands made on the WAN are significantly reduced by employing multi-hopping techniques, including dynamically avoiding the unnecessary decompression of data at intermediate hops, and exploiting video mosaicing, cut-and-paste and audio mixing technologies so that significantly fewer wide area transmission paths are required while maintaining the high quality of the transmitted audio/video.

Claims

1. A method of conducting a teleconference among a plurality of participants having workstations with associated monitors for displaying visual images, and with associated AV capture and reproduction capabilities for capturing and reproducing video images and spoken audio of the participants, the workstations being interconnected by a first network, the network providing a data path for carrying digital data signals among the workstations, the method comprising the steps of:

(a) managing a data conference during which data is shared in real-time among a plurality of the participants and displayed on the monitors of their respective workstations;

(b) managing a videoconference during which the video image and spoken audio of one of the participants is reproduced in real-time at the workstation of another of the participants;

(c) providing at least one AV device with associated capabilities of providing at least audio and/or video signals to a workstation;

(d) providing at least one directory of the AV devices and each device'"'"'s associated capabilities;

(e) processing a workstation request for provision of audio or video signals to cause an appropriate AV device to provide the requested signals to the workstation;

(f) tracking the audio and video capabilities associated with each workstation; and

(g) processing a call, from a second to a first participant, based on the capabilities associated with the first participant, such that, if at least one capability of the set of capabilities consisting of audio capture, audio reproduction, video capture, video reproduction, and the capability of connecting to the first network, is not available to at least one of the participants, each of the plurality of participants can participate in the teleconference to the extent of the capabilities available to the participant.

2. The method of claim 1, further comprising the step of:

(a) converting signals of one format to another format to enable the teleconferencing system to support capture and reproduction devices based on different signal format standards.

3. The method of claim 1, wherein the AV device

(i) has associated capabilities of any one of the group consisting of

(1) storing one or more of digital data, audio or video signals;

(2) retrieving one or more of stored digital data, audio, and video signals;

(3) generating one or more of digital data, audio and video signals; and

(4) accessing a remote source of digital data, audio and video signals and providing the accessed signals to a workstation.

4. The apparatus of claim 3, wherein the AV device is any one or more of:

(a) a VCR;

(b) a laser disk player;

(c) a compact disk player;

(d) a multimedia resource;

(e) a television signal source; and

(f) a fax machine.

5. The method of claim 1, wherein:

at least the audio and/or the video signals are delivered to at least one workstation over a UTP network.

6. A teleconferencing system for conducting a teleconference among a plurality of participants, the system comprising:

(a) a workstation associated with each of at least three participants, each workstation having at least one origination and at least one reproduction capability, each selected from the group consisting of audio, video and data origination/reproduction capabilities;

(b) a first network providing a data path for carrying digital data signals among the workstations;

(c) an AV path for carrying AV signals, representing video images and spoken audio of the participants;

(d) a plurality of AV devices each having capabilities for providing audio and/or video signals to a workstation; and

(e) a directory of each AV device and its associated capabilities, wherein the system is configured to;

(i) manage a data conference during which images, based on digital data carried among the workstations, are displayed at the workstations of a plurality of the participants;

(ii) manage reproduction of video images and audio at the workstation of a participant by addressing a workstation request for provision of audio or video signals, to cause an appropriate AV device to provide the requested signals to the workstation,

(iii) track the audio and video origination and reproduction capabilities associated with each workstation, and

(iv) to process a call, from a second to a first participant, based on which capabilities are associated with the workstation associated with first participant, such that

(1) if any capability of the set of capabilities consisting of audio capture, audio reproduction, video capture, video reproduction, and the capability of connecting to the first network, is not available to at least one of the participants, each participant can participate in the teleconference to the extent of the capabilities available to the participant.

7. The teleconferencing system of claim 6, wherein the system is further configured to:

(a) associate a participant with an identifier entered when the participant logs into any one of a plurality of workstations, and

(b) to route a subsequent call to initiate a videoconference with that participant to the workstation at which that participant is logged in.

8. The teleconferencing system of claim 6, wherein the system is further configured to:

(a) convert signals of one format to another format to support originating and reproduction devices based on different signal format standards.

9. The teleconferencing system of claim 6, wherein the AV path connects the workstation of a first participant at a first location to the workstation of a second of the participants at a second location via a third location, the system further comprising:

(a) an AV signal switcher at the third location, coupled to the AV path to receive and route the AV signals to a location other than the third location if the AV signals are intended to be processed at the other location.

10. The teleconferencing system of claim 9, further comprising:

(a) at least a first and second codec at the first and second locations respectively, each configured to compress the AV signals and decompress compressed AV signals and wherein video and audio, compressed by the first codec, can be routed from the first location to the second location via the AV signal switcher without being decompressed at the third location.

11. The teleconferencing system of claim 10, where in the video image and spoken audio of the first participant routed to the second location, via the third location, can be reproduced at the workstations of both the first and second participants.

12. The teleconferencing system of claim 11, further comprising a video mosaic generator for combining the captured images of a first and second participant into a mosaic image for reproduction at least one workstation.

13. The teleconferencing system of claim 12, further comprising a distributed mosaic generator for combining a portion of the mosaic image with a captured image of a third participant to generate a composite mosaic image for production at least one workstation.

14. The teleconferencing system of claim 11, further comprising an audio summer for receiving the captured audio of a first, second and third participant and combining the received audio of the second and third participants into an audio sum for reproduction at the workstation of the first participant.

15. The teleconferencing system of claim 11, wherein the system is configured to:

(a) route at least the AV signals among participant'"'"'s workstations in such a way so as to optimize the carrying of AV signals between the workstations.

16. A teleconferencing system of claim 15, wherein the routing is optimized based on either the actual or anticipated state of the AV path.

17. The teleconferencing system of claim 6, further comprising:

(a) a video mosaic generator for combining the captured images of a first and second participant into a mosaic image; and

(b) a distributed mosaic generator for combining a portion of the mosaic image with a captured image of a third participant to generate a composite mosaic image of the captured images of the first, second and third participants.

18. The teleconferencing system of claim 6, further comprising:

(a) a video mosaic generator, for combining the captured images of a first and second of the participants into a mosaic image of the captured images, whereby the mosaic image can be reproduced at the workstations of the first and second participants;

wherein the system is configured to

allow a participant to select the image one participant whose image is reproduced in the mosaic image upon which the mosaic image is replaced with the selected image.

19. The teleconferencing system of claim 6, further comprising:

(a) a video mosaic generator for the captured images of a first and second of the participants into a mosaic image of the captured images; and

(b) an audio summer for receiving the captured audio of first, second and third participants and combining the received audio of second and third participants into an audio sum for production at the workstation of the first participant.

20. The system of claim 6, wherein the AV device

(i) has associated capabilities of any one of the group consisting of

(1) storing one or more of digital data, audio or video signals;

(2) retrieving one or more of stored digital data, audio, and video signals;

(3) generating one or more of digital data, audio and video signals; and

(4) accessing a remote source of digital data, audio and video signals and providing the accessed signals to a workstation.

21. The system of claim 20, wherein the AV device is any one or more of:

(a) a VCR;

(b) a laser disk player;

(c) a compact disk player;

(d) a multimedia resource;

(e) a television signal source; and

(f) a fax machine.

22. The system of claim 6, wherein the AV path

(i) is at least partly defined by unshielded twisted pair (UTP) wiring.

23. A teleconferencing system for conducting a teleconference among a plurality of participants, the system comprising:

(a) a workstation

(i) associated with each of at least two participants, and

(ii) having at least one origination and at least one reproduction capability,

(1) each selected from the group consisting of audio, video and data origination/reproduction capabilities;

(b) an AV path

(i) configured to carry AV signals,

(1) representing video images and spoken audio of the participants

(ii) among the workstations;

(c) at least one AV device

(i) having capabilities for providing at least audio and/or video signals

(1) to a workstation, and

(ii) configured to address a request

(1) for providing audio and/or video signals

(2) to one of the workstations; and

(d) at least one directory of

(i) each workstation and its origination/reproduction capabilities, and/or

(ii) each AV reproduction device and its associated capabilities,

wherein the system is configured

(i) to manage the reproduction

(1) of video images and audio

(2) at the workstation of a participant

(ii) by interacting with the directory

(iii) to address a request,

(1) generated at a workstation,

(2) audio and/or video signals,

(iv) to cause an appropriate AV device

(1) to provide the requested signals to the workstation

(v) to track the audio and video origination and reproduction capabilities associated with each workstation, and

(vi) to process a call, from a second to a first participant, based on which capabilities are associated with the first participant, and to manage a teleconference among a plurality of participants such that, if at least one capability from the set of capabilities consisting of audio capture, audio reproduction, video capture, video reproduction, and the capability conducting a data conference is not available to any participant, each participant can participate in the teleconference to the extent of the capabilities available to that participant, and

wherein the system is further configured

(i) to associate a participant

(1) with each workstation at which the participant logs in and

(ii) to route a videoconference call,

(1) for that participant,

(2) to the workstation at which that participant is logged in.

24. The teleconferencing system of claim 23, wherein the system is further configured to:

(a) convert signals of one format to another format, whereby the teleconferencing system can support originating and reproduction devices based on different signal format standards.

25. The teleconferencing system of claim 23, wherein the system is configured to combine the captured images of a first and second participant into a mosaic image for reproduction at least one workstation.

26. The teleconferencing system of claim 23, further comprising an audio summer for receiving the captured audio of a first, second and third participant and combining the received audio of the second and third participants into an audio sum for reproduction at the workstation of the first participant.

27. The teleconferencing system of claim 23, further comprising:

(a) at least one signal router for routing at least the AV signals among participant'"'"'s workstations in such a way so as to optimize the carrying of AV signals between the workstations.

28. The teleconferencing system of claim 23, wherein the system is further configured to:

(a) allow a participant to select the image one participant whose image is reproduced in the mosaic image and thereby replace the mosaic image with the selected image.

29. The system of claim 23, wherein the AV device

(i) has associated capabilities of any one of the group consisting of

(1) storing one or more of digital data, audio or video signals;

(2) retrieving one or more of stored digital data, audio, and video signals;

(3) generating one or more of digital data, audio and video signals; and

(4) accessing a remote source of digital data, audio and video signals and providing the accessed signals to a workstation.

30. The system of claim 29, wherein the AV device is any one or more of:

(a) a VCR;

(b) a laser disk player;

(c) a compact disk player;

(d) a multimedia resource;

(e) a television signal source; and

(f) a fax machine.

31. The system of claim 23, wherein the AV path

(i) is at least partly defined by unshielded twisted pair (UTP) wiring.

32. A method for conducting a teleconference among a plurality of participants having workstations with associated monitors for displaying visual images, and with associated AV capture and reproduction capabilities for capturing and reproducing video images and spoken audio of the participants, the workstations being interconnected by a first network, the network providing a data path for carrying digital data signals among the workstations, the method comprising the steps of:

(a) managing a data conference during which data is shared in real-time among a plurality of the participants and displayed on the monitors of their respective workstations;

(b) managing a videoconference during which the video image and spoken audio of one of the participants is reproduced in real-time at the workstation of another of the participants;

(c) providing at least one AV device with associated capabilities of providing at least audio and/or video signals to a workstation;

(d) defining at least one directory of AV devices and each device'"'"'s associated capabilities;

(e) processing a request for a audio and/or video signals to cause an appropriate AV device to provide the requested signals to the workstation; and

(f) managing connections between participants by

(i) associating a participant

(1) with each workstation at which the participant logs in and

(ii) routing a videoconference call,

(1) for that participant,

(2) to the workstation at which that participant is logged in,

wherein the step of managing the video conference is conducted among a plurality of participants such that, if at least one capability of the set of capabilities consisting of audio capture, audio reproduction, video capture, video reproduction, and the capability of connecting to the first network, is not available to at least one of the participants, each of the plurality of participants can participate in the teleconference to the extent of the capabilities available to the participant.

33. The method of claim 32, further comprising the steps of:

(a) tracking the audio and video capabilities associated with each workstation; and

(b) processing a call, from a second to a first participant, by including a request for a service with respect to the first participant, based on the capabilities associated with the first participant.

34. The method of claim 32, wherein the AV device

(I) has associated capabilities of any one of the group consisting of

(1) storing one or more of digital data, audio or video signals;

(2) retrieving one or more of stored digital data, audio, and video signals;

(3) generating one or more of digital data, audio and video signals; and

(4) accessing a remote source of digital data, audio and video signals and providing the accessed signals to a workstation.

35. The method of claim 34, wherein the AV device is any one or more of:

(a) a VCR;

(b) a laser disk player;

(c) a compact disk player;

(d) a multimedia resource;

(e) a television signal source; and

(f) a fax machine.

36. The method of claim 32, wherein:

at least the audio and/or the video signals are delivered to at least one workstation over a UTP network.