Systems and methods for real-time audio-visual communication and data collaboration in a network conference environment

• US 7,634,533 B2
• Filed: 04/30/2004
• Issued: 12/15/2009
• Est. Priority Date: 04/30/2004
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Cohen, Michael F, Rui, Yong, He, Li-Wei, Rudolph, Eric, Malvar, Henrique S, Tashev, Ivan
• Primary Examiner(s): Pwu; Jeffrey
• Assistant Examiner(s): Ali; Farhad
• Application Number:
  US10/836,778
• Publication Number:
  US 20050262201A1
• Time in Patent Office: 2,055 Days
• Field of Search: 709/204, 709/203
• US Class Current: 709/203
• CPC Class Codes: H04L 12/1827 Network arrangements for conference optimisation or adaptation
• CPC Class Codes: H04L 12/1831 Tracking arrangements for later retrieval, e.g. recording contents, participants activities or behavior, network status
• CPC Class Codes: H04L 65/1101 Session protocols
• CPC Class Codes: H04L 65/403 Arrangements for multi-party communication, e.g. for conferences data switching systems for conference H04L12/18; arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities H04M3/56; television conferencing systems H04N7/15
• CPC Class Codes: H04L 65/4038 with floor control

Abstract

Systems and methods are disclosed that facilitate real-time information exchange in a multimedia conferencing environment. Data Client(s) facilitate data collaboration between users and are maintained separately from audio/video (AV) Clients that provide real-time communication functionality. Data Clients can be remotely located with respect to one another and with respect to a server. A remote user Stand-in Device can be provided that comprises a display to present a remote user to local users, a digital automatic pan/tilt/zoom camera to capture imagery in, for example, a conference room and provide real-time information to an AV Client in a remote office, and a microphone array that can similarly provide real-time audio information from the conference room to an AV Client in the remote office. The invention further facilitates file transfer and presentation broadcast between Data Clients in a single location or in a plurality of disparate locations.

Claims

1. A system that facilitates real-time information exchange between users in a conference environment, comprising:

at least two Data Clients that received conference-related information and present it to respective users, each Date Client comprising a user interface with a Presentation Chute icon therein, wherein, at least one Data Client of the at least two Data Clients comprises a receiving Data Client; and

a Room Server comprising a Data Server that communicates with each of the at least two Data Clients and permit data transfer over at least on real-time communication channel;

wherein;

the Presentation Chute icon permits a user of a sending Data Client to drag-and-drop a file over the Presentation Chute icon to immediately upload the file to the Data Server, from which the Data Server facilitates presentation of the file to each particular receiving Data Client based on the viewer capabilities of each particular receiving Data Client such that, based on the file format, each particular receiving Data Client will launch an appropriate Data Viewer; and

the Presentation Chute enables sharing an imaged, non-editable version of the file over which the user retains control with each particular receiving Data Client; and

wherein the Data Server determines viewer capabilities of each particular receiving Data Client and formats a respective file for each particular receiving Data Client for a respective viewer based upon the viewer capabilities of each particular receiving Data Client.

2. The system of claim 1, at least one Data Client is located locally to the Room Server and at least one Data Client is located remotely from the Room Server.

3. The system of claim 2, the conference environment comprises at least one remote person Stand-in Device that represents a remote user and an audio/video (AV) client that provides audio and video communication information to the at least one remote user represented by the at least one remote person Stand-in Device.

4. The system of claim 3, the at least one remotely located Data Client is run on a remote personal computer that comprises a microphone and a web camera.

5. The system of claim 4, the remote personal computer comprises an AV Client that coordinates audio and video information in a remote location.

6. The system of claim 2, at least one of the at least two Data Clients runs on at least one of a personal computer, a laptop, a tablet, a handheld computing device, a personal desktop assistant, or a cell phone.

7. The system of claim 6, further comprising a video component that captures still images and/or video in the local location and provides the captured data to the AV Client in the local location for dissemination to one or more AV Clients located at one or more remote locations.

8. The system of claim 7, the video component comprises a digital camera that captures wide-angle video of users present at a conference room table.

9. The system of claim 8, further comprising a Virtual Director that controls the digital camera.

10. The system of claim 8, the video component further comprises a web camera that captures high-resolution images of documents for real-time dissemination.

11. The system of claim 6, further comprising an audio component that captures audio information in a conference room and provides the captured data to the AV Client in the conference room for dissemination to one or more AV Clients in one or more remote locations.

12. The system of claim 11, further comprising a Virtual Director that exerts control over and/or makes inferences regarding the audio component.

13. The system of claim 11, the audio component reduces ambient noise in the conference room.

14. The system of claim 11, the audio component provides echo mitigation in the conference room.

15. The system of claim 11, the audio component comprises a microphone array that facilitates at least one of audio data capture, noise reduction, or automatic echo mitigation.

16. The system of claim 15, the audio component determines speaker position in the conference room.

17. The system of claim 1, the user interface further comprises a first region comprising server file icons and a second region comprising local file icons.

18. The system of claim 1, the file is an electronic slide presentation.

19. A system that facilitates real-time multimedia information exchange in a conference environment, comprising:

a Room Server located in a conference room;

one or more local Data Clients that run on personal computing devices to facilitate data collaboration, located in the conference room;

one or more remote Data Clients that run on one or more remote personal computing devices in at least one disparate location from the conference room; and

zero or more audio/video (AV) clients that run at one or more of the conference room or at least one disparate location;

at least one local Data Client or at least one remote Data Clients comprising a recipient Data Client;

the one or more local Data Clients and the one or more remote Data Clients each including a respective user interface, each respective user interface including a Presentation Chute icon wherein;

the Presentation Chute icon permits a user of a sending Data Client to drag-and-drop a file over the Presentation Chute icon to immediately unload the file to the Data Server, from which the Data Server facilitates presentation of the file to each particular recipient Data Client based on the viewer capabilities of each particular recipient Data Client such that, based on the file format, each particular Data Client launches an appropriate Data Viewer; and

the Presentation Chute enables sharing an imaged, non-editable version of the file over which the user retains control with each particular receiving Data Client;

the Room Server comprising a Data Server that;

communicates with each of the at least one local Data Client, the at least one remote Data Clients, and the AV client;

facilitates data transfer over at least one real-time communication channel;

determines viewer capabilities of each-particular recipient Data Client; and

formats a file for each-particular receiving Data Client for a respective viewer based upon the viewer capabilities of each-particular recipient Data Client.

20. The system of claim 19, the one or more remote personal computing device comprises a personal computer equipped with a microphone, a web camera, and an AV Client.

21. The system of claim 19, the one or more remote personal computing device comprises a cellular phone.

22. The system of claim 19, further comprising a remote Stand-in Device connected to the Room Server that permits a remote user to receive real-time audio and video data from the conference room and to be seen in the conference room by users in the conference room.

23. The system of claim 22, the remote Stand-in Device comprises a flat-panel display, a digital automatic pan/tilt/zoom (DPTZ) camera, a microphone array, and at least one speaker.

24. The system of claim 23, the remote user controls the DPTZ camera to pan, tilt, and/or zoom to specific regions of a digital image captured by the DPTZ camera.

25. The system of claim 24, the DPZT camera is controlled by a Virtual Director to automatically pan, tilt, and/or zoom to a speaking person in the conference room based at least in part on at least one of:

position information gleaned from the microphone array regarding the location of the speaking person within the conference room;

or

vision-based tracking information gleaned from video captured by the DPTZ camera.

26. The system of claim 25, the vision-based tracking information is based on at least one of motion of a speaker, appearance of a speaker, or particle filter-based analysis.

27. The system of claim 25, the Virtual Director presents a localized region of a digital video image in which a speaking person is located in higher resolution than the rest of the digital video image.

28. The system of claim 19, the conference environment comprises at least one of a plurality of conference rooms or a plurality of remote locations, each of which comprises at least one Data Client or one AV Client.

29. The system of claim 28, each of the plurality of conference rooms and the plurality of remote locations comprises an AV Client.

30. A system that facilitates real-time multimedia information exchange in a conference room, comprising:

a Room Server that resides on a Room Server PC and provides centralized control of information exchange, the room server comprising a Data Server that communicates with the one or more Data Clients;

a local audio/video (AV) client that controls AV communication and resides in the Room Server PC; and

one or more Data Clients connected to the Room Server, the one or more Data Clients being separate and distinct from the AV Client, the one or more Data Clients each include a user interface and each of the user interfaces include a Presentation Chute icon, wherein, at least one Data Client comprises a receiving Data Client, the one or more receiving Data Clients being permitted to view the document asynchronously where each of the one or more receiving Data Clients are allowed to skip ahead or revert backwards within the document from a view occupied by the sending Data Client, each of the user interfaces further include a Sync icon that automatically synchronizes the document to the view occupied by the sending Data Client when selected wherein;

the Presentation Chute icon permits a user of a sending Data Client to drag-and-drop a file over the Presentation Chute icon to immediately unload the file to the Data Server, from which the Data Server facilitates presentation of the file to each particular receiving Data Client based on the viewer capabilities of each particular receiving Data Client such that, based on the file format, each particular receiving Data Client will launch an appropriate Data Viewer; and

the Presentation Chute enables sharing an imaged, non-editable version of the file over which the user retains control with each particular receiving Data Client; and

the Data Server determines viewer capabilities of each-particular receiving Data Client and formats a file for each-particular receiving Data Client for a respective viewer based upon the viewer capabilities of each particular receiving Data Client.

31. The system of claim 30, the Room Server comprises an AV Server coupled to the local AV Client and zero or more remote AV Clients.

32. The system of claim 30, further comprising a remote Stand-in Device that represents one or more remote users.

33. The system of claim 32, the remote Stand-in Device comprises a monitor, a wide-angle web camera that captures video of the conference room, a microphone array that captures audio in the conference room, and at least one speaker.

34. The system of claim 33, further comprising a web camera that captures high-resolution images and/or video of documents.

35. A system that facilitates real-time multimedia information exchange, comprising:

means for capturing audio/video information in at least one of a first location or a second location remote from the first location;

means for presenting captured audio/video information to at least two users, each user is located in at least one of the first or second locations;

means for permitting data collaboration between the at least two users;

means for providing a communications link between the at least two users;

means for permitting real-time document transfer from a first user to at least a second user, the means for permitting comprising a Presentation Chute icon permitting the first user to drag-and-drop a file over the Presentation Chute icon to immediately upload the file to a Data Server, the Presentation Chute icon comprising a means enabling sharing an imaged, non-editable version of the real-time document over which the first user retains control with the device employed by the second user while the first user retains control of the imaged, non-editable version of the real-time document;

means for determining capabilities associated with a device employed by the second users, the means for determining comprising the Data Server, which facilitates presentation of the file to the device employed by the second user based on the viewer capabilities of the device employed by the second user;

means for formatting the real-time document based upon the determined capabilities of the device when communicating the real-time document to the second user, the means for formatting operating such that, based at least on the file format, the device employed by the second user will launch an appropriate Data Viewer; and

means for synchronizing a view of the real-time document employed by the second user to a view of the real-time document employed by the first user based upon input from the second user subsequent to permitting desynchronous review of the real-time document by the second user.

36. The system of claim 35, further comprising:

means for permitting real-time document transfer in binary format from the first user to the second user; and

means for recognizing a transferred document as a specified type of document and presenting it to a recipient user according to the specified type.

37. The system of claim 35, further comprising means for inviting a user to participate in a conference.

38. A method for real-time information exchange in a multimedia conferencing environment, comprising:

providing a Data Client that permits data collaboration;

providing an AV Client that permits AV communication, the AV Client and the Data Client operate independently of each other;

presenting a user interface to a sending user with the Data Client, the user interface comprises a Presentation Chute icon wherein;

the Presentation Chute icon permits a user of the Data Client to drag-and-drop a file over the Presentation Chute icon to immediately upload the file to a Data Server, from which the Data Server facilitates presentation of the file to at least one receiving Data Client based on the viewer capabilities of a particular receiving Data Client such that, based on the file format, the particular receiving Data Client launches an appropriate Data Viewer; and

the Presentation Chute enables sharing an imaged, non-editable version of the file over which the user retains control including at the particular receiving Data Client;

receiving a real-time file dragged-and-dropped upon the Presentation Chute icon by the sending user employing the Data Client;

uploading the real-time file to the Data Server, wherein the Data Server determines viewer capabilities of a disparate, recipient Data Client and formats the real-time file for the disparate, recipient Data Client for a respective viewer based upon the viewer capabilities of the disparate, recipient Data Client;

transmitting the real-time file to the disparate, recipient Data Client from the Data Server;

facilitating shared control such that;

in an event that the user has relinquished control of the real-time file;

permitting desynchronization of the real-time file to enable a receiving user utilizing the disparate, recipient Data Client to scroll through the real-time file at a differing pace from a pace employed by the sending user, and

allowing the receiving user utilizing the disparate, recipient Data Client to synchronize the real-time file to a view employed by the sending user based upon input from the receiving user;

in an event that the user has not relinquished control of the real-time file;

not permitting desynchronization of the real-time file to enable the receiving user utilizing the disparate, recipient Data Client to scroll through the real-time file at a differing pace from a pace employed by the sending user.