Systems and methods for novel real-time audio-visual communication and data collaboration

• US 20050262201A1
• Filed: 04/30/2004
• Published: 11/24/2005
• Est. Priority Date: 04/30/2004
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Rui, Yong, Malvar, Henrique S., He, Li-Wei, Cohen, Michael F., Rudolph, Eric, Tashev, Ivan
• Granted Patent: US 7,634,533 B2
• Time in Patent Office: Days
• US Class Current: 709/205
• CPC Class Codes: H04L 12/1827 Network arrangements for conference optimisation or adaptation
• CPC Class Codes: H04L 12/1831 Tracking arrangements for later retrieval, e.g. recording contents, participants activities or behavior, network status
• CPC Class Codes: H04L 65/1101 Session protocols
• CPC Class Codes: H04L 65/403 Arrangements for multi-party communication, e.g. for conferences data switching systems for conference H04L12/18; arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities H04M3/56; television conferencing systems H04N7/15
• CPC Class Codes: H04L 65/4038 with floor control

Abstract

Systems and methods are disclosed that facilitate real-time information exchange in a multimedia conferencing environment. Data Client(s) facilitate data collaboration between users and are maintained separately from audio/video (AV) Clients that provide real-time communication functionality. Data Clients can be remotely located with respect to one another and with respect to a server. A remote user Stand-in Device can be provided that comprises a display to present a remote user to local users, a digital automatic pan/tilt/zoom camera to capture imagery in, for example, a conference room and provide real-time information to an AV Client in a remote office, and a microphone array that can similarly provide real-time audio information from the conference room to an AV Client in the remote office. The invention further facilitates file transfer and presentation broadcast between Data Clients in a single location or in a plurality of disparate locations.

Claims

1. A system that facilitates real-time information exchange between users in a conference environment, comprising:

at least two Data Clients that receive conference-related information and present it to respective users, each Data Client comprising a user interface with a Presentation Chute icon therein; and

a Room Server and a Data Server that communicate with each of the at least two data clients and permit data transfer over at least one real-time communication channel.

2. The system of claim 1, at least one Data Client is located locally to the Room Server and at least one Data Client is located remotely from the Room Server.

3. The system of claim 2, wherein the conference environment comprises at least one remote person Stand-in Device that represents a remote user and an audio/video (AV) client that provides audio and video communication information to the at least one remote user represented by the at least one remote person Stand-in Device.

4. The system of claim 3, wherein the at least one remotely located Data Client is run on a remote personal computer that comprises a microphone and a web camera.

5. The system of claim 4, wherein the remote personal computer comprises an AV Client that coordinates audio and video information in a remote location.

6. The system of claim 2, wherein a Data Client runs on at least one of a personal computer, a laptop, a tablet, a handheld computing device, a personal desktop assistant, and a cell phone.

7. The system of claim 6, further comprising a video component that captures still images and/or video in the local location and provides the captured data to the AV Client in the local location for dissemination to one or more AV Clients one or more remote locations.

8. The system of claim 7, wherein the video component comprises a digital camera that captures wide-angle video of users present at a conference room table.

9. The system of claim 8, further comprising a Virtual Director that controls the digital camera.

10. The system of claim 8, wherein the video component further comprises a web camera that captures high-resolution images of documents for real-time dissemination.

11. The system of claim 6, further comprising an audio component that captures audio information in a conference room and provides the captured data to the AV Client in the conference room for dissemination to one or more AV Clients in one or more remote locations.

12. The system of claim 11, further comprising a Virtual Director that exerts control over and/or makes inferences regarding the audio component.

13. The system of claim 11, wherein the audio component reduces ambient noise in the conference room.

14. The system of claim 11, wherein the audio component provides echo mitigation in the conference room.

15. The system of claim 11, wherein the audio component comprises a microphone array that facilitates at least one of audio data capture, noise reduction, and automatic echo mitigation.

16. The system of claim 15, wherein the audio component determines speaker position in the conference room.

17. The system of claim 1, wherein the user interface further comprises a first region comprising server file icons and a second region comprising local file icons.

18. The system of claim 17, wherein the Presentation Chute icon permits a user of a first Data Client to drag-and-drop a file over the Presentation Chute icon for immediate upload to the server and dissemination to at least a second Data Client.

19. The system of claim 18, wherein the file is an electronic slide presentation.

20. A system that facilitates real-time multimedia information exchange in a conference environment, comprising:

a Room Server located in a conference room,

one or more local Data Clients that run on personal computing devices to facilitate data collaboration, located in the conference room;

one or more remote Data Clients that run on one or more remote personal computing devices in at least one disparate location from the conference room; and

zero or more audio/video (AV) clients that run at least one of the conference room and at least one disparate location.

21. The system of claim 20, wherein the one or more remote personal computing device comprises a personal computer equipped with a microphone, a web camera, and an AV Client.

22. The system of claim 20, wherein the one or more remote personal computing device comprises a cellular phone.

23. The system of claim 20, further comprising a remote Stand-in Device connected to the Room Server that permits a remote user to receive real-time audio and video data from the conference room and to be seen in the conference room by users in the conference room.

24. The system of claim 23, wherein the remote Stand-in Device comprises a flat-panel display, a digital automatic pan/tilt/zoom (DPTZ) camera, a microphone array, and at least one speaker.

25. The system of claim 24, wherein the remote user controls the DPTZ camera to pan, tilt, and/or zoom to specific regions of a digital image captured by the DPTZ camera.

26. The system of claim 25, wherein the DPZT camera is controlled by a Virtual Director to automatically pan, tilt, and/or zoom to a speaking person in the conference room based at least in part on at least one of:

position information gleaned from the microphone array regarding the location of the speaking person within the conference room; and

vision-based tracking information gleaned from video captured by the DPTZ camera.

27. The system of claim 26, wherein the vision-based tracking information is based on at least one of motion of a speaker, appearance of a speaker, and particle filter-based analysis.

28. The system of claim 26, wherein the Virtual Director presents a localized region of a digital video image in which a speaking person is located in higher resolution than the rest of the digital video image.

29. The system of claim 20, wherein the conference environment comprises at least one of a plurality of conference rooms and a plurality of remote locations, each of which comprises at least one Data Client or one AV Client.

30. The system of claim 29, wherein each of the plurality of conference rooms and the plurality of remote locations comprises an AV Client.

31. A system that facilitates real-time multimedia information exchange in a conference room, comprising:

a Room Server that resides on a Room Server PC and provides centralized control of information exchange;

a local audio/video (AV) client that controls AV communication and resides in the Room Server PC; and

one or more Data Clients connected to the Room Server, wherein the one or more Data Client is separate and distinct from the AV Client.

32. The system of claim 31, the Room Server comprises a Data Server that communicates with the one or more Data Clients.

33. The system of claim 32, the Room Server comprises an AV Server coupled to the local AV Client and zero or more remote AV Clients.

34. The system of claim 31, further comprising a remote Stand-in Device that represents one or more remote users.

35. The system of claim 34, the remote Stand-in Device comprises a monitor, a wide-angle web camera that captures video of the conference room, a microphone array that captures audio in the conference room, and at least one speaker.

36. The system of claim 35, further comprising a web camera that captures high-resolution images and/or video of documents.

37. A method that facilitates automatic discovery of a Data Client in a real-time multimedia conference environment, comprising:

activating a server in a conference room that can centrally govern a plurality of Data Clients;

introducing a new Data Client to the conference environment;

determining the identity of the new Data Client;

verifying authorization of the new Data Client to participate in the conference; and

initiating a communications link between the server and the new Data Client upon verification of authorization.

38. The method of claim 37, further comprising inviting a user to participate in a conference via a notification to a Data Client associated with the user.

39. The method of claim 37, wherein the identity of the Data Client is determined via at least one of an email address, an internet protocol address, and a universal resource locator identity assigned to the Data Client.

40. The method of claim 37, further comprising prompting a user to enter a password to verify user identity in order to ensure that the user of the Data Client is the person with whom the Data Client is associated.

41. A system that facilitates real-time multimedia information exchange, comprising:

means for capturing audio/video information in at least one of a first location and a second location remote from the first location;

means for presenting captured audio/video information to at least two users, each user is located in at least one of the first and second locations;

means for permitting data collaboration between the at least two users; and

means for providing a communications link between the at least two users.

42. The system of claim 41, further comprising:

means for permitting real-time document transfer in binary format from a first user to at least a second user; and

means for recognizing a transferred document as a specified type of document and presenting it to a recipient user according to the specified type.

43. The system of claim 41, further comprising means for inviting a user to participate in a conference.

44. A method for real-time information exchange in a multimedia conferencing environment, comprising:

providing a data client that permits data collaboration; and

providing an AV Client that permits AV communication;

wherein the AV Client and the Data Client operate independently of each other.