SYSTEM AND METHOD FOR DISTRIBUTED MEETINGS

• US 20090046139A1
• Filed: 08/13/2008
• Published: 02/19/2009
• Est. Priority Date: 06/26/2003
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Rui, Yong, Gupta, Anoop, Cutler, Ross
• Granted Patent: US 8,111,282 B2
• Time in Patent Office: Days
• US Class Current: 348/14.80
• CPC Class Codes: H04N 7/15 Conference systems
• CPC Class Codes: H04N 7/152 Multipoint control units therefor
• CPC Class Codes: H04N 7/155 involving storage of or access to video conference sessions tracking arrangements for later retrieval of a computer conference content or participants activities H04L12/1831

Abstract

A system and method for teleconferencing and recording of meetings. The system uses a variety of capture devices (a novel 360° camera, a whiteboard camera, a presenter view camera, a remote view camera, and a microphone array) to provide a rich experience for people who want to participate in a meeting from a distance. The system is also combined with speaker clustering, spatial indexing, and time compression to provide a rich experience for people who miss a meeting and want to watch it afterward.

Claims

1. A computer-implemented process for recording an event, comprising the process actions of:

recording images captured by multiple different cameras of different types simultaneously capturing different portions of an event;

recording audio synchronized with said recorded images;

storing said images and synchronized audio as event data on a server capable of distributing the said event data.

2. The computer-implemented process of claim 1 further comprising the process action of said server transmitting said event data to one or more clients in network connection with said server.

3. The computer-implemented process of claim 2 further comprising the process action of said one or more clients playing back said stored images and synchronized audio.

4. The computer-implemented process of claim 1 wherein said multiple cameras are selected from a group comprising:

a 360-degree camera centrally positioned to monitor in substantially 360-degrees the space in which the event occurs;

a remote view camera positioned so as to capture a view of event participants in the space in which the event occurs to be transmitted to a viewer over said network;

a presenter view camera positioned so as to capture an overview of the space in which said event occurs; and

a whiteboard capture camera positioned so as to capture strokes written on a whiteboard.

5. The computer-implemented process of claim 1 further comprising a process action of capturing graphics data of said event.

6. The computer-implemented process of claim 5 further comprising a process action of indexing said stored event data, wherein said indexing process action comprises determining graphics key frames that summarize sub-events of said event by analyzing said images captured by a graphics capture device and using said key frames as an index to the corresponding portion of audio and images of the stored event data.

7. The computer-implemented process of claim 4 further comprising the process action of indexing said stored event data by one of a group of process actions comprising:

determining at least one whiteboard key frame that summarizes a sub-event by analyzing said images captured by said whiteboard camera and using said whiteboard key frames as an index to the corresponding portion of audio and images of the meeting;

determining a timeline outlining the sequence of sub-events occurring in the event and using said timeline as an index to a specific portion of the audio or images of the event; and

using speech recognition to transcribe the audio of the event and using word searches to locate specific portions of said audio.

8. The computer-implemented process of claim 7 wherein said word searches are by phonetic spelling.

9. The computer-implemented process of claim 1 further comprising the process action of indexing said stored event data for playback.

10. The computer-implemented process of claim 9 wherein the process action of indexing said stored event data comprises determining who is speaking by analyzing audio of said event and segmenting the audio of each speaker to be accessible on an individual basis to be used as an index of said stored event data.

11. The computer-implemented process of claim 9 wherein the process action of indexing said recorded event data comprises analyzing said event audio to categorize tone of voice for event participants speaking at said event and indexing said stored event data by tone of voice.

12. The computer-implemented process of claim 9 wherein the process action of indexing said stored event data comprises determining speaker turn rate, wherein speaker turn rate comprises the number of speakers of a given period of time, and using said speaker turn rate to index said stored event data.

13. The computer-implemented process of claim 9 wherein the process action of indexing said stored event data comprises determining when event recording was turned on and off using event turn on and turn off events to index said stored event data.

14. The computer-implemented process of claim 9 wherein the process action of indexing said stored event data comprises determining when a shared application is started or stopped and using this to index said stored event data.

15. The computer-implemented process of claim 9 wherein the process action of indexing said stored event data comprises determining when someone is pointing at an object and using the pointing event to index to said stored event data.

16. The computer-implemented process of claim 3 further comprising the process action of segmenting the audio from a given speaker in said event.

17. The computer-implemented process of claim 16 further comprising the process action of providing a fast forward function that allows skipping of a portion of the audio of one or more speakers.

18. A graphical user interface for viewing and playing back event data comprising:

a speaker image display pane displaying a speaker speaking at an event;

a whiteboard display pane displaying an image of content written on a whiteboard;

a panoramic image display pane displaying a panoramic image of said event;

a thumbnail view of a key frame image summarizing content written on a whiteboard, wherein selecting the thumbnail view will display the key frame in the whiteboard display pane; and

a graphics display pane displaying an image captured by a graphical capture device.

19. The graphical user interface of claim 18 wherein a user can jump to a starting point of at least one of audio or video corresponding to the key frame image in a meeting playback timeline by selecting said key frame thumbnail.

20. A computer-implemented process for viewing an event comprising the process actions of:

simultaneously capturing images of different sub-events of an event with multiple cameras each capturing a different sub-event;

capturing audio associated with the different sub-events synchronized with said captured images;

storing said captured images and associated audio on a server; and

viewing the captured sub-events and associated audio by one or more clients in network connection with said server; and

indexing event data, wherein said indexing process action comprises determining graphics key frames that summarize sub-events of said event by analyzing images captured by a graphics capture device and using said key frames as an index to jump to the corresponding portion of audio and images of the event data while the event is ongoing.