Automated online broadcasting system and method using an omni-directional camera system for viewing meetings over a computer network

• US 7,515,172 B2
• Filed: 07/29/2005
• Issued: 04/07/2009
• Est. Priority Date: 06/14/2001
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Cadiz, Johnathan J., Cutler, Ross Garrett, Rui, Yong, Gupta, Anoop
• Primary Examiner(s): Ramakrishnaiah; Melur
• Application Number:
  US11/193,702
• Publication Number:
  US 20050285933A1
• Time in Patent Office: 1,348 Days
• Field of Search: 348 1401- 1409, 348/14.1, 348 1411- 1416, 348/36, 348/37, 348/39, 348/143, 348169-172, 348/211.99, 348/211.83
• US Class Current: 348/14.03
• CPC Class Codes: H04N 7/142 Constructional details of the terminal equipment, e.g. arrangements of the camera and the display
• CPC Class Codes: H04N 7/147 Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals selecting H04Q
• CPC Class Codes: H04N 7/15 Conference systems

Abstract

An automated system and method for broadcasting meetings over a computer network. The meeting is filmed using an omni-directional camera system and capable of being presented to a viewer both live and on-demand. The system of the present invention includes an automated camera management system for controlling the camera system and an analysis module determining the location of meeting participants in the meeting environments. The method of the present invention includes using the system of the present invention to broadcast an event to a viewer over a computer network. In particular, the method includes filming the event using an omni-directional camera system. Next, the method determines the location of each event participant in the event environment. Finally, a viewer is provided with a user interface for viewing the broadcast event. This user interface allows a viewer to choose which event participant that the viewer would like to view.

Claims

1. An automated event presentation system for capturing and viewing an event having event participants, comprising:

an omni-directional camera system that captures a seamless omni-directional image of the event and that automatically tracks event participants simultaneously to determine the event participants that are speaking using audio-based tracking techniques including a microphone-array sound source localization technique to alleviate camera view switching delays;

an automated online broadcasting system including a tracker module that controls and uses the omni-directional camera system and video-based tracking techniques to monitor and keep track of each of the tracked event participants simultaneously;

a virtual director module that receives as input audio and video tracking results and uses probabilistic rules to automatically select in a probabilistic manner which portion of the omni-directional image to use as an output view; and

a viewer platform in communication with the automated online broadcasting system that allows a viewer to view the output view of the broadcasted event.

2. The automated event presentation system of claim 1, wherein the virtual director module further comprises a switching module that provides instantaneous switching between the multiple camera views of the event.

3. The automated event presentation system as set forth in claim 2, wherein the switching module provides negative switching of an on-demand or recorded event to switch the output view to a person speaking before the person begins to speak.

4. The automated event presentation system of claim 1, wherein the probabilistic rules are implemented using a finite state machine.

5. The automated event presentation system of in claim 1, wherein the omni-directional camera system requires no physical movement to capture the event participants.

6. The automated event presentation system of claim 1, further comprising a user interface on the viewer platform that allows an arbitrary number of viewers to view an arbitrary number of viewpoints of the broadcasted event.

7. The automated event presentation system of claim 1, wherein the omni-directional image provides an infinite number of viewpoints, with each of the viewpoints corresponding to a portion of the omni-directional image, such that instantaneous switching is supported for an infinite number of viewers that select arbitrarily different viewpoints.

8. The automated event presentation system of claim 1, further comprising a tracker module obtains the audio and video tracking results by combining in a in a probabilistic manner output from the audio-based tracking techniques and the video-based tracking techniques.

9. The automated event presentation system of claim 8, further comprising a plurality of viewer platforms in communication with the automated event presentation system that allows a plurality of remote viewers to simultaneously view different views of the broadcasted event.

10. A method for displaying at least a portion of an omni-directional image capturing an event occurring within an event environment, comprising:

capturing the event and automatically tracking multiple event participants simultaneously using audio and video processing techniques and a single omni-directional camera system having a single camera to produce the omni-directional image;

transmitting the omni-directional image from a broadcasting platform to a viewer platform using a computer network;

using probabilistic rules that receive as input audio and video tracking results from the audio and video processing techniques to automatically select in a probabilistic manner which portion of the omni-directional image to view;

using the viewer platform to allow a viewer to select which portion of the omni-directional image the viewer would like to view and displaying this selected portion of the omni-directional image to the viewer.

11. The method as set forth in claim 10, wherein the viewer selects to view multiple portions of the omni-directional image.

12. The method as set forth in claim 10, wherein the omni-directional image contains all event participants within the event environment.

13. The method of claim 10, further comprising implementing the probabilistic rules using a finite state machine.

14. The method as set forth in claim 10, further comprising:

transmitting a low-resolution version of the omni-directional image to the viewer platform, wherein the omni-directional image produced by the omni-directional camera system is a high-resolution omni-directional image;

selecting which portion of the omni-directional image to view, the selection being made by at least one of;

(a) manually by the viewer;

(b) automatically by a virtual director module; and

transmitting a high-resolution version of the selected portion of the omni-directional image to the viewer platform.

15. A computer-implemented method for capturing and viewing over a computer network an event having event participants, comprising:

capturing an omni-directional image of the event;

determining a number of the event participants and automatically tracking each of the event participants simultaneously within the omni-directional image using audio-based tracking techniques and video-based tracking techniques to obtain audio and video tracking results;

automatically selecting at least a portion of the omni-directional image for use as an output view containing a first event participant using a first set of probabilistic rules based on the audio and video tracking results;

switching the output view to a second event participant that is different from the first event participant when the second event participant begins speaking, unless the first event participant has been shown in the output view for less than a first specified amount of time;

switching the output view for a short duration, to a randomly-selected event participant, if an event participant who is speaking in the output view has been shown in the output view for longer than a second specified amount of time and then switching back to the output view containing the event participant who is speaking; and

displaying the output view to a viewer.

16. The computer-implemented method of claim 15, further comprising computing the audio and video tracking results by using a second set of probabilistic rules to combine outputs from the audio-based tracking techniques and the video-based tracking techniques.

17. The computer-implemented method of claim 16, further comprising:

implementing first set of probabilistic rules as a first probabilistic finite state machine; and

implementing second set of probabilistic rules as a second probabilistic finite state machine.

18. The computer-implemented method of claim 17, further comprising implementing first and second sets of probabilistic rules using.

19. The computer-implemented method of claim 15, further comprising allowing the viewer to choose which of the event participants to include in the output view, the choice being at least one of:

(a) a manual choice made by the viewer;

(b) an automatic choice based on the audio and video tracking results.

20. The computer-implemented method of claim 19, further comprising changing the output view by switching instantaneously between views of the event participants in the omni-directional image in response to the choice.