Portable solution for automatic camera management

• US 20060005136A1
• Filed: 06/30/2004
• Published: 01/05/2006
• Est. Priority Date: 06/30/2004
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Rui, Yong, He, Li-wei, Wallick, Michael N.
• Granted Patent: US 7,512,883 B2
• Time in Patent Office: Days
• US Class Current: 715/726
• CPC Class Codes: H04M 9/082 using echo cancellers echo cancellers per se H04B3/23

Abstract

A “virtual video studio”, as described herein, provides a highly portable real-time capability to automatically capture, record, and edit a plurality of video streams of a presentation, such as, for example, a speech, lecture, seminar, classroom instruction, talk-show, teleconference, etc., along with any accompanying exhibits, such as a corresponding slide presentation, using a suite of one or more unmanned cameras controlled by a set of videography rules. The resulting video output may then either be stored for later use, or broadcast in real-time to a remote audience. This real-time capability is achieved by using an abstraction of “virtual cameramen” and physical cameras in combination with a scriptable interface to the aforementioned videography rules for capturing and editing the recorded video to create a composite video of the presentation in real-time under the control of a “virtual director.”

Claims

1. An automated video production system for creating a composite video of a presentation in a lecture room given a set of one or more camera inputs, comprising:

a set of one or more cameras for capturing a plurality of views within the lecture room during a presentation;

a scriptable set of videography rules tailored to the set of one or more cameras and the views within the lecture room;

a set of virtual cameramen, each said virtual cameramen receiving and processing an input video stream of one of the views from one of the set of cameras;

a virtual director that automatically and continuously interprets the set of scriptable videography rules in real-time during the presentation for automatically ordering a broadcast transition to one the virtual cameramen in accordance with the set of videography rules; and

selecting whichever of the virtual cameramen is ordered to a broadcast transition as each automatic transition order occurs, and providing the processed input video stream of the selected virtual cameramen as a current input to a continuous composite video stream of the presentation.

2. The automated video production system of claim 1 wherein the set of videography rules includes:

at least one virtual cameraman definition;

at least one state definition; and

at least one set of transition rules for determining whether a broadcast transition is to be ordered by the virtual director as a result of satisfying any of the transition state definations.

3. The automated video production system of claim 2 wherein the at least one set of transition rules includes a bias setting for favoring transition to one broadcast over another during a random selection of broadcast states.

4. The automated video production system of claim 2 wherein the scriptable set of videography rules further includes one or more of:

a minimum time for transitioning between broadcast states;

a maximum time for transitioning between broadcast states;

a confidence level for indicating whether a particular virtual cameraman is currently successfully capturing its associated view of the lecture room; and

a score which indicates how good a particular video shot is likely to be.

5. The automated video production system of claim 1, wherein a single high resolution camera provides the input video stream of one of the views to two or more of the virtual cameramen.

6. The automated video production system of claim 1, wherein the set of virtual cameramen includes one or more of:

a virtual overview cameraman for capturing an overview of the lecture room;

a virtual slide tracking cameraman for capturing a view of an area of the presentation room in which presentation slides are displayed; and

a virtual speaker tracking cameraman for capturing a view of a presenter within the presentation room.

7. The automated video production system of claim 6, wherein the virtual speaker tracking cameraman automatically tracks a current location of the presenter within the presentation room.

8. The automated video production system of claim 7, wherein automatically tracking the current location of the presenter within the presentation room comprises using a histogram-based tracking system for locating a horizontal position of the presenter and an approximate vertical position of the presenters head.

9. The automated video production system of claim 7, tracking the current location of the presenter within the presentation room further comprises automatically bracketing an area around the location of the presenter with one of the set of one or more cameras so as to ensure that the presenter is within the image frames of the input video stream being received by the virtual speaker tracking cameraman.

10. The automated video production system of claim 1, further comprising a user interface for editing the scriptable set of videography rules.

11. A computer-readable medium having computer-executable instructions for performing the automated video production system of claim 1.

12. A method for capturing a composite video of a presentation in a lecture room, comprising:

determining a number, type, and layout of video cameras to be used to capture a presentation in a particular lecture room;

providing a control script comprising a set of videography rules specific to the number, type, and layout of video cameras, and to the particular lecture room;

providing a generic abstract control script interpreter for automatically interpreting the videography rules in real-time;

providing a set of one or more generic video processors, each video processor having a defined type, and each video processor being responsible for monitoring an input from at least one of the number of cameras;

using the generic abstract control script interpreter to control output states and transitions for each generic video processor; and

combining a sequential output from one or more of the generic video processors to create a composite video of the presentation in compliance with the set of videography rules of the control script.

13. The method of claim 12 wherein the defined types of video processors comprise at least one of:

a virtual overview cameraman for capturing an overview of the lecture room;

a virtual slide tracking cameraman for capturing a view of an area of the presentation room in which presentation slides are displayed; and

a virtual speaker tracking cameraman for tracking and capturing a presenter within the presentation room.

14. The method of claim 13 wherein at least one of the virtual cameramen is capable of directing pan, tilt and zoom operations for the camera being monitored by that virtual cameraman.

15. The method of claim 14 wherein the pan, tilt and zoom operations are preformed digitally on a digital video input.

16. The method of claim 14 wherein the pan, tilt and zoom operations are directed to be performed by a camera with automatic pan, tilt and zoom capabilities.

17. The method of claim 12 further comprising providing a user interface for editing the control script.

18. The method of claim 12 wherein the control script is in a human readable format.

19. A computer-readable medium having computer executable instructions for providing a portable video production system, said computer executable instructions comprising:

capturing a plurality of input video streams of a presentation in a lecture room using a set of one or more cameras coupled to a computer;

providing a user configurable set of video production rules;

providing a set of one or more unique virtual cameramen, with each virtual cameraman receiving an input video stream from one of the cameras; and

providing a virtual director, said virtual director automatically interpreting the set of user configurable video production rules on an ongoing basis in real-time for the purpose of ordering one or more of the virtual cameramen to output a processed version of the input video streams received by those virtual cameramen.

20. The computer-readable medium of claim 19 wherein ordering one or more of the virtual cameramen to output a processed version of the input video streams includes transitioning from a video broadcast of one virtual cameraman to another in accordance with the set of video production rules.

21. The computer-readable medium of claim 20 wherein the user configurable set of video production rules include one or more of:

at least one camera definition;

a bias setting for favoring transition to one video broadcast over another during a random selection of video broadcasts from between one or more of the virtual cameramen; and

at least one set of transition rules for determining when a transition from one video broadcast to another is to be ordered by the virtual director.

22. The computer-readable medium of claim 21 wherein the set of video production rules further includes one or more of:

a minimum time for transitioning between video broadcasts;

a maximum time for transitioning between video broadcasts;

a confidence level for indicating whether a particular virtual cameraman is currently successfully capturing a particular view of the lecture room; and

a score which indicates how good a particular video frame is likely to be.

23. The computer-readable medium of claim 19, wherein a single high resolution camera provides the input video stream to two or more of the virtual cameramen while producing different views of the presentation via any of a digital pan, tilt, or zoom of the input video stream by one or more of the virtual cameramen.

24. The computer-readable medium of claim 19, wherein the set of virtual cameramen includes one or more of:

a virtual overview cameraman for capturing an overview of the lecture room;

a virtual slide tracking cameraman for capturing a view of an area of the presentation room in which presentation slides are displayed; and

a virtual speaker tracking cameraman for capturing a view of a presenter within the lecture room.

25. The computer-readable medium of claim 24, wherein the virtual speaker tracking cameraman automatically tracks a current location of the presenter within the lecture room by using a histogram-based tracking system for locating a horizontal position of the presenter and an approximate vertical position of the presenters head.