Portable solution for automatic camera management

• US 7,512,883 B2
• Filed: 06/30/2004
• Issued: 03/31/2009
• Est. Priority Date: 06/30/2004
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Wallick, Michael, He, Le-wei, Rui, Yong
• Primary Examiner(s): Huynh; Ba
• Assistant Examiner(s): Augustine; Nicholas
• Application Number:
  US10/883,123
• Publication Number:
  US 20060005136A1
• Time in Patent Office: 1,735 Days
• Field of Search: 715/716, 715719-726, 715/730, 715/757, 715764-765, 715/767, 348/14.05, 348/211.12, 348/159, 709/205
• US Class Current: 715/717
• CPC Class Codes: H04M 9/082 using echo cancellers echo cancellers per se H04B3/23

Abstract

A “virtual video studio”, as described herein, provides a highly portable real-time capability to automatically capture, record, and edit a plurality of video streams of a presentation, such as, for example, a speech, lecture, seminar, classroom instruction, talk-show, teleconference, etc., along with any accompanying exhibits, such as a corresponding slide presentation, using a suite of one or more unmanned cameras controlled by a set of videography rules. The resulting video output may then either be stored for later use, or broadcast in real-time to a remote audience. This real-time capability is achieved by using an abstraction of “virtual cameramen” and physical cameras in combination with a scriptable interface to the aforementioned videography rules for capturing and editing the recorded video to create a composite video of the presentation in real-time under the control of a “virtual director.”

Claims

1. An automated video production system for creating a composite video of a presentation in an arbitrary lecture room given a set of one or more camera inputs, comprising:

a set of predefined user selectable abstracted objects which include one or more predefined virtual directors and one or more predefined virtual cameramen;

a set of one or more physical cameras, each physical camera capable of capturing one or more views within the lecture room during a presentation;

a graphical user interface for selecting a virtual director and one or more virtual cameramen from the set of abstracted objects for inclusion in a user scriptable set of videography rules;

wherein the user scriptable set of videography rules includes a user selected assignment of one or more of the set of physical cameras to one or more of the selected virtual cameramen, said assignments being made via the graphical user interface, and wherein the user scriptable set of videography rules defines a layout of the arbitrary lecture room;

wherein each of the virtual cameramen receives and processes an input video stream from its assigned physical camera, said processing of the input video stream being performed in accordance with predefined videography rules associated with each of the particular virtual cameramen;

wherein the virtual director automatically and continuously interprets the set of user scriptable videography rules in real-time during the presentation for automatically ordering broadcast transitions to specific virtual cameramen in accordance with the set of videography rules; and

selecting whichever of the virtual cameramen is ordered to a broadcast transition by the virtual director as each automatic transition order occurs, and providing the processed input video stream of the selected virtual cameramen as a current input to a continuous composite video stream of the presentation.

2. The automated video production system of claim 1 wherein the set of videography rules further includes:

at least one state definition; and

at least one set of transition rules for determining whether a broadcast transition is to be ordered by the virtual director as a result of satisfying any of the transition state definitions.

3. The automated video production system of claim 2 wherein the at least one set of transition rules includes a bias setting for favoring transition to one broadcast over another during a random selection of broadcast states.

4. The automated video production system of claim 2 wherein the scriptable set of videography rules further includes one or more of:

a minimum time for transitioning between broadcast states;

a maximum time for transitioning between broadcast states;

a confidence level for indicating whether a particular virtual cameraman is currently successfully capturing its associated view of the lecture room; and

a score which indicates how good a particular video shot is likely to be.

5. The automated video production system of claim 1, wherein two or more of the virtual cameramen each process an output of a single high resolution camera to provide a different one of the views within the lecture room.

6. The automated video production system of claim 1, wherein the predefined virtual cameramen include one or more of:

a virtual overview cameraman for capturing an overview of the lecture room;

a virtual slide tracking cameraman for capturing a view of an area of the presentation room in which presentation slides are displayed; and

a virtual speaker tracking cameraman for capturing a view of a presenter within the presentation room.

7. The automated video production system of claim 6, wherein the virtual speaker tracking cameraman automatically tracks a current location of the presenter within the presentation room.

8. The automated video production system of claim 7, wherein automatically tracking the current location of the presenter within the presentation room comprises using a histogram-based tracking system for locating a horizontal position of the presenter and an approximate vertical position of the presenters head.

9. The automated video production system of claim 7, tracking the current location of the presenter within the presentation room further comprises automatically bracketing an area around the location of the presenter with one of the set of one or more cameras so as to ensure that the presenter is within the image frames of the input video stream being received by the virtual speaker tracking cameraman.

10. A computer-readable medium having computer-executable instructions for performing the automated video production system of claim 1.

11. A method for capturing a composite video of a presentation in a lecture room, comprising:

providing a set of predefined user selectable abstracted objects which include one or more predefined virtual directors and one or more predefined virtual cameramen;

providing a graphical user interface for editing a user adjustable control script to specify one or more of the user selectable abstracted objects, including one of the virtual directors and a set of at least one virtual cameramen;

automatically determining a user configurable number, type, and layout of video cameras to be used to capture a presentation in a particular lecture room based on user selectable abstracted objects specified in the user adjustable control script to automatically construct a set of videography rules specific to the number, type, and layout of video cameras, and to the particular lecture room;

wherein the specified virtual director represents an abstract control script interpreter for automatically interpreting the videography rules in real-time;

wherein the specified set of virtual cameramen represent a set of one or more video processors, each video processor having a defined type, and each video processor being responsible for monitoring an input from at least one of the number of cameras;

using the abstract control script interpreter to control output states and transitions for each video processor; and

wherein the abstract control script interpreter directs combining a sequential output from one or more of the video processors to create a composite video of the presentation in compliance with the set of videography rules of the control script.

12. The method of claim 11 wherein the defined types of video processors comprise at least one of:

a virtual overview cameraman for capturing an overview of the lecture room;

a virtual slide tracking cameraman for capturing a view of an area of the presentation room in which presentation slides are displayed; and

a virtual speaker tracking cameraman for tracking and capturing a presenter within the presentation room.

13. The method of claim 12 wherein at least one of the virtual cameramen is capable of directing pan, tilt and zoom operations for the camera being monitored by that virtual cameraman.

14. The method of claim 13 wherein the pan, tilt and zoom operations are preformed digitally on a digital video input.

15. The method of claim 13 wherein the pan, tilt and zoom operations are directed to be performed by a camera with automatic pan, tilt and zoom capabilities.

16. The method of claim 11 wherein the control script is in a human readable format.

17. A computer-readable storage medium having computer executable instructions for providing a portable video production system, said computer executable instructions comprising:

providing a user interface for configuring a user adaptable set of video production rules;

wherein the video production rules include a set of predefined user selectable abstracted objects which include one or more predefined virtual directors and one or more predefined virtual cameramen, and wherein the user scriptable set of videography rules define a layout of an arbitrary lecture room;

wherein the video production rules further include a user selectable set of one or more cameras coupled to a computer, and wherein assignments of each selected camera to one or more of the selected virtual cameramen are specified via the user interface;

capturing a plurality of input video streams of a presentation in the lecture room by using the selected cameras to capture one or more unique views in accordance with video capture characteristics associated with the virtual cameramen assigned to the specific cameras;

wherein each virtual cameraman receives an input video stream from one of the cameras; and

wherein the virtual director automatically interprets the set of user adaptable video production rules on an ongoing basis in real-time for the purpose of ordering one or more of the virtual cameramen to output a processed version of the input video streams received by those virtual cameramen.

18. The computer-readable medium of claim 17 wherein ordering one or more of the virtual cameramen to output a processed version of the input video streams includes transitioning from a video broadcast of one virtual cameraman to another in accordance with the set of video production rules.

19. The computer-readable medium of claim 18 wherein the user configurable set of video production rules further include one or more of:

a bias setting for favoring transition to one video broadcast over another during a random selection of video broadcasts from between one or more of the virtual cameramen; and

at least one set of transition rules for determining when a transition from one video broadcast to another is to be ordered by the virtual director.

20. The computer-readable medium of claim 19 wherein the set of video production rules further includes one or more of:

a minimum time for transitioning between video broadcasts;

a maximum time for transitioning between video broadcasts;

a confidence level for indicating whether a particular virtual cameraman is currently successfully capturing a particular view of the lecture room; and

a score which indicates how good a particular video frame is likely to be.

21. The computer-readable medium of claim 17, wherein a single high resolution camera provides the input video stream to two or more of the virtual cameramen while producing different views of the presentation via any of a digital pan, tilt, or zoom of the input video stream by one or more of the virtual cameramen.

22. The computer-readable medium of claim 17, wherein the set of virtual cameramen includes one or more of:

a virtual overview cameraman for capturing an overview of the lecture room;

a virtual slide tracking cameraman for capturing a view of an area of the presentation room in which presentation slides are displayed; and

a virtual speaker tracking cameraman for capturing a view of a presenter within the lecture room.

23. The computer-readable medium of claim 22, wherein the virtual speaker tracking cameraman automatically tracks a current location of the presenter within the lecture room by using a histogram-based tracking system for locating a horizontal position of the presenter and an approximate vertical position of the presenters head.