Automated video production system and method using expert video production rules for online publishing of lectures

• US 20020196327A1
• Filed: 06/14/2001
• Published: 12/26/2002
• Est. Priority Date: 06/14/2001
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Rui, Yong, Gupta, Anoop, Liu, Qiong
• Granted Patent: US 7,349,005 B2
• Time in Patent Office: Days
• US Class Current: 348/14.11
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources
• CPC Class Codes: H04N 7/188 Capturing isolated or intermittent images triggered by the occurrence of a predetermined event, e.g. an object reaching a predetermined position signal generation from motion picture films H04N5/253

Abstract

An automated system and method for producing videos using expert video production rules. The automated video production system and method of the present invention is particularly well-suited for the online publishing of lectures. The system of the present invention includes a camera system for capturing the lecture, such as an audience-tracking camera and a non-intrusive lecturer-tracking camera. Moreover, each camera may be controlled by a cinematographer. Tracking of a subject is performed using a history-based, reduced-motion tracker that sets up a camera shot based on the subject'"'"'s movement history and leaves the shot fixed until the camera is switched. The system also includes a virtual director module having a probabilistic finite state machine (FSM) module that uses probabilistic rules and the expert video production rules to determine a current camera view.

Claims

1. An automated video production system for online publishing of a lecture, comprising:

a camera system that provides multiple camera views of the lecture;

a virtual director that selects a current camera view from the multiple camera views and is capable changing the current camera view by switching between the multiple camera views in response to a triggering event; and

a set of expert video production rules that is applied by the virtual director to select the current camera view.

2. The automated video production system as set forth in claim 1, wherein the set of expert video production rules is applied by the virtual director to determine to which of the multiple camera views to switch.

3. The automated video production system as set forth in claim 1, wherein the set of expert video production rules are applied by the virtual director to generate the triggering event and determine to when to switch the current camera view.

4. The automated video production system as set forth in claim 1, wherein the camera system includes a single camera.

5. The automated video production system as set forth in claim 1, wherein the camera system includes a plurality of cameras.

6. The automated video production system as set forth in claim 1, further comprising a virtual cinematographer that controls a camera in tracking an object within the lecture.

7. The automated video production system as set forth in claim 1, wherein the camera system includes an audience-tracking camera that provides a camera view of an audience.

8. The automated video production system as set forth in claim 7, further comprising a microphone-array audience tracker that controls the audience-tracking camera in tracking a member of the audience.

9. The automated video production system as set forth in claim 7, further comprising an audience-tracking status module that provides status information of the audience-tracking camera to the virtual director.

10. The automated video production system as set forth in claim 9, wherein the status information includes a plurality of possible statuses.

11. The automated video production system as set forth in claim 1, wherein the camera system includes a lecturer-tracking camera that provides a camera view of a lecturer.

12. The automated video production system as set forth in claim 11, further comprising a history-based, reduced-motion tracker that controls the lecturer-tracking camera in tracking the lecturer based on a history of the lecturer'"'"'s movement.

13. The automated video production system as set forth in claim 11, further comprising a lecturer-tracking status module that provides status information of the lecturer-tracking camera to the virtual director.

14. The automated video production system as set forth in claim 1, wherein the virtual director includes an event generator that generates the triggering event.

15. The automated video production system as set forth in claim 14, wherein the event generator further comprises a time transition module that determines when to switch the current camera view.

16. The automated video production system as set forth in claim 14, wherein the event generator further comprises a location transition module that determines to which of the multiple camera views to switch.

17. The automated video production system as set forth in claim 16, wherein the location transition module is a probabilistic finite state machine having multiple states.

19. A computer-readable medium having computer-executable instructions for performing the method recited in claim 18.

20. The method as set forth in claim 18, wherein the set of expert video production rules are a set of constraints for producing the video.

21. The method as set forth in claim 18, wherein the set of expert video production rules is obtained by determining rules used by human experts in a video production field.

22. The method as set forth in claim 18, wherein the current camera view is changed in response to a triggering event.

23. The method as set forth in claim 22, wherein the triggering event is generated based on the set of expert video production rules.

24. The method as set forth in claim 18, wherein determining to which of the multiple camera views the current camera view should change is based on probabilistic approach.

25. The method as set forth in claim 24, wherein the probabilistic approach uses a probabilistic finite state machine having multiple states that are fully connected to allow transition from one state to another.

26. The method as set forth in claim 18, wherein the camera system includes an audience-tracking camera that provides a camera view of an audience of the lecture.

27. The method as set forth in claim 26, further comprising using the set of expert video production rules to control the audience-tracking camera in tracking of members of the audience.

28. The method as set forth in claim 18, wherein the camera system includes a lecturer-tracking camera.

29. The method as set forth in claim 28, wherein the lecturer-tracking camera tracks the lecturer using the set of expert video production rules.

30. The method as set forth in claim 28, wherein the lecturer-tracking camera tracks the lecturer based on a history of the lecturer'"'"'s movement.

18. A method for automatically producing a video of a lecture for online publishing, comprising:

providing a set of expert video production rules;

capturing the lecture using a camera system that includes multiple camera views; and

using the set of expert video production rules to determine a current camera view from the multiple camera views, when the current camera view should change, and to which of the multiple camera views the current camera view should change.

31. An automated video production system for capturing images of a scene, comprising:

an audience-tracking camera that provides images of an audience within the scene;

a lecturer-tracking camera that non-intrusively tracks a lecturer within the scene;

a set of expert video production rules containing video production constraints; and

a virtual director module that receives multiple camera views from the audience-tracking camera and the lecturer-tracking camera and uses the set of expert video production rules to select an current camera view from the multiple camera views.

32. The automated video production system as set forth in claim 31, further comprising an overview camera that provides an overview image of the scene.

33. The automated video production system as set forth in claim 31, wherein the current camera view is capable of changing to one of the multiple camera views in response to a triggering event.

34. The automated video production system as set forth in claim 33, wherein the set of expert video production rules is used to determine when the triggering event is generated.