Automated camera management system and method for capturing presentations using videography rules

• US 7,349,008 B2
• Filed: 11/30/2002
• Issued: 03/25/2008
• Est. Priority Date: 11/30/2002
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Grudin, Jonathan Thomas, Rui, Yong, Gupta, Anoop
• Primary Examiner(s): RAO, ANAND SHASHIKANT
• Application Number:
  US10/307,088
• Publication Number:
  US 20040105004A1
• Time in Patent Office: 1,942 Days
• Field of Search: 348/14.08, 348169-172, 348/722, 386/69, 715/723
• US Class Current: 348/169
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources
• CPC Class Codes: H04N 7/185 from a mobile camera, e.g. for remote control
• CPC Class Codes: H04N 7/188 Capturing isolated or intermittent images triggered by the occurrence of a predetermined event, e.g. an object reaching a predetermined position signal generation from motion picture films H04N5/253

Abstract

An automated camera management system and method for capturing presentations using videography rules. The system and method use technology components and aesthetic components represented by the videography rules to capture a presentation. In general, the automated camera management method captures a presentation using videography rules to determine camera positioning, camera movement, and switching or transition between cameras. The videography rules depend on the type of presentation room and the number of audio-visual camera units used to capture the presentation. The automated camera management system of the invention uses the above method to capture a presentation in a presentation room. The system includes a least one audio-visual (A-V) camera unit for capturing and tracking a subject based on vision or sound. The (A-V) camera unit includes any combination of the following components: (1) a pan-tilt-zoom (PTZ) camera; (2) a fixed camera; and (3) a microphone array.

Claims

1. A method for capturing a presentation occurring in a presentation room, comprising:

determining a number of audio-visual camera units to use in the capture of the presentation;

determining a type of the presentation room;

determining applicable videography rules based on the number of audio-visual camera units and the type of presentation room;

capturing the presentation using the audio-visual camera units;

determining, setting, and fixing a zoom level of at least one of the audio-visual camera units based on a history of a presenter'"'"'s movements in the presentation room such that if the presenter moves around the presentation room a great deal then the zoom level is zoomed out more than if the presenter does not move around the presentation room a great deal;

changing the zoom level only if the presenter moves out of a field-of-view of the audio-visual camera unit; and

managing the capture of the presentation based on the applicable videography rules and the history of the presenter'"'"'s movements in the presentation room;

wherein the applicable videography rules further comprise camera positioning rules that determine a location in the presentation room of the audio-visual camera units;

wherein the camera positioning rules further comprise positioning a first one of the audio-visual camera units to avoid capture by any other audio-visual camera units.

2. The method as set forth in claim 1, wherein the camera positioning rules further comprise positioning the audio-visual camera units to avoid crossing a line of interest.

3. The method as set forth in claim 1, wherein the camera positioning rules further comprise positioning the audio-visual camera units to capture a target in one of the following ways:

(a) straight on with the target;

(b) at a slight angle with the target.

4. The method as set forth in claim 1, further comprising:

providing a presenter-tracking camera for capturing a presenter; and

providing an overview camera for capturing an overview of the presentation room.

5. The method as set forth in claim 4, wherein the camera positioning rules further comprise positioning the presenter-tracking camera and overview camera approximately at eye-level.

6. The method as set forth in claim 5, wherein the camera positioning rules further comprise positioning the presenter-tracking camera and overview camera such that obstructions from an audience are avoided.

7. The method as set forth in claim 1, wherein the applicable videography rules further comprise camera movement rules that determine the movement of the audio-visual camera units.

8. The method as set forth in claim 7, wherein the camera movement rules further comprise presenter tracking and framing rules for determining how to move the audio-visual camera units to frame and track a presenter in the presentation room.

9. The method as set forth in claim 8, wherein the presenter tracking and framing rules further comprise keeping a head shot with proper space above a head of the presenter.

10. The method as set forth in claim 9, wherein the head shot is one of:

(a) a tight head shot;

(b) a medium head shot.

11. The method as set forth in claim 9, wherein the proper space above the head of the presenter is one-half of a height of the head of the presenter.

12. The method as set forth in claim 8, wherein the presenter tracking and framing rules further comprise:

centering the presenter a majority of the time; and

giving lead room for the presenter'"'"'s gaze based on at least one of;

(a) a gaze direction of the presenter;

(b) a head orientation of the presenter.

13. The method as set forth in claim 8, wherein the presenter tracking and framing rules further comprise tracking the presenter.

14. The method as set forth in claim 8, wherein at least one of the audio-visual camera units is a presenter-tracking camera, and wherein the presenter tracking and framing rules further comprise restricting the movement of the presenter-tracking camera to when the presenter moves outside a specified zone.

15. The method as set forth in claim 7, wherein the camera movement rules further comprise audience tracking and framing rules for determining how to move the audio-visual camera units to frame and track audience members in the presentation room.

16. The method as set forth in claim 15, wherein the audience tracking and framing rules further comprise promptly showing an audience member that has done at least one of:

(a) asking a question;

(b) making a comment.

17. The method as set forth in claim 16, wherein the audience tracking and framing rules further comprise showing a wide shot of the audience if the audience member cannot be located.

18. The method as set forth in claim 16, wherein the audience tracking and framing rules further comprise remaining with a presenter if the audience member cannot be located.

19. The method as set forth in claim 15, wherein the audience tracking and framing rules further comprise avoiding shots showing few audience members relative to a size of the presentation room.

20. The method as set forth in claim 15, wherein the audience tracking and framing rules further comprise occasionally showing audience members for several seconds.

21. The method as set forth in claim 1, wherein the applicable videography rules further comprise shot transition rules that determine how and when switching between the audio-visual camera units should occur.

22. The method as set forth in claim 21, wherein the shot transition rules further comprise maintaining frequent and random shot changes to avoid making the shot changes mechanical and predictable.

23. The method as set forth in claim 21, wherein the shot transition rules further comprise maintaining a shot longer than a minimum shot duration to avoid distracting viewers.

24. The method as set forth in claim 23, wherein the minimum shot duration is between from three seconds to five seconds.

25. The method as set forth in claim 21, wherein the shot transition rules further comprise maintaining a shot for no longer than a maximum shot duration.

26. The method as set forth in claim 25, wherein the maximum shot duration is dependent on the type camera used in the audio-visual camera units.

27. The method as set forth in claim 21, wherein the shot transition rules further comprise switching shots when motivated by at least one of:

(a) shot duration;

(b) an event.

28. The method as set forth in claim 21, wherein the shot transition rules further comprise switching shots when a presenter has finished a concept.

29. The method as set forth in claim 21, wherein the shot transition rules further comprise switching shots when an audience member has finished asking a question.

30. A method for capturing a presentation, comprising:

providing at least one audio-visual camera unit that includes a presenter-tracking camera;

positioning and moving the audio-visual camera units based on videography rules;

focusing the presenter-tracking camera on a presenter'"'"'s current location in a presentation room;

maintaining a trajectory of the presenter'"'"'s location in the presentation room during a previous time period and designating the trajectory during the time period as an activity area;

defining a bounding box as a rectangle containing the activity area;

determining, setting, and fixing a zoom level of the presenter-tracking camera based on a history of the presenter'"'"'s movements such that if the presenter moves around the presentation room a great deal then the zoom level is zoomed out more than if the presenter does not move around the presentation room a great deal and such that the entire activity area is captured to frame the presenter with the presenter-tracking camera within the bounding box, determining the zoom level, Z_L, further comprising using the equation;

$Z_L=\min <br><br>\left(\frac{\mathrm{HFOV}}{\mathrm{\angle <br><br>}<br><br>\left(X_R,X_L\right)},\frac{\mathrm{VFOV}}{\mathrm{\angle <br><br>}<br><br>\left(Y_B,Y_T\right)}\right),$

where HFOV is the horizontal field of view of the presenter-tracking camera, VFOV is the vertical field of view of the presenter-tracking camera, and ∠

represents an angle spanned by the two arguments in a coordinate system of the presenter-tracking camera;

changing the zoom level only if the presenter moves out of a field-of-view of the audio-visual camera unit; and

automatically capturing the presentation using the at least one audio-visual camera unit and the videography rules;

wherein the videography rules are based on a type of presentation room in which the presentation occurs and the type of presentation room is a large presentation room;

wherein the audio-visual camera unit further comprises a presenter-tracking camera and the videography rules further comprise positioning the presenter-tracking camera higher from a floor in the large presentation room relative to a position of the presenter-tracking camera in a medium presentation room.

31. The method as set forth in claim 30, wherein the videography rules are based on a number of audio-visual camera units used to automatically capture the presentation.

32. The method as set forth in claim 31, wherein:

there is a plurality of audio-visual camera units;

at least one of the plurality of audio-visual camera units contains a plurality of cameras; and

switching between cameras is based on the videography rules.

33. A computer-readable medium having computer-executable instructions for managing capture of a presentation in a presentation room comprising:

determining videography rules for managing the capture based on the type of presentation room;

positioning two audio-visual camera units based on the videography rules;

determining a history of a presenter'"'"'s movement in the presentation room and using the history to determine a framing of the presenter based on the videography rules; and

determining a zoom level of at least one of the two audio-visual camera units based on the history of the presenter'"'"'s movements such that an entire activity area of the presenter is captured;

determining a location of an audience member in an audience of the presentation room by using a sound source localization technigue that uses a hybrid weighting function, the hybrid weighting function defined as;

$W<br><br>\left(\mathrm{\omega <br><br>}\right)=\frac{1}{\mathrm{\gamma <br><br>}<br><br><br><br>G_{x_1<br><br>x_2}<br><br>\left(\mathrm{\omega <br><br>}\right)<br><br>+\left(1-\mathrm{\gamma <br><br>}\right)<br><br>{<br><br>N<br><br>\left(\mathrm{\omega <br><br>}\right)<br><br>}^2},$

where γ

ε

[0,1] is a proportion factor and,

|N′

(ω

)|²=H(ω

)|²|S(ω

)|²+|N(ω

)|².

34. The computer-readable medium as set forth in claim 33, wherein the type of presentation room is a medium presentation room.

35. The computer-readable medium as set forth in claim 33, wherein the type of presentation room is a large presentation room.

36. The computer-readable medium as set forth in claim 33, wherein the type of presentation room is a small presentation room.

37. A method for automatically capturing a presentation in a presentation room, comprising:

determining videography rules for the capture based on the type of presentation room;

positioning three audio-visual camera units based on the videography rules using a virtual cameraman to control the audio-visual camera units;

determining a history of a presenter'"'"'s movement in the presentation room;

framing shots of the presenter from at least one of the audio-visual camera units based on the history and the videography rules; and

determining a zoom level of at least one of the three audio-visual camera units based on the history of the presenter'"'"'s movements such that an entire activity area of the presenter is captured;

using a virtual director to select and determine which of the audio-visual camera units is an active camera;

using the virtual cameramen to report a status to the virtual director, the status further comprising the following commands;

(a) mode status;

(b) action status;

(c) scene status;

(d) score status;

(e) confidence status.

38. A method for capturing a moving presenter giving a presentation in a presentation room, comprising:

determining applicable videography rules for managing the capture of the presentation;

determining a history of the presenter'"'"'s movement; and

positioning two audio-visual camera units based on the videography rules using a virtual cameraman; and

determining, setting, and fixing a zoom level of at least one of the two audio-visual camera units based on the history of the presenter'"'"'s movements such that an entire activity area of the presenter is captured, such that if the presenter moves around the presentation room a great deal then the zoom level is zoomed out more than if the presenter does not move around the presentation room a great deal;

changing the zoom level only if the presenter moves out of a field-of-view of any one of the two audio-visual camera units;

using a virtual director to select and determine which of the audio-visual camera units is an active camera;

using the virtual cameramen to report a status to the virtual director, the status further comprising the following commands;

(a) mode status;

(b) action status;

(c) scene status;

(d) score status;

(e) confidence status.