System and method for servoing on a moving fixation point within a dynamic scene

• US 7,027,083 B2
• Filed: 02/12/2002
• Issued: 04/11/2006
• Est. Priority Date: 02/12/2001
• Status: Active Grant

• Current Assignee: Carnegie Mellon University
• Original Assignee: Carnegie Mellon University
• Inventors: Kanade, Takeo, Miller, Ryan, Hua, Wei, Collins, Robert, Amidi, Omead
• Primary Examiner(s): Le, Vu
• Assistant Examiner(s): SENFI, BEHROOZ M
• Application Number:
  US10/074,314
• Publication Number:
  US 20020118286A1
• Time in Patent Office: 1,519 Days
• Field of Search: 348/143, 348/222, 348/211.9, 348/169, 348/170, 348/142, 348/171, 348/172, 382/103, 382/154, 382/23
• US Class Current: 348/159
• CPC Class Codes: H04N 23/662 by using master/slave camera arrangements for affecting the control of camera image capture, e.g. placing the camera in a desirable condition to capture a desired image
• CPC Class Codes: H04N 23/695 Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects
• CPC Class Codes: H04N 5/222 Studio circuitry; Studio devices; Studio equipment cameras or camera modules comprising electronic image sensors, or control thereof H04N23/00
• CPC Class Codes: H04N 5/262 Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
• CPC Class Codes: H04N 5/2627 for providing spin image effect, 3D stop motion effect or temporal freeze effect 2D image animation in general G06T13/80

Abstract

A system and method for servoing on a moving target within a dynamic scene. According to one embodiment, the system includes a master variable pointing camera system and a plurality of slave variable pointing camera systems positioned around the scene. The system also includes a master control unit in communication with the master variable pointing camera system. The master control unit is for determining, based on parameters of the master variable pointing camera system, parameters for each of the slave variable pointing camera systems such that, at a point in time, the master variable pointing camera system and the slave variable pointing camera systems are aimed at the target and a size of the target in an image from each of the master variable pointing camera system and the slave variable pointing camera systems is substantially the same. The system also includes a plurality of slave camera control units in communication with the master control unit. The slave camera control units are for controlling at least one of the slave variable pointing camera systems based on the parameters for each of the slave variable pointing camera systems. The system may also include a video image sequence generator in communication with the master control unit and the slave camera control units. The video image sequence generator may generate a video image sequence of the target by outputting an image from certain of the master variable pointing camera system and the slave variable pointing camera systems in sequence according to the position of the master variable pointing camera system and the slave variable pointing camera systems around the scene.

Claims

1. A system for servoing on a moving target within a dynamic scene, comprising:

a master variable pointing camera system;

a plurality of slave variable pointing camera systems, wherein the slave variable pointing camera systems and the master variable pointing camera system are positioned around the scene;

a master control unit in communication with the master variable pointing camera system for determining, based on parameters of the master variable pointing camera system and mapping data between the master variable pointing camera system and the slave variable pointing camera systems, parameters for each of the slave variable pointing camera systems such that, at a point in time, the master variable pointing camera system and the slave variable pointing camera systems are aimed at the target and a size of the target in an image from each of the master variable pointing camera system and the slave variable pointing camera systems is substantially the same; and

a plurality of slave camera control units in communication with the master control unit, wherein each slave camera control unit is for controlling at least one of the slave variable pointing camera systems based on the parameters for each of the slave variable pointing camera systems,

wherein the mapping data includes;

data regarding the geometric relationship of the camera systems to the scene;

data regarding the relationship between the zoom and the angular field of view for each camera system; and

data regarding the relationship between the focus and the depth of field for each camera system.

2. The system of claim 1, wherein the parameters of the master variable pointing camera system and the parameters for the slave variable pointing camera systems include pointing parameters and optical parameters.

3. The system of claim 2, wherein:

the master variable pointing camera system includes a master pan/tilt camera system; and

the plurality of slave variable pointing cameras includes a plurality of slave pan/tilt camera systems.

4. The system of claim 3, wherein the wherein the parameters of the master variable pointing camera system and the parameters for the slave variable pointing camera systems include pan, tilt, zoom and focus parameters.

5. The system of claim 2, further comprising a video image sequence generator in communication with the master control unit and the slave camera control units.

6. The system of claim 5, wherein the video image sequence generator is for generating a video image sequence of the target by outputting an image from certain of the master variable pointing camera system and the slave variable pointing camera systems in sequence according to the position of the master variable pointing camera system and the slave variable pointing camera systems around the scene.

7. The system of claim 5, further comprising a computer vision control unit in communication with the master control unit and the master variable pointing camera system.

8. The system of claim 5, further comprising a remote operator interface unit in communication with the master control unit and the master variable pointing camera system.

9. The system of claim 5, wherein the master control unit includes:

a target determination module for determining a position of the target within the scene and a size of the target at the position in an image from the master variable pointing camera system based on the parameters of the master variable pointing camera system; and

a slave control module in communication with the target determination module for determining the parameters for each of the slave variable pointing camera systems based on the position of the target and the size of the target in the image from the master variable pointing camera system.

10. The system of claim 6, wherein the image sequence generator is for outputting images from each of the master variable pointing camera system and the slave variable pointing camera systems according to their position around the scene, wherein the images are from a common point in time so as to generate a 3 D stop-motion video image sequence.

11. A method for servoing on a moving target within a dynamic scene, comprising:

reading parameters of a first variable pointing camera system;

determining parameters for a plurality of other variable pointing camera systems based on the parameters of the first variable pointing camera system and mapping data for the camera systems, wherein the first variable pointing camera system and the plurality of other variable pointing camera systems are positioned around the scene, such that, at a point in time, each of the variable pointing camera systems is aimed at the target and a size of the target in an image from each of the variable pointing camera systems is substantially the same; and

controlling the plurality of other variable pointing camera systems based on the parameters for the plurality of other variable pointing camera systems

wherein the mapping data includes;

data regarding the geometric relationship of the camera systems to the scene;

data regarding the relationship between the zoom and the angular field of view for each camera system; and

data regarding the relationship between the focus and the depth of field for each camera system.

12. The method of claim 11, wherein:

reading parameters of the first variable pointing camera system includes reading mechanical and optical parameters of the first variable pointing camera system; and

determining parameters for the plurality of other variable pointing camera systems includes determining mechanical and optical parameters for the plurality of other variable pointing camera systems.

13. The method of claim 12, wherein the first variable pointing camera system includes a first pan/tilt camera system, wherein the plurality of other variable pointing camera systems include a plurality of other pan/tilt camera systems, and wherein:

reading parameters of the first pan/tilt camera system includes reading pan, tilt, zoom and focus parameters of the first pan/tilt camera system; and

determining parameters for the plurality of other pan/tilt camera systems includes determining pan, tilt, zoom and focus parameters for the plurality of other pan/tilt camera systems.

14. The method of claim 11, further comprising:

storing digitized, time-stamped images from the variable pointing camera systems; and

generating a video image sequence of the target by outputting an image from certain of the variable pointing camera systems in sequence according to the position of the variable pointing camera systems around the scene.

15. The method of claim 11, wherein determining parameters for the plurality of other variable pointing camera systems includes:

determining a position of the target within the scene and a size of the target at the position in an image from the first variable pointing camera system based on the parameters of the first variable pointing camera system; and

determining the parameters for each of the other variable pointing camera systems based on the position of the target and the size of the target in the image from the first variable pointing camera system.

16. The method of claim 11, further comprising selecting one of the other variable pointing camera systems to be the first variable pointing camera system.

17. The method of claim 14, wherein generating the video image sequence includes generating a 3 D stop-motion video image sequence of the target by outputting images from each of the variable pointing camera systems according to their position around the scene, wherein the images are from a common point in time.

18. A system for servoing on a moving target within a dynamic scene, comprising:

a plurality of master variable pointing camera systems;

a plurality of slave variable pointing camera systems, wherein the slave variable pointing camera systems and the master variable pointing camera systems are positioned around the scene;

at least one master control unit in communication with each of the master variable pointing camera systems for determining, based on parameters of the master variable pointing camera systems and mapping data for the camera systems, parameters for certain of the plurality of the slave variable pointing camera systems such that, at a point in time, each of the master variable pointing camera systems and the slave variable pointing camera systems are aimed at the target and a size of the target in an image from each of the master variable pointing camera systems and the slave variable pointing camera systems is substantially the same; and

a plurality of slave camera control units, each slave camera control unit in communication with at least one of the master control units, wherein each slave camera control unit is for controlling at least one of the slave variable pointing camera systems based on the parameters for each of the slave variable pointing camera systems,

wherein the mapping data includes;

data regarding the geometric relationship of the camera systems to the scene;

data regarding the relationship between the zoom and the angular field of view for each camera system; and

data regarding the relationship between the focus and the depth of field for each camera system.

19. The system of claim 18, wherein the at least one master control unit includes a plurality of master control units, and wherein each of the master control units is in communication with at least one master variable pointing camera system and is for determining, based on parameters of the master variable pointing camera systems, parameters for certain of the plurality of the slave variable pointing camera systems such that, at a point in time, each of the master variable pointing camera systems and the slave variable pointing camera systems are aimed at the target and a size of the target in an image from each of the master variable pointing camera systems and the slave variable pointing camera systems is substantially the same.