System and method for stabilizing rotational images

• US 7,102,666 B2
• Filed: 02/12/2002
• Issued: 09/05/2006
• Est. Priority Date: 02/12/2001
• Status: Expired due to Fees

• Current Assignee: Carnegie Mellon University
• Original Assignee: Carnegie Mellon University
• Inventors: Kanade, Takeo, Collins, Robert
• Primary Examiner(s): An, Shawn S.
• Application Number:
  US10/074,315
• Publication Number:
  US 20020118969A1
• Time in Patent Office: 1,666 Days
• Field of Search: 348/36, 348/39, 348 46- 48, 348/135, 348/139, 348/142, 348/143, 348152-159, 348169-172, 382/103, 382/154
• US Class Current: 348/159
• CPC Class Codes: G03B 41/00 Special techniques not covered by groups G03B31/00 - G03B39/00; Apparatus therefor
• CPC Class Codes: H04N 23/67 Focus control based on electronic image sensor signals
• CPC Class Codes: H04N 23/695 Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects
• CPC Class Codes: H04N 5/2627 for providing spin image effect, 3D stop motion effect or temporal freeze effect 2D image animation in general G06T13/80
• CPC Class Codes: H04N 5/2628 Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation

Abstract

A method and a system of generating an image sequence of an object within a scene. According to one embodiment, the method includes capturing an image (images I_1-N) of the object with a plurality of camera systems, wherein the camera systems are positioned around the scene. Next, the method includes 2D projective transforming certain of the images (I_2-N) such that a point of interest in each of the images is at a same position as a point of interest in a first image (I₁) from one of the camera systems. The method further includes outputting the transformed images (I_2′-N′) and the first image (I₁) in a sequence corresponding to a positioning of the corresponding camera systems around the scene.

Claims

1. A method of generating an image sequence of an object within a scene, comprising:

capturing an image of the object with a plurality of networked variable pointing camera systems, wherein the camera systems are positioned around the scene, and wherein pointing and optical parameters from a first of the camera systems and mapping data for the camera systems are used to compute pointing and optical parameters for the remainder of the camera systems such that, at a point in time, the camera systems are aimed at the object and a size of the object in images from each of the camera systems is substantially the same;

2D projective transforming certain of the images captured by the camera systems such that a point of interest in each of the images is at a same position as a point of interest in a first image from one of the camera systems;

outputting the 2D projective transformed images and the first image in a sequence corresponding to a positioning of the corresponding camera systems around the scene; and

determining the mapping data for the camera systems by;

determining the pose of each camera system with respect to the scene;

determining the relationship of a zoom control parameter to the angular field of view for each camera system; and

determining the relationship of a focus control parameter to the distance to objects for each camera system.

2. The method of claim 1, wherein capturing the image of the object with the plurality of camera systems includes capturing the image of the object at the same instant in time with the plurality of camera systems.

3. The method of claim 2, further comprising receiving an input corresponding to the particular point in time.

4. The method of claim 1, further comprising, after capturing the images, receiving an input corresponding to the point of interest from a user via a user interface.

5. The method of claim 1, wherein capturing the images includes capturing an image with a plurality of pan/tilt camera systems positioned around the scene.

6. A system for generating an image sequence of an object within a scene, comprising:

a plurality of networked variable pointing camera systems positioned around the scene;

means for determining, based on pointing and optical parameters from a first of the variable pointing camera system and mapping data for the camera systems, pointing and optical parameters for the remainder of the variable pointing camera systems such that, at a point in time, the camera systems are aimed at the object and a size of the object in images from the camera systems is substantially the same;

means for controlling the remainder of the variable pointing camera systems based on the determined pointing and optical parameters for the remainder of the camera systems;

storage means for storing images captured by the plurality of camera systems; and

a frame-sequencing module in communication with the storage means for 2D projective transforming stored images such that a point of interest in images from each of the plurality of camera systems is at the same position, wherein the mapping data includes;

data regarding the pose of each camera system with respect to the scene;

data regarding the relationship of a zoom control parameter to the angular field view for each camera system; and

data regarding the relationship of a focus control parameter to the distance to objects for each camera system.

7. The system of claim 6, wherein the frame-sequencing module is further for outputting images from the plurality of camera systems in sequence corresponding to the positioning of the camera systems around the scene.

8. The system of claim 7, wherein the images from the camera systems are from the same point in time.

9. The system of claim 8, further comprising an interface module in communication with the frame-sequencing module for allowing an operator to specify the point in time.

10. The system of claim 7, further comprising an interface module for allowing the operator to specify the point of interest.

11. The system of claim 7, wherein the frame-sequencing module is for outputting the images from the plurality of camera systems in a clockwise sequence.

12. A system for generating an image sequence of an object within a scene, comprising:

a plurality of networked variable pointing camera systems positioned around the scene;

means, based on pointing and optical parameters from a first of the variable pointing camera system and mapping data for the camera system, for determining pointing and optical parameters for the remainder of the variable pointing camera systems such that, at a point in time, the camera systems are aimed at the object and a size of the object in images from the camera systems is substantially the same;

means for controlling the remainder of the variable pointing camera systems based on the determined pointing and optical parameters for the remainder of the camera systems;

storage means for storing images captured by the plurality of camera systems; and

means in communication with the storage means, for 2D projective transforming the images from the plurality of camera systems such that a point of interest in images from each of the plurality of camera systems is at the same position, wherein the mapping data includes;

data regarding the pose of each camera system with respect to the scene;

data regarding the relationship of a zoom control parameter to the angular field view for each camera system; and

data regarding the relationship of a focus control parameter to the distance to objects for each camera system.

13. The system of claim 12, further comprising means for outputting the images from the plurality of camera systems in sequence corresponding to the positioning of the camera systems around the scene.

14. The system of claim 13, wherein the images from the camera systems are from the same point in time.

15. The system of claim 14, further comprising means for allowing an operator to specify the point in time.

16. The system of claim 13, further comprising means for allowing the operator to specify the point of interest.

17. A system, comprising:

a plurality of pan/tilt camera systems positioned around a scene;

means for controlling the camera systems based on computed pointing and optical parameters for certain of the camera systems, wherein the pointing and optical parameters are computed based on pointing and optical parameters from at least one of the camera systems and based on mapping data for the camera systems, such that each camera system is aimed at a target within the scene and such that a size of the target in an image from each camera system is substantially the same at a particular time;

a video storage unit in communication with each pan/tilt camera system for storing images captured by each camera system; and

a frame-sequencing module in communication with the video storage unit, wherein the frame-sequencing module is for 2D projective transforming images from certain of the camera systems such that a point of interest in an untransformed image from one of the camera systems is at a same position as a point of interest in each of the 2D projective transformed images, wherein the mapping data includes;

data regarding the pose of each camera system with respect to the scene;

data regarding the relationship of a zoom control parameter to the angular field view for each camera system; and

data regarding the relationship of a focus control parameter to the distance to objects for each camera system.

18. The system of claim 17, wherein the frame-sequencing module is further for outputting the 2D projective transformed images and the untransformed image in sequence corresponding to the positioning of the corresponding camera systems around the scene.

19. The system of claim 17, further comprising an interface module for allowing an operator to specify the point of interest in the untransformed image and in the 2D projective transformed images.