System and method for manipulating the point of interest in a sequence of images

• US 20020145660A1
• Filed: 02/12/2002
• Published: 10/10/2002
• Est. Priority Date: 02/12/2001
• Status: Active Grant

• Current Assignee: Carnegie Mellon University
• Original Assignee: Carnegie Mellon University
• Inventors: Kanade, Takeo, Collins, Robert
• Granted Patent: US 7,106,361 B2
• Time in Patent Office: Days
• US Class Current: 348/36
• CPC Class Codes: G03B 41/00 Special techniques not covered by groups G03B31/00 - G03B39/00; Apparatus therefor
• CPC Class Codes: H04N 23/67 Focus control based on electronic image sensor signals
• CPC Class Codes: H04N 23/695 Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects
• CPC Class Codes: H04N 5/2627 for providing spin image effect, 3D stop motion effect or temporal freeze effect 2D image animation in general G06T13/80
• CPC Class Codes: H04N 5/2628 Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation

Abstract

A method and a system of generating a video image sequence. According to one embodiment, the method includes positioning a plurality of camera systems relative to a scene such that the camera systems define a gross trajectory. The method further includes transforming images from the camera systems to superimpose a secondary induced motion on the gross trajectory. And the method includes displaying the transformed images in sequence corresponding to the position of the corresponding camera systems along the gross trajectory.

Claims

1. A method of generating a video image sequence, comprising:

positioning a plurality of camera systems relative to a scene such that the camera systems define a gross trajectory;

transforming images from the camera systems to superimpose a secondary induced motion on the gross trajectory; and

displaying the transformed images in sequence corresponding to the position of the corresponding camera systems along the gross trajectory.

2. The method of claim 1, wherein positioning the plurality of camera systems includes positioning a plurality of pan/tilt camera systems relative to the scene.

3. The method of claim 1, wherein positioning the plurality of camera systems includes positing a plurality of static camera systems relative to the scene.

4. The method of claim 1, wherein positioning the plurality of camera systems includes positioning at least one pan/tilt camera system and one static camera system relative to the scene.

5. The method of claim 1, wherein transforming images from the camera systems includes applying a 2D image transformation to the images .

6. The method of claim 5, wherein applying the 2D image transformation includes applying a 2D image transformation according to a homography defined by a one point correspondence between the images.

7. The method of claim 6, wherein applying the 2D image transformation according to a homography defined by a one point correspondence between the images includes mapping a point of interest in each image to the center of the image.

8. The method of claim 5, wherein applying the 2D image transformation includes applying a 2D image transformation according to a homography defined by a two point correspondence between the images.

9. The method of claim 8, wherein applying the 2D image transformation according to a homography defined by a two point correspondence between the images includes:

mapping a point of interest in each image to the center of the image; and

mapping a vertical unit point in each image to a point at a predetermined vertical relationship to the center of the image.

10. The method of claim 5, wherein applying the 2D image transformation includes applying a 2D image transformation according to a homography defined by a three point correspondence between the images.

11. The method of claim 10, wherein applying the 2D image transformation according to a homography defined by a three point correspondence between the images includes:

mapping a translation point in each image to the center of the image;

mapping a point of interest in each image to the translation point in each image; and

mapping a vertical unit point in each image to a point at a predetermined vertical relationship to the translation point.

12. The method of claim 1, further comprising:

generating an image corresponding to an image from a virtual camera system positioned along the gross trajectory between first and second camera systems of the plurality of camera systems; and

displaying the image between display of the transformed image from the first camera system and display of the transformed image from the second camera system.

13. The method of claim 12, wherein generating the image corresponding to an image from a virtual camera system positioned along the gross trajectory between first and second camera systems of the plurality of camera systems includes generating an image corresponding to an image from a virtual camera system having a rotation and translation interpolated from a rotation and translation of the first and second camera systems.

14. The method of claim 1, wherein positioning the plurality of camera systems relative to the scene includes positing the plurality of camera systems in a close-ended configuration relative to the scene.

15. The method of claim 1, wherein positioning the plurality of camera systems relative to the scene includes positing the plurality of camera systems in an array configuration.

17. The system of claim 16, further comprising means for controlling the plurality of camera systems such that the camera systems are simultaneously aimed at a target within the scene and a size of the target in the images from the camera systems is substantially the same over time.

18. The system of claim 17, further comprising means for outputting the transformed images in sequence corresponding to the position of the corresponding camera systems along the gross trajectory.

19. The system of claim 18, wherein the means for transforming images from the camera systems includes means for applying a 2D image transformation to the images.

20. The system of claim 19, wherein the means for applying the 2D image transformation includes means for applying a 2D image transformation according to a homography defined by a one-point correspondence between the images.

21. The system of claim 20, wherein the means for applying the 2D image transformation according to a homography defined by a one point correspondence between the images includes means for mapping a point of interest in each image to the center of the image.

22. The system of claim 21, wherein the point of interest is not a point of the target.

23. The system of claim 21, wherein the point of interest is a point of the target.

24. The system of claim 19, wherein the means for applying the 2D image transformation includes applying a 2D image transformation according to a homography defined by a two-point correspondence between the images.

25. The system of claim 24, wherein the means for applying the 2D image transformation according to a homography defined by a two-point correspondence between the images includes:

means for mapping a point of interest in each image to the center of the image; and

means for mapping a vertical unit point in each image to a point at a predetermined vertical relationship to the center of the image.

26. The system of claim 19, wherein the means for applying the 2D image transformation includes means for applying a 2D image transformation according to a homography defined by a three-point correspondence between the images.

27. The system of claim 26, wherein the means for applying the 2D image transformation according to a nomography defined by a three-point correspondence between the images includes:

means for mapping a translation point in each image to the center of the image;

means for mapping a point of interest in each image to the translation point in each image; and

means for mapping a vertical unit point in each image to a point at a predetermined vertical relationship to the translation point.

28. The system of claim 16, further comprising:

means for generating an image corresponding to an image from a virtual camera system positioned along the gross trajectory between first and second camera systems of the plurality of camera systems; and

means for outputting the image between display of the transformed image from the first camera system and display of the transformed image from the second camera system.

29. The system of claim 28, wherein the means for generating the image corresponding to an image from a virtual camera system positioned along the gross trajectory between first and second camera systems of the plurality of camera systems includes means for generating an image corresponding to an image from a virtual camera system having a rotation and translation interpolated from a rotation and translation of the first and second camera systems.

52. The computer readable medium of claim 1, having further stored thereon instructions which, when executed by the processor, cause the processor to:

generate an image corresponding to an image from a virtual camera system positioned along the gross trajectory between first and second camera systems of the plurality of camera systems; and

display the image between display of the transformed image from the first camera system and display of the transformed image from the second camera system.

53. The computer readable medium of claim 52, having further stored thereon instructions which, when executed by the processor, cause the processor to generate the image corresponding to an image from a virtual camera system positioned along the gross trajectory between first and second camera systems of the plurality of camera systems by generating an image corresponding to an image from a virtual camera system having a rotation and translation interpolated from a rotation and translation of the first and second camera systems.

16. A system for generating a video image sequence of an object within a scene, comprising:

means for capturing an image from a plurality of camera systems positioned relative to the scene such that the camera systems define a gross trajectory; and

means for transforming images from the camera systems to superimpose a secondary induced motion on the gross trajectory.

30. A system for generating a video image sequence of an object within a scene, comprising:

a plurality of camera systems positioned relative to the scene such that the camera systems define a gross trajectory;

a video storage unit in communication with the camera systems; and

a frame-sequencing module in communication with the video storage unit for transforming images of the camera systems retrieved from the video storage unit to superimpose a secondary induced motion on the gross trajectory.

31. The system of claim 30, further comprising means for controlling the plurality of camera systems such that the camera systems are simultaneously aimed a target within the scene and a size of the target in the images from the camera systems is substantially the same over time.

32. The system of claim 31, wherein the frame-sequencing module is further for outputting the transformed images in sequence corresponding to the position of the corresponding camera systems along the gross trajectory.

33. The system of claim 32, wherein the frame-sequencing module is further for:

generating an image corresponding to an image from a virtual camera system positioned along the gross trajectory between first and second camera systems of the plurality of camera systems; and

outputting the image between display of the transformed image from the first camera system and display of the transformed image from the second camera system.

34. The system of claim 33, wherein the frame-sequencing module is for generating the image corresponding to an image from a virtual camera system positioned along the gross trajectory between first and second camera systems of the plurality of camera systems by generating an image corresponding to an image from a virtual camera system having a rotation and translation interpolated from a rotation and translation of the first and second camera systems.

35. The system of claim 30, wherein the frame-sequencing module is for transforming images from the camera systems by applying a 2D image transformation to the images.

36. The system of claim 35, wherein the 2D image transformation includes a 2D image transformation according to a nomography defined by a one-point correspondence between the images.

37. The system of claim 36, wherein frame-sequencing module is for applying the 2D image transformation according to a homography defined by a one point correspondence between the images includes by mapping a point of interest in each image to the center of the image.

38. The system of claim 37, wherein the point of interest is not a point of the target.

39. The system of claim 37, wherein the point of interest is a point of the target.

40. The system of claim 35, wherein the 2D image transformation includes a 2D image transformation according to a homography defined by a two-point correspondence between the images.

41. The system of claim 40, wherein the frame-sequencing module is for applying the 2D image transformation according to a homography defined by a two-point correspondence between the images by:

mapping a point of interest in each image to the center of the image; and

mapping a vertical unit point in each image to a point at a predetermined vertical relationship to the center of the image.

42. The system of claim 30, wherein the 2D image transformation includes a 2D image transformation according to a homography defined by a three-point correspondence between the images.

43. The system of claim 42, wherein the frame-sequencing module is for applying the 2D image transformation according to a homography defined by a three-point correspondence between the images by:

mapping a translation point in each image to the center of the image;

mapping a point of interest in each image to the translation point in each image; and

mapping a vertical unit point in each image to a point at a predetermined vertical relationship to the translation point.

44. A computer readable medium, having stored thereon instructions which, when executed by a processor, cause the processor to:

transform images from a plurality of camera systems positioned relative to a scene to define a gross trajectory to superimpose a secondary induced motion on the gross trajectory; and

output the transformed images in sequence corresponding to the position of the corresponding camera systems along the gross trajectory.

45. The computer readable medium of claim 44, having further stored thereon instructions which, when executed by the processor, cause the processor to transform the images from the camera systems by applying a 2D image transformation to the images.

46. The computer readable medium of claim 45, having further stored thereon instructions which, when executed by the processor, cause the processor to apply the 2D image transformation by applying a 2D image transformation according to a homography defined by a one point correspondence between the images.

47. The computer readable medium of claim 46, having further stored thereon instructions which, when executed by the processor, cause the processor to apply the 2D image transformation according to a nomography defined by a one point correspondence between the images by mapping a point of interest in each image to the center of the image.

48. The computer readable medium of claim 45, having further stored thereon instructions which, when executed by the processor, cause the processor to apply the 2D image transformation by applying a 2D image transformation according to a homography defined by a two point correspondence between the images.

49. The computer readable medium of claim 48, having further stored thereon instructions which, when executed by the processor, cause the processor to apply the 2D image transformation according to a homography defined by a two point correspondence between the images by mapping a point o f interest in each image to the center of the image;

and

mapping a vertical unit point in each image to a point at a predetermined vertical relationship to the center of the image.

50. The computer readable medium of claim 45, having further stored thereon instructions which, when executed by the processor, cause the processor to apply the 2D image transformation includes applying a 2D image transformation according to a homography defined by a three point correspondence between the images.

51. The computer readable medium of claim 50, having further stored thereon instructions which, when executed by the processor, cause the processor to apply the 2D image transformation according to a homography defined by a three point correspondence between the images by:

mapping a translation point in each image to the center of the image;

mapping a point of interest in each image to the translation point in each image; and

mapping a vertical unit point in each image to a point at a predetermined vertical relationship to the translation point.