3-D instant replay system and method

• US 20020190991A1
• Filed: 05/16/2002
• Published: 12/19/2002
• Est. Priority Date: 05/16/2001
• Status: Abandoned Application

• Current Assignee: Zaxel Systems, Inc.
• Original Assignee: Zaxel Systems, Inc.
• Inventors: Ollis, Mark, Williamson, Todd, Efran, Daniel
• Application Number:
  US10/150,533
• Publication Number:
  US 20020190991A1
• Time in Patent Office: Days
• US Class Current: 345/475
• CPC Class Codes: H04N 5/2224 related to virtual studio applications
• CPC Class Codes: H04N 5/262 Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
• CPC Class Codes: H04N 5/2625 for obtaining an image which is composed of images from a temporal image sequence, e.g. for a stroboscopic effect sequence generated by event triggered capturing H04N7/188
• CPC Class Codes: H04N 5/2627 for providing spin image effect, 3D stop motion effect or temporal freeze effect 2D image animation in general G06T13/80

Abstract

A 3-D Instant Replay system is disclosed for the capture, adjustment, and generation substantially in real time of complex 3-D or other video effects. A camera array is configured about a desired scene for effect generation. Image information from multiple cameras is sent instantaneously and simultaneously to a capture system. Multiple capture devices may be used together in the capture system to capture video information from a large number of cameras. Once inside each capture device, image data is made available in the memory element of the capture system for generation of realtime effects. A host system, connected via high speed networking elements to the capture system, selects relevant portions of available image data from the capture system (or each capture device) based on preset criteria or user input. An effect generation algorithm, optionally including image correction and adjustment processes, on the host system creates the desired video effect and outputs generated image frames in a desired format for viewing.

Claims

1. A video effect generation system, comprising:

an imaging array comprising a first imaging device and a second imaging device;

an image capture system configured to capture a first set of image information from said first imaging device and a second set of image information from said second imaging device;

an image processing system for selecting a first subset from said first set of image information and a second subset from said second set of image information to produce a generated video effect sequence.

2. The video effect generation system as in claim 1, wherein said imaging array conforms to a generally smooth trajectory path.

3. The video effect generation system as in claim 2, wherein said smooth trajectory path corresponds to a virtual trajectory which would be followed by a single virtual camera to produce a video sequence corresponding to said generated video effect sequence.

4. The video effect generation system as in claim 3, wherein said first imaging device and said second imaging device are disposed at different points along said smooth trajectory path.

5. The video effect generation system as in claim 4, wherein said first imaging device and said second imaging device are oriented toward a common scene, said first imaging device depicting a first viewpoint of said common scene, and said second imaging device depicting a second viewpoint of said common scene.

6. The video effect generation system as in claim 5, wherein said first viewpoint comprises a first field of view and said second viewpoint comprises a second field of view, said first field of view corresponding to the field of view of said virtual camera on said virtual trajectory at substantially the same spatial location and orientation as said first imaging device, and said second field of view corresponding to the field of view of said virtual camera on said virtual trajectory at substantially the same spatial location and orientation as said second imaging device.

7. The video effect generation system as in claim 1, wherein the said imaging array is provided as a calibrated array such that extrinsic and intrinsic parameters of each said imaging device are generated.

8. The video effect generation system as in claim 1, further comprising a synchronization means such that at time Ti, image frame 11 from said first set of image information and image frame 12 from said second set of image information are captured in said image capture system substantially simultaneously.

9. The video effect generation system as in claim 1, wherein said image capture system comprises a first image capture device and a second image capture device, said first image capture device being coupled to said first imaging device and said second image capture device being coupled to said second imaging device.

10. The video effect generation system as in claim 9, wherein the imaging array includes additional imaging devices forming a plurality of imaging devices, each additional imaging device disposed along said trajectory path and oriented toward said common scene such that additional viewpoints, each said additional viewpoint including an additional field of view corresponding to the field of view of said virtual camera on said virtual trajectory at substantially the same spatial location and orientation as each said additional imaging device.

11. The video effect generation system as in claim 10, wherein said first image capture device is coupled to a first set of imaging devices from said plurality of imaging devices, and said second image capture device is coupled to a second set of imaging devices from said plurality of imaging devices.

12. The video effect generation system as in claim 1, further comprising an image adjustment means.

13. The video effect generation system as in claim 12, wherein said image adjustment means comprises a vibration calibration routine, a color correction process, and a perspective warping process.

14. The video effect generation system as in claim 1, further comprising a user interface for interacting with one or more system elements such that said generated video effect sequence corresponds to a desired set of parameters set by a user.

16. A method as in claim 15, wherein said positioning step further comprises the steps of:

calibrating said first imaging device and said second imaging device;

providing calibration data in said processing system

17. The method as in claim 15, wherein said capturing step comprises the steps of:

synchronizing said first imaging device and said second imaging device;

triggering a capture mode in said image capture system;

capturing image information in said image capture system.

18. The method as in claim 15, wherein said selecting step further comprises the steps of:

providing video effect sequence parameters;

determining a first relevant set of image information from said first set of image

information based on said parameters;

determining a second relevant set of image information from said second set of image information based on said second parameters;

providing said relevant sets of image information to said image processing system.

19. The method as in claim 18, wherein said first set of relevant image information comprises a first image frame and said second set of relevant image information comprises a second image frame.

20. The method as in claim 19, wherein said step of generating a video sequence further comprises the steps of:

adjusting said first image frame and said second image frame;

sequentially ordering said first image frame and said second image frame in a desired order;

creating a video effect sequence.

21. The method as in claim 15, wherein said step of generating a video sequence is performed substantially in real time, such that said video effect sequence may be played back to a viewer immediately after said selecting step.

15. A method of generating video effects comprising the steps of:

positioning a first imaging device and a second imaging device in an array;

capturing a first set of image information from said first imaging device and a second set of image information from said second image device in an image capture system coupled to said array;

selecting a first subset of image information from said first set and a second subset of image information from said second set in an image processing system coupled to said capture system;

generating a video effect sequence from said first subset of image information and said second subset of image information in said image processing system.