Virtual video projection system to synch animation sequences
First Claim
1. A method of synchronizing an animation sequence with a video footage, the method comprising:
- obtaining a digital three-dimensional (3D) model of a site captured in the video footage by a video camera installed in the site, the video camera having a set of camera parameters;
identifying a plurality of key frames of the video footage, wherein a moving object is at a respective position in a respective key frame of the plurality of key frames;
placing a first virtual camera in the digital 3D model at a first location and a first orientation corresponding to a location and an orientation of the video camera in the site, the first virtual camera having a first set of virtual camera parameters, at least some virtual camera parameters of the first set of virtual camera parameters are same as some corresponding camera parameters of the set of camera parameters of the video camera;
generating a first set of virtual frames of the animation sequence by projecting the digital 3D model onto a first scene frame from a viewpoint of the first virtual camera using the first set of virtual camera parameters, the first set of virtual frames including a first plurality of virtual key frames, each virtual key frame corresponding to a respective key frame in the video footage;
rendering at least a subset of a set of pixels of the video footage in each virtual frame of the first set of virtual frames;
for each respective virtual key frame, placing a virtual object corresponding to the moving object at a respective location in the digital 3D model that matches with the respective position of the moving object in a corresponding key frame; and
playing the first set of virtual frames of the animation sequence while the video footage is overlaid on the subset of the set of pixels, wherein the animation sequence and the video footage are played simultaneously by stepping through time such that the first plurality of virtual key frames is in synch with the plurality of key frames of the video footage.
1 Assignment
0 Petitions
Accused Products
Abstract
A method of synchronizing an animation sequence with a video includes placing a virtual camera in a 3D model of a site captured in the video at the same location and the same orientation as those of a video camera that captured the video, generating a set of virtual frames of the animation sequence by projecting the 3D model onto a scene frame from a viewpoint of the virtual camera, for each of a plurality of virtual key frames, placing a virtual object corresponding to a moving object captured in the video in the 3D model at a respective location that matches with the respective position of the moving object in a corresponding key frame of the video, and playing the set of virtual frames by stepping through time such that the plurality of virtual key frames is in synch with the plurality of key frames of the video.
13 Citations
18 Claims
-
1. A method of synchronizing an animation sequence with a video footage, the method comprising:
-
obtaining a digital three-dimensional (3D) model of a site captured in the video footage by a video camera installed in the site, the video camera having a set of camera parameters; identifying a plurality of key frames of the video footage, wherein a moving object is at a respective position in a respective key frame of the plurality of key frames; placing a first virtual camera in the digital 3D model at a first location and a first orientation corresponding to a location and an orientation of the video camera in the site, the first virtual camera having a first set of virtual camera parameters, at least some virtual camera parameters of the first set of virtual camera parameters are same as some corresponding camera parameters of the set of camera parameters of the video camera; generating a first set of virtual frames of the animation sequence by projecting the digital 3D model onto a first scene frame from a viewpoint of the first virtual camera using the first set of virtual camera parameters, the first set of virtual frames including a first plurality of virtual key frames, each virtual key frame corresponding to a respective key frame in the video footage; rendering at least a subset of a set of pixels of the video footage in each virtual frame of the first set of virtual frames; for each respective virtual key frame, placing a virtual object corresponding to the moving object at a respective location in the digital 3D model that matches with the respective position of the moving object in a corresponding key frame; and playing the first set of virtual frames of the animation sequence while the video footage is overlaid on the subset of the set of pixels, wherein the animation sequence and the video footage are played simultaneously by stepping through time such that the first plurality of virtual key frames is in synch with the plurality of key frames of the video footage. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9)
-
-
10. A system for synchronizing an animation sequence with a video footage, the system comprising:
-
a memory for storing a digital three-dimensional (3D) model of a site captured in the video footage by a video camera installed in the site, the video footage including a plurality of key frames, wherein a moving object is at a respective position in a respective key frame of the plurality of key frames, and the video camera having a set of camera parameters; a projection module configured to place a first virtual camera in the digital 3D model at a first location and a first orientation corresponding to a location and an orientation of the video camera in the site, the first virtual camera having a first set of virtual camera parameters, at least some virtual camera parameters of the first set of virtual camera parameters are same as some corresponding camera parameters of the set of camera parameters of the video camera; a processor configured to generate a first set of virtual frames of the animation sequence by projecting the digital 3D model onto a first scene frame from a viewpoint of the first virtual camera using the first set of virtual camera parameters, the first set of virtual frames including a first plurality of virtual key frames, each virtual key frame corresponding to a respective key frame in the video footage; a graphics shader configured to render at least a subset of a set of pixels of the video footage in each virtual frame of the first set of virtual frames; an interpolator configured to, for each respective virtual key frame, place a virtual object corresponding to the moving object at a respective location in the digital 3D model that matches with the respective position of the moving object in a corresponding key frame; and a key-frame timer for synchronizing playing of the first set of virtual frames of the animation sequence while the video footage is overlaid on the subset of the set of pixels such that the first plurality of virtual key frames is in synch with the plurality of key frames of the video footage. - View Dependent Claims (11, 12, 13, 14, 15, 16, 17, 18)
-
Specification