REAL-TIME SYNTHETICALLY GENERATED VIDEO FROM STILL FRAMES
0 Assignments
0 Petitions
Accused Products
Abstract
Systems and methods for generating synthetic video are disclosed. For example, a system may include a memory unit and a processor configured to execute the instructions to perform operations. The operations may include receiving video data, normalizing image frames, generating difference images, and generating an image sequence generator model. The operations may include training an autoencoder model using difference images, the autoencoder comprising an encoder model and a decoder model. The operations may include identifying a seed image frame and generating a seed difference image from the seed image frame. The operations may include generating, by the image sequence generator model, synthetic difference images based on the seed difference image. In some aspects, the operations may include using the decoder model to synthetic normalized image frames from the synthetic difference images. The operations may include generating synthetic video by adding background to the synthetic normalized image frames.
-
Citations
40 Claims
-
1-20. -20. (canceled)
-
21. A system for generating synthetic video, the system comprising:
-
one or more memory units for storing instructions; and one or more processors configured to execute the instructions to perform operations comprising; training, using a sequence of difference images, an image sequence generator model to generate synthetic difference images; generating a sequence of synthetic difference images based on a seed difference image, wherein the sequence of synthetic difference images is generated by iteratively using the image sequence generator model to accept a previous synthetic difference image as an input and return a subsequent synthetic difference image as an output, starting from the seed difference image; and generating a sequence of synthetic images based on the sequence of synthetic difference images, the generating comprising implementing a decoder model to perform at least one of a forward-step decoding or backward-step decoding. - View Dependent Claims (22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38)
-
-
39. A method for generating synthetic video comprising:
-
training, using a sequence of difference images, an image sequence generator model to generate synthetic difference images; generating a sequence of synthetic difference images based on a seed difference image, wherein the sequence of synthetic difference images is generated by iteratively using the image sequence generator model to accept a previous synthetic difference image as an input and return a subsequent synthetic difference image as an output, starting from the seed difference image; and generating a sequence of synthetic images based on the sequence of synthetic difference images, the generating comprising implementing a decoder model to perform at least one of a forward-step decoding or backward-step decoding.
-
-
40. A system for generating synthetic video, the system comprising:
-
one or more memory units for storing instructions; and one or more processors configured to execute the instructions to perform operations comprising; training, using a sequence of difference images, an image sequence generator model to generate synthetic difference images; training an autoencoder model using the sequence of difference images and a sequence of images, the autoencoder model comprising an encoder model and the decoder model, wherein; a difference image in the sequence of difference images corresponds to a preceding image in a sequence of images and a subsequent image in the sequence of images; and training the autoencoder model comprises least one of; training the encoder model to generate to the difference image from the preceding image and training the decoder model to generate the subsequent image from the difference image;
ortraining the encoder model to generate the difference image from the subsequent image and training the decoder model to generate the preceding image from the difference image. generating a sequence of synthetic difference images based on a seed difference image, wherein the sequence of synthetic difference images is generated by iteratively using the image sequence generator model to accept a previous synthetic difference image as an input and return a subsequent synthetic difference image as an output, starting from the seed difference image; and generating a sequence of synthetic images based on the sequence of synthetic difference images, the generating comprising implementing a decoder model to perform at least one of a forward-step decoding or backward-step decoding; and generating synthetic video by adding background to individual ones of the sequence of synthetic images.
-
Specification