Artistic style transfer for videos
First Claim
1. An image processing method, comprising:
- obtaining a first source image;
extracting an artistic style from the first source image, wherein the artistic style is stored as a plurality of layers in a neural network, wherein each layer of the plurality of layers is configured to process a target image and produce a filtered version of the target image, and wherein a degree to which a given layer of the plurality of layers constrains pixel values from the target image during the processing of the target image is based on how high the given layer is in the neural network;
obtaining a first target image captured at a first time;
applying the artistic style to the first target image to create a stylized version of the first target image;
obtaining a second target image captured at a second time;
applying the artistic style to the second target image to create a stylized version of the second target image;
combining the stylized version of the first target image and the stylized version of the second target image to create a first stylized video sequence; and
storing the first stylized video sequence in a memory.
1 Assignment
0 Petitions
Accused Products
Abstract
Techniques are disclosed herein for applying an artistic style extracted from one or more source images, e.g., paintings, to one or more target images. The extracted artistic style may then be stored as a plurality of layers in a neural network. In some embodiments, two or more stylized target images may be combined and stored as a stylized video sequence. The artistic style may be applied to the target images in the stylized video sequence using various optimization methods and/or pixel- and feature-based regularization techniques in a way that prevents excessive content pixel fluctuations between images and preserves smoothness in the assembled stylized video sequence. In other embodiments, a user may be able to semantically annotate locations of undesired artifacts in a target image, as well as portion(s) of a source image from which a style may be extracted and used to replace the undesired artifacts in the target image.
-
Citations
20 Claims
-
1. An image processing method, comprising:
-
obtaining a first source image; extracting an artistic style from the first source image, wherein the artistic style is stored as a plurality of layers in a neural network, wherein each layer of the plurality of layers is configured to process a target image and produce a filtered version of the target image, and wherein a degree to which a given layer of the plurality of layers constrains pixel values from the target image during the processing of the target image is based on how high the given layer is in the neural network; obtaining a first target image captured at a first time; applying the artistic style to the first target image to create a stylized version of the first target image; obtaining a second target image captured at a second time; applying the artistic style to the second target image to create a stylized version of the second target image; combining the stylized version of the first target image and the stylized version of the second target image to create a first stylized video sequence; and storing the first stylized video sequence in a memory. - View Dependent Claims (2, 3, 4, 5, 6, 7)
-
-
8. A non-transitory program storage device comprising instructions stored thereon to cause one or more processors to:
-
obtain a first source image; extract an artistic style from the first source image, wherein the artistic style is stored as a plurality of layers in a neural network, wherein each layer of the plurality of layers is configured to process a target image and produce a filtered version of the target image, and wherein a degree to which a given layer of the plurality of layers constrains pixel values from the target image during the processing of the target image is based on how high the given layer is in the neural network; obtain a first target image captured at a first time; apply the artistic style to the first target image to create a stylized version of the first target image; obtain a second target image captured at a second time; apply the artistic style to the second target image to create a stylized version of the second target image; combine the stylized version of the first target image and the stylized version of the second target image to create a first stylized video sequence; and store the first stylized video sequence in a memory. - View Dependent Claims (9, 10, 11, 12, 13)
-
-
14. A device, comprising:
-
a memory; and one or more processors operatively coupled to the memory, wherein the one or more processors are configured to execute instructions to cause the one or more processors to; obtain a first source image; extract an artistic style from the first source image, wherein the artistic style is stored as a plurality of layers in a neural network, wherein each layer of the plurality of layers is configured to process a target image and produce a filtered version of the target image, and wherein a degree to which a given layer of the plurality of layers constrains pixel values from the target image during the processing of the target image is based on how high the given layer is in the neural network; obtain a first target image captured at a first time; apply the artistic style to the first target image to create a stylized version of the first target image; obtain a second target image captured at a second time; apply the artistic style to the second target image to create a stylized version of the second target image; combine the stylized version of the first target image and the stylized version of the second target image to create a first stylized video sequence; and store the first stylized video sequence in the memory. - View Dependent Claims (15, 16, 17, 18, 19, 20)
-
Specification