Compression and editing of movies by multi-image morphing
First Claim
Patent Images
1. A method for representing a movie having an original frame sequence, comprising:
- generating a pixel level frame to frame correspondence;
representing the original frame sequence as a multi-image morph, wherein representing the original frame sequence as a multi-image morph comprises representing each frame in the sequence as a point (a1, a2, . . . , an) in an n-dimensional affine morph space defined by a set of warping mappings Wi and associated color definitions fi (1≦
i≦
n) , where each ai is a number and each Wi and fi is an operator on a planar image space, the coefficients ai satisfying the constraint a1+a2+ . . . +an=1;
deriving from the multi-image morph a morph basis of feature vectors describing principal changes among frames; and
representing the movie in terms of the morph basis.
1 Assignment
0 Petitions
Accused Products
Abstract
A computer-implemented method for representing a movie having a sequence of frames includes generating a pixel level frame to frame correspondence; representing the original frame sequence as a multi-image morph; and deriving from the multi-image morph a basis of feature vectors describing principal changes among frames and representing the movie in terms of the basis.
-
Citations
27 Claims
-
1. A method for representing a movie having an original frame sequence, comprising:
-
generating a pixel level frame to frame correspondence;
representing the original frame sequence as a multi-image morph, wherein representing the original frame sequence as a multi-image morph comprises representing each frame in the sequence as a point (a1, a2, . . . , an) in an n-dimensional affine morph space defined by a set of warping mappings Wi and associated color definitions fi (1≦
i≦
n) , where each ai is a number and each Wi and fi is an operator on a planar image space, the coefficients ai satisfying the constraint a1+a2+ . . . +an=1;
deriving from the multi-image morph a morph basis of feature vectors describing principal changes among frames; and
representing the movie in terms of the morph basis. - View Dependent Claims (5, 6, 7, 8, 9, 10, 11, 12, 13, 15)
selecting a compression level; and
using the compression level to limit the number of feature vectors in the morph basis.
-
-
15. The method of claim 1, further comprising representing the original frame sequence as a parametric curve fit to the points (a1, a2, . . . , an) in the morph space.
-
2. A computer program stored on a computer-readable medium for compressing a movie having an original frame sequence, the program comprising instructions to:
-
generate a pixel level frame to frame correspondence;
represent the original frame sequence as a multi-image morph, wherein instructions to represent the original frame sequence as a multi-image morph comprise instructions to represent each frame in the sequence as a point (a1, a2, . . . , an) in an n-dimensional affine morph space defined by a set of warping mappings Wi and associated color definitions fi (1≦
i≦
n), where each ai is a number and each Wi and fi is an operator on a planar image space, the coefficients ai satisfying the constraint a1+a2+ . . . +an=1;
derive from the multi-image morph a morph basis of feature vectors describing principal changes among frames; and
represent the movie in terms of the morph basis. - View Dependent Claims (14, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27)
constructing a set of new basis frames from the basis of feature vectors, each new basis frame corresponding to one of the feature vectors in the basis of feature vectors, wherein the new basis frames are not in the original frame sequence.
-
-
16. The program of claims 4 or 2, wherein the correspondence generating instruction processes the frames using an optical flow method.
-
17. The program of claims 4 or 2, wherein the deriving instruction uses a principal components method.
-
18. The program of claim 17, further comprising instructions for compensating for vibrations of movie frames represented in terms of the morph basis by discarding one or more high weight principal components.
-
19. The program of claim 2, wherein the movie has a first frame rate, further comprising instructions for resampling the movie represented in terms of the morph basis at a second frame rate.
-
20. The program of claim 19, wherein the movie is represented as a curve defined in an affine morph space and resampling the movie comprises calculating points on the curve at intervals defined by the second frame rate.
-
21. The program of claims 4 or 2, wherein the movie has a character with a gesture, further comprising instructions for editing the gesture.
-
22. The program of claims 4 or 2, further comprising instructions to compress the movie by using a subset of the basis.
-
23. The program of claims 4 or 2, further comprising instructions to display the movie by performing a multi-image morphing.
-
24. The program of claim 2, further comprising instructions to:
-
select a compression level; and
use the compression level to limit the number of feature vectors in the morph basis.
-
-
25. The program of claim 2, further comprising instructions to:
construct a set of new basis frames from the basis of feature vectors, each new basis frame corresponding to one of the feature vectors in the basis of feature vectors, wherein the new basis frames are not in the original frame sequence.
-
26. The program of claim 2, further comprising instructions to represent the original frame sequence as a parametric curve fit to the points (a1, a2, . . . , an) in the morph space.
-
27. The program of claim 2 for generating a motion blur in the movie, the program further comprising instructions to:
calculate an average color to substitute for an original color for each point in a blurred frame in the movie, the average color being calculated by averaging colors taken from each basis frame along a velocity field generated by a tangent vector and the warping transformations at the blurred frame, the tangent vector being a vector tangent to a curve representing the movie in a morph space defined by the morph basis.
-
3. A method for representing a movie having an original frame sequence and for generating a motion blur in the movie, the method comprising:
-
generating a pixel level frame to frame correspondence;
representing the original frame sequence as a multi-image morph;
deriving from the multi-image morph a morph basis of feature vectors describing principal changes among frames;
representing the movie in terms of the morph basis; and
calculating an average color to substitute for an original color for each point in a blurred frame in the movie, the average color being calculated by averaging colors taken from each basis frame along a velocity field generated by a tangent vector and the warping transformations at the blurred frame, the tangent vector being a vector tangent to a curve representing the movie in a morph space defined by the morph basis.
-
-
4. A computer program stored on a computer-readable medium for compressing a movie having an original frame sequence and for generating a motion blur in the movie, the program comprising instructions to:
-
generate a pixel level frame to frame correspondence;
represent the original frame sequence as a multi-image morph;
derive from the multi-image morph a morph basis of feature vectors describing principal changes among frames;
represent the movie in terms of the morph basis; and
calculate an average color to substitute for an original color for each point in a blurred frame in the movie, the average color being calculated by averaging colors taken from each basis frame along a velocity field generated by a tangent vector and the warping transformations at the blurred frame, the tangent vector being a vector tangent to a curve representing the movie in a morph space defined by the morph basis.
-
Specification