Interpolation of a sequence of images using motion analysis
First Claim
1. A method for generating a transition of a plurality of output images from a first sequence of images to a second sequence of images wherein an image at an end of the first sequence is not contiguous with an image at a beginning of the second sequence, the method comprising:
- for each output image, selecting a pair of a first image from the first sequence and a second image from the second sequence such that the output image has a point in time between the first image and the second image in the transition;
for each selected pair of first and second images, determining a set of motion vectors that describes motion between the first and second images;
for each output image, calculating a factor that represents the point in time, between the first and second images selected for the output image, at which the output image occurs;
for each output image, performing motion compensated interpolation to generate the output image according to the determined set of motion vectors and the calculated factor.
9 Assignments
0 Petitions
Accused Products
Abstract
Two images are analyzed to compute a set of motion vectors that describes motion between the first and second images. A motion vector is computed for each pixel in an image at a time between the first and second images. This set of motion vectors may be defined at any time between the first and second images, such as the midpoint. The motion vectors may be computed using any of several techniques. An example technique is based on the constant brightness constraint, also referred to as optical flow. Each vector is specified at a pixel center in an image defined at the time between the first and second images. The vectors may point to points in the first and second images that are not on pixel centers. The motion vectors are used to warp the first and second images to a point in time of an output image between the first and second images using a factor that represents the time between the first and second image at which the output image occurs. The warped images are then blended using this factor to obtain the output image at the desired point in time between the first and second images. The point in time at which the output image occurs may be different from the time at which the motion vectors are determined. The same motion vectors may be used to determine two or more output images at different times between the first and second images. The images may be warped using a technique in which many small triangles are defined in an image corresponding in time to the point in time between the first and second images at which the motion vectors are determined. A transform for each small triangle from the point in time at which the motion vectors are determined to the desired interpolated image time is determined, e.g., the triangle is warped using the motion vectors associated with its vertices. For each pixel in each triangle in the output image, corresponding points in the first and second images are determined, and the first and second images are spatially sampled at these points. These samples for each pixel are combined to produce a value for that pixel in the output image.
-
Citations
12 Claims
-
1. A method for generating a transition of a plurality of output images from a first sequence of images to a second sequence of images wherein an image at an end of the first sequence is not contiguous with an image at a beginning of the second sequence, the method comprising:
for each output image, selecting a pair of a first image from the first sequence and a second image from the second sequence such that the output image has a point in time between the first image and the second image in the transition;
for each selected pair of first and second images, determining a set of motion vectors that describes motion between the first and second images;
for each output image, calculating a factor that represents the point in time, between the first and second images selected for the output image, at which the output image occurs;
for each output image, performing motion compensated interpolation to generate the output image according to the determined set of motion vectors and the calculated factor. - View Dependent Claims (2, 3)
-
4. A method for processing a jump cut from a first image at an end of a first segment of sequence of images and corresponding audio and a second image at a beginning of a second segment in the sequence of images and corresponding audio, comprising:
-
processing the corresponding audio to identify an audio break between the audio corresponding to the first segment and the audio corresponding to the second segment;
determining a set of motion vectors that describes motion between the first and second images; and
performing motion compensated interpolation to generate one or more images between the first image and the second image according to the determined set of motion vectors at a point in time corresponding to the audio break. - View Dependent Claims (5, 6, 7)
-
-
8. A method for changing duration of an input sequence of images with associated audio, wherein the input sequence of images and associated audio has a duration, comprising:
-
receiving an indication of a selection of an operation by an operator indicative of a desired duration of an output sequence of images, and, in response to the received indication;
selecting a first image and a second image in the sequence of images;
determining a set of motion vectors that describes motion between the first and second images;
performing motion compensated interpolation to generate one or more images between the first image and the second image according to the determined set of motion vectors;
repeating the selecting, determining and performing steps for multiple pairs of first and second images in the sequence of images to provide the output sequence of images; and
adjusting the duration of the associated audio to retain synchronization with the output sequence of images. - View Dependent Claims (9, 10, 11)
-
-
12. A method for performing color correction, comprising:
-
generating a first color histogram from first image from a first sequence of images;
generating a second color histogram from a second image from a second sequence of images;
determining a set of motion vectors from the first and second color histograms that describes motion between the first color histogram and the second color histogram;
generating a table of color correction values from the set of motion vectors; and
applying the table of color correction values to a sequence of images.
-
Specification