Interpolation of a sequence of images using motion analysis

• US 6,665,450 B1
• Filed: 09/08/2000
• Issued: 12/16/2003
• Est. Priority Date: 09/08/2000
• Status: Expired due to Term

• Current Assignee: Avid Technology Incorporated
• Original Assignee: Avid Technology Incorporated
• Inventors: Fasciano, Peter J., Cornog, Katherine H., Fayan, Randy M., Dickie, Garth A.
• Primary Examiner(s): COUSO, YON JUNG
• Application Number:
  US09/657,699
• Time in Patent Office: 1,194 Days
• Field of Search: 382/284, 382/276, 382/293, 382/294, 382/300, 382/107, 358/450, 358/540, 345/629, 345/639, 345/640, 345/646, 345/648, 345/683, 348/416.1, 348/407.1, 348/413.1
• US Class Current: 382/276
• CPC Class Codes: G06T 13/80 2D [Two Dimensional] animation, e.g. using sprites
• CPC Class Codes: G06T 2210/44 Morphing
• CPC Class Codes: G06T 7/215 Motion-based segmentation
• CPC Class Codes: H04N 5/147 Scene change detection
• CPC Class Codes: H04N 7/014 involving the use of motion vectors motion estimation and compensation in video coding H04N19/51

Abstract

Two images are analyzed to compute a set of motion vectors that describes motion between the first and second images. A motion vector is computed for each pixel in an image at a time between the first and second images. This set of motion vectors may be defined at any time between the first and second images, such as the midpoint. The motion vectors may be computed using any of several techniques. An example technique is based on the constant brightness constraint, also referred to as optical flow. Each vector is specified at a pixel center in an image defined at the time between the first and second images. The vectors may point to points in the first and second images that are not on pixel centers. The motion vectors are used to warp the first and second images to a point in time of an output image between the first and second images using a factor that represents the time between the first and second image at which the output image occurs. The warped images are then blended using this factor to obtain the output image at the desired point in time between the first and second images. The point in time at which the output image occurs may be different from the time at which the motion vectors are determined. The same motion vectors may be used to determine two or more output images at different times between the first and second images. The images may be warped using a technique in which many small triangles are defined in an image corresponding in time to the point in time between the first and second images at which the motion vectors are determined. A transform for each small triangle from the point in time at which the motion vectors are determined to the desired interpolated image time is determined, e.g., the triangle is warped using the motion vectors associated with its vertices. For each pixel in each triangle in the output image, corresponding points in the first and second images are determined, and the first and second images are spatially sampled at these points. These samples for each pixel are combined to produce a value for that pixel in the output image.

Claims

1. A method for generating an output image associated with a point in time between a first image and a second image, comprising:

determining a motion vector for each pixel in an image at a map time between the first image and the second image, wherein the map time is different from the point in time of the output image, wherein the motion vector describes motion of a pixel of the image at the map time to a first point in the first image and a second point in the second image;

calculating a factor that represents the point in time between the first image and the second image at which the output image occurs;

warping the first image according to the determined motion vectors and the factor;

warping the second image according to the determined motion vectors and the factor; and

blending the warped first image and the warped second image according to the factor to obtain the output image.

2. The method of claim 1, wherein the first image is in a first sequence of images and the second image is in a second sequence of images such that the first image is not contiguous with the second image in a sequence of images.

3. The method of claim 2, wherein the first sequence has associated audio and the second sequence has associated audio, the method further comprising:

dissolving the audio associated with the first sequence to the audio associated with the second sequence.

4. The method of claim 1, wherein the output image and the first and second images are an output sequence of images with a duration at playback different from a duration of an input sequence of images containing the first and second images at playback, and wherein the input sequence of images has associated audio with a duration, the method further comprising:

adjusting the duration of the audio to match the duration of the output sequence of images.

5. The method of claim 1, wherein warping the first image and the second image and blending the warped images, comprises:

determining a primary transform for each triangle in a set of triangles, defined in an image at the map time, from the map time to the output time using the determined motion vectors;

for each triangle, identifying any pixels in the output image that are contained within the triangle using the primary transform;

determining a first transform for each triangle in the set of triangles from the output time to a time of the first image;

for each pixel in each triangle at the output time, identifying a point in the first image using the first transform and spatially sampling the first image around the point;

determining a second transform for each triangle in the set of triangles from the output time to a time of the second image;

for each pixel in each triangle at the output time, identifying a point in the second image using the second transform and spatially sampling the second image around the point; and

for each pixel in each triangle at the output time, combining the spatially sampled first image and the spatially sampled second image to obtain a value for the pixel in the output image.

21. The method of claim 1, wherein determining motion vectors comprises processing the first and second images to remove invalid image data.

22. The method of claim 1, wherein warping comprises:

identifying any motion vector that transforms a point in the output image to an area outside of one of the first and second images; and

providing no contribution from one of the first and second images to the output image for the identified motion vector.

23. The method of claim 1, wherein blending comprises initializing an output image to a blend of the first and second images according to the determined factor.

6. A method for generating a plurality of output images, wherein each output image is associated with a different point in time between a first image and a second image, the method comprising:

determining a motion vector for each pixel in an image at a map time between the first image and the second image, wherein the motion vector describes motion of a pixel of the image at the map time to a first point in the first image and a second point in the second image;

for each output image, calculating a factor that represents the point in time between the first image and the second image at which the output image occurs;

for each output image, warping the first image according to the determined motion vectors and the factor for the output image;

for each output image, warping the second image according to the determined motion vectors and the factor for the output image; and

for each output image, blending the warped first image and the warped second image according to the factor for the output image.

7. The method of claim 6, wherein the first image is in a first sequence of images and the second image is in a second sequence of images such that the first image is not contiguous with the second image in a sequence of images.

8. The method of claim 7, wherein the first sequence has associated audio and the second sequence has associated audio, the method further comprising:

dissolving the audio associated with the first sequence to the audio associated with the second sequence.

9. The method of claim 6, wherein an output sequence of images includes the plurality of images and has a duration at playback different from a duration of an input sequence of images containing the first and second images at playback, and wherein the input sequence of images has associated audio with a duration, the method further comprising:

adjusting the duration of the audio to match the duration of the output sequence of images.

10. The method of claim 6, wherein warping the first image and the second image and blending the warped images, comprises:

determining a primary transform for each triangle in a set of triangles, defined in an image at the map time, from the map time to the output time using the determined motion vectors;

for each triangle, identifying any pixels in the output image that are contained within the triangle using the primary transform;

determining a first transform for each triangle in the set of triangles from the output time to a time of the first image;

for each pixel in each triangle at the output time, identifying a point in the first image using the first transform and spatially sampling the first image around the point;

determining a second transform for each triangle in the set of triangles from the output time to a time of the second image;

for each pixel in each triangle at the output time, identifying a point in the second image using the second transform and spatially sampling the second image around the point; and

for each pixel in each triangle at the output time, combining the spatially sampled first image and the spatially sampled second image to obtain a value for the pixel in the output image.

11. A method for generating a plurality of output images, wherein each output image is associated with a different point in time between a first image of a first sequence of one or more images and a second image of a second sequence of one or more images, the method comprising:

for each output image, selecting a pair of a first image from the first sequence and a second image from the second sequence;

for each selected pair of first and second images, determining a motion vector for each pixel in an image at a map time between the first image and the second image, wherein the motion vector describes motion of a pixel of the image at the map time to a first point in the first image and a second point in the second image;

for each output image, calculating a factor that represents the point in time, between the first and second images selected for the output image, at which the output image occurs;

for each output image, warping the first image selected for the output image according to the factor for the output image and the motion vectors determined for the first and second images selected for the output image;

for each output image, warping the second image selected for the output image according to the factor for the output image and the motion vectors determined for the first and second images selected for the output image; and

for each output image, blending the warped first image and the warped second image according to the factor for the output image.

12. The method of claim 11, wherein the first sequence has associated audio and the second sequence has associated audio, the method further comprising:

dissolving the audio associated with the first sequence to the audio associated with the second sequence.

13. The method of claim 11, wherein an output sequence of images includes the plurality of output images and has a duration at playback different from a duration of an input sequence of images containing the first and second images at playback, and wherein the input sequence of images has associated audio with a duration, the method further comprising:

adjusting the duration of the audio to match the duration of the output sequence of images.

14. The method of claim 11, wherein warping the first image and the second image and blending the warped images, comprises:

determining a primary transform for each triangle in a set of triangles, defined in an image at the map time, from the map time to the output time using the determined motion vectors;

for each triangle, identifying any pixels in the output image that are contained within the triangle using the primary transform;

determining a first transform for each triangle in the set of triangles from the output time to a time of the first image;

for each pixel in each triangle at the output time, identifying a point in the first image using the first transform and spatially sampling the first image around the point;

determining a second transform for each triangle in the set of triangles from the output time to a time of the second image;

for each pixel in each triangle at the output time, identifying a point in the second image using the second transform and spatially sampling the second image around the point; and

for each pixel in each triangle at the output time, combining the spatially sampled first image and the spatially sampled second image to obtain a value for the pixel in the output image.

15. A method for warping a first image and a second image to obtain an output image at an output time between the first image and the second image, comprising:

determining a set of motion vectors at a map time and that describes motion between the first and second images;

determining a primary transform for each triangle in a set of triangles, defined in an image at the map time, from the map time to the output time using the determined set of motion vectors;

for each triangle, identifying any pixels in the output image that are contained within the triangle using the primary transform;

determining a first transform for each triangle in the set of triangles from the output time to a time of the first image;

for each pixel in each triangle at the output time, identifying a point in the first image using the first transform and spatially sampling the first image around the point;

determining a second transform for each triangle in the set of triangles from the output time to a time of the second image;

for each pixel in each triangle at the output time, identifying a point in the second image using the second transform and spatially sampling the second image around the point; and

for each pixel in each triangle at the output time, combining the spatially sampled first image and the spatially sampled second image to obtain a value for the pixel in the output image.

16. The method of claim 15, wherein the map time is between the first image and the second image.

17. The method of claim 15, wherein the map time is different from the output time.

18. A method for warping a first image and a second image to obtain an output image at an output time between the first image and the second image, comprising:

determining a set of motion vectors at a map time and that describes motion between the first and second images;

determining a primary transform for each triangle in a set of triangles, defined in an image at the map time, from the map time to the output time using the determined set of motion vectors;

for each triangle, identifying any pixels in the output image that are contained within the triangle at the output time using the primary transform;

for each pixel in each triangle at the output time, spatially sampling the first image and the second image at points corresponding to the pixel and combining the spatially sampled first image and the spatially sampled second image to obtain a value for the pixel in the output image.

19. The method of claim 18, wherein the map time is between the first image and the second image.

20. The method of claim 18, wherein the map time is different from the output time.

24. A computer program product, comprising:

a computer readable medium;

computer program instructions stored on the computer readable medium that, when processed by a computer, instruct the computer to perform a method for generating an output image associated with a point in time between a first image and a second image, comprising;

determining a motion vector for each pixel in an image at a map time between the first image and the second image, wherein the map time is different from the point in time of the output image, wherein the motion vector describes motion of a pixel of the image at the map time to a first point in the first image and a second point in the second image;

calculating a factor that represents the point in time between the first image and the second image at which the output image occurs;

warping the first image according to the determined motion vectors and the factor;

warping the second image according to the determined motion vectors and the factor; and

blending the warped first image and the warped second image according to the factor to obtain the output image.

25. The computer program product of claim 24, wherein the first image is in a first sequence of images and the second image is in a second sequence of images such that the first image is not contiguous with the second image in a sequence of images.

26. The computer program product of claim 25, wherein the first sequence has associated audio and the second sequence has associated audio, and further comprising:

dissolving the audio associated with the first sequence to the audio associated with the second sequence.

27. The computer program product of claim 24, wherein the output image and the first and second images are an output sequence of images with a duration at playback different from a duration of an input sequence of images containing the first and second images at playback, and wherein the input sequence of images has associated audio with a duration, and further comprising:

adjusting the duration of the audio to match the duration of the output sequence of images.

28. The computer program product of claim 24, wherein warping the first image and the second image and blending the warped images, comprises:

determining a primary transform for each triangle in a set of triangles, defined in an image at the map time, from the map time to the output time using the determined motion vectors;

for each triangle, identifying any pixels in the output image that are contained within the triangle using the primary transform;

determining a first transform for each triangle in the set of triangles from the output time to a time of the first image;

for each pixel in each triangle at the output time, identifying a point in the first image using the first transform and spatially sampling the first image around the point;

determining a second transform for each triangle in the set of triangles from the output time to a time of the second image;

for each pixel in each triangle at the output time, identifying a point in the second image using the second transform and spatially sampling the second image around the point; and

for each pixel in each triangle at the output time, combining the spatially sampled first image and the spatially sampled second image to obtain a value for the pixel in the output image.

29. The computer program product of claim 24, wherein determining motion vectors comprises processing the first and second images to remove invalid image data.

30. The computer program product of claim 24, wherein warping comprises:

identifying any motion vector that transforms a point in the output image to an area outside of one of the first and second images; and

providing no contribution from one of the first and second images to the output image for the identified motion vector.

31. The computer program product of claim 24, wherein blending comprises initializing an output image to a blend of the first and second images according to the determined factor.

32. A computer program product, comprising:

a computer readable medium;

computer program instructions stored on the computer readable medium that, when processed by a computer, instruct the computer to perform a method for generating a plurality of output images, wherein each output image is associated with a different point in lime between a first image and a second image, comprising;

determining a motion vector for each pixel in an image at a map time between the first image and the second image, wherein the motion vector describes motion of a pixel of the image at the map time to a first point in the first image and a second point in the second image;

for each output image, calculating a factor that represents the point in time between the first image and the second image at which the output image occurs;

for each output image, warping the first image according to the determined motion vectors and the factor for the output image;

for each output image, warping the second image according to the determined motion vectors and the factor for the output image; and

for each output image, blending the warped first image and the warped second image according to the factor for the output image.

33. The computer program product of claim 32, wherein the first image is in a first sequence of images and the second image is in a second sequence of images such that the first image is not contiguous with the second image in a sequence of images.

34. The computer program product of claim 33, wherein the first sequence has associated audio and the second sequence has associated audio, and further comprising:

dissolving the audio associated with the first sequence to the audio associated with the second sequence.

35. The computer program product of claim 32, wherein an output sequence of images includes the plurality of images and has a duration at playback different from a duration of an input sequence of images containing the first and second images at playback, and wherein the input sequence of images has associated audio with a duration, and further comprising:

adjusting the duration of the audio to match the duration of the output sequence of images.

36. The computer program product of claim 32, wherein warping the first image and the second image and blending the warped images, comprises:

determining a primary transform for each triangle in a set of triangles, defined in an image at the map time, from the map time to the output time using the determined motion vectors;

for each triangle, identifying any pixels in the output image that are contained within the triangle using the primary transform;

determining a first transform for each triangle in the set of triangles from the output time to a time of the first image;

for each pixel in each triangle at the output time, identifying a point in the first image using the first transform and spatially sampling the first image around the point;

determining a second transform for each triangle in the set of triangles from the output time to a time of the second image;

for each pixel in each triangle at the output time, identifying a point in the second image using the second transform and spatially sampling the second image around the point; and

for each pixel in each triangle at the output time, combining the spatially sampled first image and the spatially sampled second image to obtain a value for the pixel in the output image.

37. A computer program product, comprising:

a computer readable medium;

computer program instructions stored on the computer readable medium that, when processed by a computer, instruct the computer to perform a method for generating a plurality of output images, wherein each output image is associated with a different point in time between a

first image of a first sequence of one or more images and a second image of a second sequence of one or more images, comprising;

for each output image, selecting a pair of a first image from the first sequence and a second image from the second sequence;

for each selected pair of first and second images, determining a motion vector for each pixel in an image at a map time between the first image and the second image, wherein the motion vector describes motion of a pixel of the image at the map time to a first point in the first image and a second point in the second image;

for each output image, calculating a factor that represents the point in time, between the first and second images selected for the output image, at which the output image occurs;

for each output image, warping the first image selected for the output image according to the factor for the output image and the motion vectors determined for the first and second images selected for the output image;

for each output image, warping the second image selected for the output image according to the factor for the output image and the motion vectors determined for the first and second images selected for the output image; and

for each output image, blending the warped first image and the warped second image according to the factor for the output image.

38. The computer program product of claim 37, wherein the first sequence has associated audio and the second sequence has associated audio, and further comprising:

dissolving the audio associated with the first sequence to the audio associated with the second sequence.

39. The computer program product of claim 37, wherein an output sequence of images includes the plurality of output images and has a duration at playback different from a duration of an input sequence of images containing the first and second images at playback, and wherein the input sequence of images has associated audio with a duration, and further comprising:

adjusting the duration of the audio to match the duration of the output sequence of images.

40. The computer program product of claim 37, wherein warping the first image and the second image and blending the warped images, comprises:

determining a primary transform for each triangle in a set of triangles, defined in an image at the map time, from the map time to the output time using the determined motion vectors;

for each triangle, identifying any pixels in the output image that are contained within the triangle using the primary transform;

determining a first transform for each triangle in the set of triangles from the output time to a time of the first image;

for each pixel in each triangle at the output time, identifying a point in the first image using the first transform and spatially sampling the first image around the point;

determining a second transform for each triangle in the set of triangles from the output time to a time of the second image;

for each pixel in each triangle at the output time, identifying a point in the second image using the second transform and spatially sampling the second image around the point; and

for each pixel in each triangle at the output time, combining the spatially sampled first image and the spatially sampled second image to obtain a value for the pixel in the output image.

41. A computer program product, comprising:

a computer readable medium;

computer program instructions stored on the computer readable medium that, when processed by a computer, instruct the computer to perform a method for warping a first image and a second image to obtain an output image at an output time between the first image and the second image, comprising;

determining a set of motion vectors at a map time and that describes motion between the first and second images;

determining a primary transform for each triangle in a set of triangles, defined in an image at the map time, from the map time to the output time using the determined set of motion vectors;

for each triangle, identifying any pixels in the output image that are contained within the triangle using the primary transform;

determining a first transform for each triangle in the set of triangles from the output time to a time of the first image;

for each pixel in each triangle at the output time, identifying a point in the first image using the first transform and spatially sampling the first image around the point;

determining a second transform for each triangle in the set of triangles from the output time to a time of the second image;

for each pixel in each triangle at the output time, identifying a point in the second image using the second transform and spatially sampling the second image around the point; and

for each pixel in each triangle at the output time, combining the spatially sampled first image and the spatially sampled second image to obtain a value for the pixel in the output image.

42. The computer program product of claim 41, wherein the map time is between the first image and the second image.

43. The computer program product of claim 41, wherein the map time is different from the output time.

44. A computer program product, comprising:

a computer readable medium;

computer program instructions stored on the computer readable medium that, when processed by a computer, instruct the computer to perform a method for warping a first image and a second image to obtain an output image at an output time between the first image and the second image, comprising;

determining a set of motion vectors at a map time and that describes motion between the first and second images;

determining a primary transform for each triangle in a set of triangles, defined in an image at the map time, from the map time to the output time using the determined set of motion vectors;

for each triangle, identifying any pixels in the output image that are contained within the triangle at the output time using the primary transform;

for each pixel in each triangle at the output time, spatially sampling the first image and the second image at points corresponding to the pixel and combining the spatially sampled first image and the spatially sampled second image to obtain a value for the pixel in the output image.

45. The computer program product of claim 44, wherein the map time is between the first image and the second image.

46. The computer program product of claim 44, wherein the map time is different from the output time.

47. Apparatus for generating an output image associated with a point in time between a first image and a second image, comprising:

means for computing a motion vector for each pixel in an image at a map time between the first image and the second image, wherein the map time is different from the point in time of the output image, wherein the motion vector describes motion of a pixel of the image at the map tabs time to a first point in the first image and a second point in the second image;

means for computing a factor that represents the point in time between the first image and the second image at which the output image occurs;

means for warping the first image according to the determined motion vectors and the factor;

means for warping the second image according to the determined motion vectors and the factor; and

means for blending the warped first image and the warped second image according to the factor to obtain the output image.

48. The apparatus of claim 47, wherein the first image is in a first sequence of images and the second image is in a second sequence of images such that the first image is not contiguous with the second image in a sequence of images.

49. The apparatus of claim 48, wherein the first sequence has associated audio and the second sequence has associated audio, and further comprising:

means for dissolving the audio associated with the first sequence to the audio associated with the second sequence.

50. The apparatus of claim 47, wherein the output image and the first and second images are an output sequence of images with a duration at playback different from a duration of an input sequence of images containing the first and second images at playback, and wherein the input sequence of images has associated audio with a duration, and further comprising:

means for adjusting the duration of the audio to match the duration of the output sequence of images.

51. The apparatus of claim 47, wherein the means for warping the first image and the second image and the means for blending the warped images, comprises:

means for determining a primary transform for each triangle in a set of triangles, defined in an image at the map time, from the map time to the output time using the determined motion vectors;

means for identifying, for each triangle, any pixels in the output image that are contained within the triangle using the primary transform;

means for determining a first transform for each triangle in the set of triangles from the output time to a time of the first image;

means for identifying, for each pixel in each triangle at the output time, a point in the first image using the first transform and spatially sampling the first image around the point;

means for determining a second transform for each triangle in the set of triangles from the output time to a time of the second image;

means for identifying, for each pixel in each triangle at the output time, a point in the second image using the second transform and spatially sampling the second image around the point; and

means for combining, for each pixel in each triangle at the output time, the spatially sampled first image and the spatially sampled second image to obtain a value for the pixel in the output image.

52. The apparatus of claim 47, wherein the means for computing motion vectors comprises means for processing the first and second images to remove invalid image data.

53. The apparatus of claim 47, wherein the means for warping comprises:

means for identifying any motion vector that transforms a point in the output image to an area outside of one of the first and second images; and

means for providing no contribution from one of the first and second images to the output image for the identified motion vector.

54. The apparatus of claim 47, wherein the means for blending comprises means for initializing an output image to a blend of the first and second images according to the determined factor.

55. Apparatus for generating a plurality of output images, wherein each output image is associated with a different point in time between a first image and a second image, comprising:

means for computing a motion vector for each pixel in an image at a map time between the first image and the second image, wherein the motion vector describes motion of a pixel of the image at the map time to a first point in the first image and a second point in the second image;

means for calculating, for each output image, a factor that represents the point in time between the first image and the second image at which the output image occurs;

means for warping, for each output image, the first image according to the determined motion vectors and the factor for the output image;

means for warping, for each output image, the second image according to the determined motion vectors and the factor for the output image; and

means for blending, for each output image, the warped first image and the warped second image according to the factor for the output image.

56. The apparatus of claim 55, wherein the first image is in a first sequence of images and the second image is in a second sequence of images such that the first image is not contiguous with the second image in a sequence of images.

57. The apparatus of claim 56, wherein the first sequence has associated audio and the second sequence has associated audio, further comprising:

means for dissolving the audio associated with the first sequence to the audio associated with the second sequence.

58. The apparatus of claim 55, wherein an output sequence of images includes the plurality of images and has a duration at playback different from a duration of an input sequence of images containing the first and second images at playback, and wherein the input sequence of images has associated audio with a duration, the apparatus further comprising:

means for adjusting the duration of the audio to match the duration of the output sequence of images.

59. The apparatus of claim 55, wherein the means for warping the first image and the second image and the means for blending the warped images, comprises:

means for determining a primary transform for each triangle in a set of triangles, defined in an image at the map time, from the map time to the output time using the determined motion vectors;

means for identifying, for each triangle, any pixels in the output image that are contained within the triangle using the primary transform;

means for determining a first transform for each triangle in the set of triangles from the output time to a time of the first image;

means for identifying, for each pixel in each triangle at the output time, a point in the first image using the first transform and spatially sampling the first image around the point;

means for determining a second transform for each triangle in the set of triangles from the output time to a time of the second image;

means for identifying, for each pixel in each triangle at the output time, a point in the second image using the second transform and spatially sampling the second image around the point; and

means for combining, for each pixel in each triangle at the output time, the spatially sampled first image and the spatially sampled second image to obtain a value for the pixel in the output image.

60. Apparatus for generating a plurality of output images, wherein each output image is associated with a different point in time between a first image of a first sequence of one or more images and a second image of a second sequence of one or more images, comprising:

means for selecting, for each output image, a pair of a first image from the first sequence and a second image from the second sequence;

means for determining, for each selected pair of first and second images, a motion vector for each pixel in an image at a map time between the first image and the second image, wherein the motion vector describes motion of a pixel of the image at the map time to a first point in the first image and a second point in the second image;

means for calculating, for each output image, a factor that represents the point in time, between the first and second images selected for the output image, at which the output image occurs;

means for warping, for each output image, the first image selected for the output image according to the factor for the output image and the motion vectors determined for the first and second images selected for the output image;

means for warping, for each output image, the second image selected for the output image according to the factor for the output image and the motion vectors determined for the first and second images selected for the output image; and

means for blending, for each output image, the warped first image and the warped second image according to the factor for the output image.

61. The apparatus of claim 60, wherein the first sequence has associated audio and the second sequence has associated audio, further comprising:

means for dissolving the audio associated with the first sequence to the audio associated with the second sequence.

62. The apparatus of claim 60, wherein an output sequence of images includes the plurality of output images and has a duration at playback different from a duration of an input sequence of images containing the first and second images at playback, and wherein the input sequence of images has associated audio with a duration, further comprising:

means for adjusting the duration of the audio to match the duration of the output sequence of images.

63. The apparatus of claim 60, wherein the means for warping the first image and the second image and the means for blending the warped images, comprises:

means for determining a primary transform for each triangle in a set of triangles, defined in an image at the map time, from the map time to the output time using the determined motion vectors;

means for identifying, for each triangle, any pixels in the output image that are contained within the triangle using the primary transform;

means for determining a first transform for each triangle in the set of triangles from the output time to a time of the first image;

means for identifying, for each pixel in each triangle at the output time, a point in the first image using the first transform and spatially sampling the first image around the point;

means for determining a second transform for each triangle in the set of triangles from the output time to a time of the second image;

means for identifying, for each pixel in each triangle at the output time, a point in the second image using the second transform and spatially sampling the second image around the point; and

means for combining, for each pixel in each triangle at the output time, the spatially sampled first image and the spatially sampled second image to obtain a value for the pixel in the output image.

64. Apparatus for warping a first image and a second image to obtain an output image at an output time between the first image and the second image, comprising:

means for determining a set of motion vectors at a map time and that describes motion between the first and second images;

means for determining a primary transform for each triangle in a set of triangles, defined in an image at the map time, from the map time to the output time using the determined set of motion vectors;

means for identifying, for each triangle, any pixels in the output image that are contained within the triangle using the primary transform;

means for determining a first transform for each triangle in the set of triangles from the output time to a time of the first image;

means for identifying, for each pixel in each triangle at the output time, a point in the first image using the first transform and spatially sampling the first image around the point;

means for determining a second transform for each triangle in the set of triangles from the output time to a time of the second image;

means for identifying, for each pixel in each triangle at the output time, a point in the second image using the second transform and spatially sampling the second image around the point; and

means for combining, for each pixel in each triangle at the output time, the spatially sampled first image and the spatially sampled second image to obtain a value for the pixel in the output image.

65. The apparatus of claim 64, wherein the map time is between the first image and the second image.

66. The apparatus of claim 64, wherein the map time is different from the output time.

67. Apparatus for warping a first image and a second image to obtain an output image at an output time between the first image and the second image, comprising:

means for determining a set of motion vectors at a map time and that describes motion between the first and second images;

means for determining a primary transform for each triangle in a set of triangles, defined in an image at the map time, from the map time to the output time using the determined set of motion vectors;

means for identifying, for each triangle, any pixels in the output image that are contained within the triangle at the output time using the primary transform;

means for spatially sampling, for each pixel in each triangle at the output time, the first image and the second image at points corresponding to the pixel and for combining the spatially sampled first image and the spatially sampled second image to obtain a value for the pixel in the output image.

68. The apparatus of claim 67, wherein the map time is between the first image and the second image.

69. The apparatus of claim 67, wherein the map time is different from the output time.