Application of speed effects to a video presentation

• US 7,725,828 B1
• Filed: 10/15/2003
• Issued: 05/25/2010
• Est. Priority Date: 10/15/2003
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Johnson, Gary
• Primary Examiner(s): Theriault; Steven B
• Application Number:
  US10/686,990
• Time in Patent Office: 2,414 Days
• Field of Search: 715/704, 715719-727, 715/732, 715/735, 704/500
• US Class Current: 715/726
• CPC Class Codes: G11B 27/005 Reproducing at a different information rate from the information rate of recording for television signals H04N5/783
• CPC Class Codes: G11B 27/034 on discs G11B27/036, G11B27/038 take precedence
• CPC Class Codes: G11B 27/036 Insert-editing
• CPC Class Codes: G11B 27/34 Indicating arrangements indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D

Abstract

Some embodiments provide a method of specifying speed effects for playing a video clip. The method defines a set of speed effects for the video clip. It then displays in real-time a presentation of the video clip that accounts for the set of speed effects defined for the video clip. In some embodiments, this method represents the playback speed of a video clip in terms of a graph that is part of a graphical user interface (“GUI”). This graph is defined along two axes, with one axis representing the playback time, and the other axis representing the content-time (i.e., the time within the video clip). In these embodiments, a user can change the playback speed of the video clip by using a set of GUI operations to select and modify the graph. For instance, a user can select and adjust the graph at different instances in time in order to change the playback speed of the video clip at these instances. Different embodiments use different types of graphs to represent playback speed. For instance, some embodiments use a deformable line bar that is superimposed on a rectangle that represents the video clip.

Claims

1. A method of specifying playback speed effects for playing a video clip, the method comprising:

a) receiving a set of playback speed effects for the video clip through a set of modifications of a user selectable, directly deformable line bar on a graph with a playback-time axis and a content-time axis that is displayed on a graphical display device, wherein the deformable line bar represents a playback-time in relation to a content-time of the video clip, wherein each of a plurality of points on the line bar represents a particular playback-time and a particular content-time, wherein the set of modifications comprises a click-and-drag operation received from a user on the line bar in order to deform the line bar; and

b) displaying in real-time a presentation of the video clip that accounts for the set of playback speed effects defined for the video clip.

2. The method of claim 1, wherein the set of playback speed effects comprises only one playback speed effect.

3. The method of claim 1, wherein the set of playback speed effects comprises a plurality of playback speed effects that specify a plurality of playback speeds for a plurality of intervals.

4. The method of claim 1 further comprising defining the set of playback speed effects, wherein defining the set of playback speed effects comprises converting the click-and-drag operation into a set of playback speed effect definitions.

5. The method of claim 1, wherein receiving said click-and-drag operation comprises monitoring a location of a cursor controlled by an electronic user input device and a status of buttons of the electronic user input device.

6. The method of claim 5, wherein deforming the line bar comprises receiving a selection, from the electronic user input device, of a portion of the line bar that appears at a first location within a window containing the graph and receiving an input from the electronic user input device commanding a movement of the selected portion to a second location within the window.

7. The method of claim 1, wherein displaying in real-time the presentation of the video clip comprises displaying the video presentation without rendering the presentation to a data storage.

8. The method of claim 7 further comprising after specifying the set of playback speed effects for the video clip, rendering the video clip to a data storage.

9. The method of claim 1, wherein the video clip comprises a plurality of frames, wherein displaying in real-time the presentation of the video clip comprises:

a) selecting a first frame for display at a first playback-time;

b) displaying the first frame for display at the first playback-time;

c) selecting a second frame for display at a second playback-time, wherein the second playback-time is based at least partly on the set of playback speed effects; and

d) displaying the second frame for display at the second playback-time.

10. The method of claim 1, wherein the video clip comprises a plurality of frames, wherein displaying in real-time the presentation of the video clip comprises:

a) selecting a first frame for display for a first playback duration based on the defined set of playback speed effects;

b) displaying the first frame during the first playback duration;

c) selecting a second frame for a second playback duration based on the defined set of playback speed effects; and

d) displaying the second frame during the second playback duration.

11. The method of claim 10 further comprising:

a) before displaying the first frame, decompressing the first frame; and

b) before displaying the second frame, decompressing the second frame.

12. The method of claim 1, wherein the video clip comprises a composite of a plurality of video clips.

13. The method of claim 12, wherein the video clip comprises at least one audio track.

14. The method of claim 1 further comprising:

a) receiving a selection of a point on the line bar;

b) displaying a line tangent to the line bar at said point, wherein the tangent line has a first end and a second end;

c) receiving user input comprising a selection of the first end of said line and a drag operation on said first end;

d) modifying a slope of the line bar at the selected point in response to the drag operation on said first end; and

e) modifying other portions of the line bar to generate a smooth line bar consistent with the modified slope of the line bar at the selected point.

15. The method of claim 14, wherein the first end of the tangent line and the second end of the tangent line are represented by knobs.

16. The method of claim 1 further comprising displaying a plurality of time markers on the playback-time axis, wherein:

a first adjacent pair of time markers is separated by a first separation;

a second adjacent pair of time markers is separated by a second separation;

the first separation and the second separation each represent a same amount of content-time; and

the first separation and the second separation each represent a different amount of playback-time.

17. The method of claim 16 further comprising:

a) displaying, in a first color, a set of time markers that indicate playback-times during which the video clip is playing forward; and

b) displaying, in a second color that is different from the first color, a set of time markers that indicate playback-times during which the video clip is playing backward.

18. The method of claim 16, wherein said computer program further comprises a set of instructions for receiving user inputs of clicking and dragging on the time markers to define changes to the playback speed effects.

19. The method of claim 1 further comprising, when the user selects, with a GUI operation, a point on the line bar, displaying an indicator box around the selected point, wherein a position of the indicator box relative to the selected point indicates the content-time represented by the selected point.

20. The method of claim 19 further comprising:

a) moving the selected point along the content-time axis of the graph in response to a user drag operation; and

b) moving the indicator box relative to the selected point to indicate a new content-time of the selected point.

21. The method of claim 19 further comprising:

a) moving the selected point along the playback-time axis of the graph in response to a user drag operation; and

b) moving the indicator box along the playback-time axis, while maintaining the position of the indicator box relative to the moving point.

22. A computer readable medium storing a computer program for specifying playback speed effects for playing a video clip, said computer program for execution by at least one processor, the computer program comprising sets of instructions for:

a) defining a set of playback speed effects for the video clip;

b) displaying in real-time a presentation of the video clip that accounts for the set of playback speed effects defined for the video clip;

c) displaying a graph that represents a playback-time of the video clip in relation to a content-time of the video clip, wherein the graph comprises a playback-time axis and a content-time axis;

d) displaying a line bar on the graph comprising a set of points that each indicate the playback-time of a particular frame of the video clip and the content-time of the particular frame;

e) receiving user inputs of modifications directly to the line bar; and

f) converting the user inputs of direct modifications of the graph into the set of playback speed effects.

23. The computer readable medium of claim 22, wherein the set of instructions for receiving user inputs of modifications directly to the line bar comprises a set of instructions for allowing the user to select a portion of the line bar that appears at a first location within a window containing the graph and to move the selected portion to a second location within the window.

24. The computer readable medium of claim 22, wherein the set of instructions for displaying in real-time the presentation of the video clip comprises a set of instructions for displaying the video presentation without rendering the presentation to a data storage.

25. The computer readable medium of claim 22, wherein the video clip comprises a plurality of frames, wherein the set of instructions for displaying in real-time the presentation of the video clip comprises sets of instructions for:

a) selecting a first frame for display at a first playback-time;

b) displaying the first frame for display at the first playback-time;

c) selecting a second frame for display at a second playback-time; and

d) displaying the second frame for display at the second playback-time, wherein said second playback-time is determined at least partly by said set of playback speed effects.

26. The computer readable medium of claim 22, wherein said computer program further comprises a set of instructions for generating a set of blended frames from at least two frames of said video clip.

27. The computer readable medium of claim 26, wherein said set of instructions for generating said set of blended frames comprises sets of instructions for:

a) multiplying a first frame by a first blending parameter;

b) multiplying a second frame by a second blending parameter; and

c) adding a result of said multiplying together to produce a blended frame.

28. The computer readable medium of claim 27, wherein said computer program further comprises a set of instructions for generating sequential frames of said set of blended frames by changing a magnitude of said first and second blending parameters.

29. The computer readable medium of claim 22, wherein receiving said user inputs of modifications comprises receiving a drag operation to an end of a line that determines a slope of a portion of the line bar.

30. The computer readable medium of claim 22, wherein said computer program further comprises sets of instructions for:

a) receiving a selection of a point on the line bar;

b) displaying a line tangent to the line bar at said point, wherein the tangent line has a first end and a second end;

c) receiving user input comprising a selection of the first end of said line and a drag operation on said first end;

d) modifying a slope of the line bar at the selected point in response to the drag operation on said first end; and

e) modifying other portions of the line bar to generate a smooth line bar consistent with the modified slope of the line bar at the selected point.

31. The computer readable medium of claim 30, wherein the first end of the tangent line and the second end of the tangent line are represented by knobs.

32. The computer readable medium of claim 22, wherein said computer program further comprises sets of instructions for displaying a plurality of time markers on the playback-time axis, wherein:

a first adjacent pair of time markers is separated by a first separation;

a second adjacent pair of time markers is separated by a second separation;

the first separation and the second separation each represent a same amount of content-time; and

the first separation and the second separation each represent a different amount of playback-time.

33. The computer readable medium of claim 32, wherein said computer program further comprises sets of instructions for:

a) displaying, in a first color, a set of time markers that indicate playback-times during which the video clip is playing forward; and

b) displaying, in a second color that is different from the first color, a set of time markers that indicate playback-times during which the video clip is playing backward.

34. The computer readable medium of claim 32, wherein said computer program further comprises a set of instructions for receiving user inputs of clicking and dragging on the time markers to define changes to the playback speed effects.

35. The computer readable medium of claim 22, wherein said computer program further comprises sets of instructions for, when a user selects, with a GUI operation, a point on the line bar, displaying an indicator box around the selected point, wherein a position of the indicator box relative to the selected point indicates the content-time represented by the selected point.

36. The computer readable medium of claim 35, wherein said computer program further comprises sets of instructions for:

a) moving the selected point along the content-time axis of the graph in response to a user drag operation; and

b) moving the indicator box relative to the selected point to indicate a new content-time of the selected point.

37. The computer readable medium of claim 35, wherein said computer program further comprises sets of instructions for:

a) moving the selected point along the playback-time axis of the graph in response to a user drag operation; and

b) moving the indicator box along the playback-time axis, while maintaining the position of the indicator box relative to the moving point.

38. A graphical user interface (“

GUI”

) method for specifying playback speed effects for a video presentation, the method comprising;

a) displaying on an graphical display device a GUI graph of a playback-time of the video presentation relative to a content-time of the video presentation, wherein said GUI graph comprises a playback-time axis, a content-time axis, and a line bar representing the playback-time versus the content-time; and

b) accepting user input that determines a playback speed effect for the video presentation by a selection and direct modification of a portion of the line bar through a drag operation.

39. The method of claim 38 further comprising providing a set of controls for selecting portions of the line bar and performing drag operations.

40. The method of claim 38, wherein

the playback-time axis represents time during a playback of the video presentation;

and

the content-time axis represents time within the video presentation.

41. The method of claim 38, wherein the selected portion of the line bar comprises a keyframe.

42. The method of claim 41, wherein at any time, the keyframe has a value along the playback-time axis and a value along the content-time axis, wherein when the keyframe is selected, the keyframe has a first content-time value, the method further comprising:

when the keyframe is selected, displaying a frame that appears in the video presentation at the first content-time value.

43. The method of claim 42 further comprising:

displaying the frame, in the video presentation, that corresponds to the first content-time value of the keyframe when the first content-time value of the keyframe changes during the drag operation.

44. The method of claim 42, wherein performing a further comprising, in response to the drag operation, moving the keyframe along the playback-time axis when:

a) the drag operation is along the playback-time axis; and

b) a particular keyboard key is pressed.

45. The method of claim 42 further comprising, in response to the drag operation, moving the keyframe along the playback-time axis when:

a) the drag operation is along the playback-time axis; and

b) a particular key on a keyboard is not pressed.

46. The method of claim 43 further comprising, in response to the drag operation, moving the keyframe along the playback-time axis when the drag operation is along the playback-time axis.

47. The method of claim 43 further comprising, in response to the drag operation, moving the keyframe along the content-time axis when the drag operation is along the content-time axis.

48. The method of claim 43 further comprising:

displaying a graphical representation of the video presentation when the keyframe is selected,

in response to the drag operation;

a) moving the keyframe along the content-time axis when the drag operation is along the content-time axis; and

b) moving the graphical representation along the playback-time axis when the drag operation is along the content-time axis.

49. The method of claim 43 further comprising:

displaying a graphical representation of the video presentation when the keyframe is selected; and

in response to the drag operation moving the graphical representation along the playback-time axis when the drag operation is along the playback-time axis.

50. The method of claim 49, further comprising, in response to the drag operation moving the keyframe along the content-time axis when the drag operation is along the playback-time axis.

51. The method of claim 44, wherein moving the keyframe along the playback-time axis comprises moving the keyframe without changing the first content-time value, while displaying the frame in the video presentation that corresponds to the content-time value of the keyframe.

52. The method of claim 51, wherein moving the keyframe further comprises setting a new playback-time value for said keyframe.

53. The method of claim 45, wherein moving the keyframe along the playback-time axis comprises moving the keyframe without changing the first content-time value, while displaying the frame in the video presentation that corresponds to the content-time value of the keyframe.

54. The method of claim 38, wherein said GUI graph displays said line bar as a curve on said GUI graph.

55. A computer readable medium storing an application, the application comprising a graphical user interface (“

GUI”

), the GUI comprising;

a display area for displaying a video presentation; and

a GUI graph of playback-time versus content-time with a selectable line bar representing a playback-time of the video presentation relative to a content-time of the video presentation, wherein a playback speed effect is specified by selecting and directly modifying the line bar.

56. The computer readable medium of claim 55, wherein the GUI further comprises a set of controls for allowing a user to modify the line bar by selecting a portion of the line bar and performing a drag operation.

57. The computer readable medium of claim 55, wherein the GUI graph is defined along at least:

a) a playback-time axis that represents time during a playback; and

b) a content-time axis that represents time within the video presentation.

58. The computer readable medium GUI of claim 57, wherein a selected portion of the line bar comprises a keyframe.

59. The computer readable medium of claim 58, wherein the keyframe has (i) a playback-time value along the playback axis and (ii) a content-time value along the content-time axis, the GUI further comprising a display of a frame that appears in the video presentation at a first content-time value corresponding to a selected playback-time value.

60. The computer readable medium of claim 59, wherein the GUI further comprises a display of the frame in the video presentation that corresponds to the content-time value of the keyframe wherein a content of said display changes when the content-time value of the keyframe changes during a drag operation.

61. The computer readable medium of claim 55, wherein the line bar comprises a curve.

62. The computer readable medium of claim 61, wherein a slope of a portion of the curve determines a playback speed of a corresponding portion of a content clip.

63. The computer readable medium of claim 62, wherein a negative slope of a portion of the curve determines that the corresponding portion of the content clip is playing backward.

64. The computer readable medium GUI of claim 61, wherein the GUI further comprises a set of controls for setting a curvature of said curve.

65. A computer readable medium storing a computer program for specifying playback speed effects for playing a video clip, said computer program for execution by at least one processor, the computer program comprising sets of instructions for:

a) providing a graphical user interface (“

GUI”

) graph of a playback-time of a video presentation relative to a content-time of the video presentation as a line bar defined along a playback-time axis and a content-time axis; and

b) accepting user inputs that specify a playback speed effect for the video presentation by selecting and directly modifying a portion of the line bar through a drag operation.

66. The computer readable medium of claim 65, wherein the GUI graph is defined along at least the playback-time axis that represents time during a playback and the content-time axis that represents time within the video presentation.

67. The computer readable medium of claim 66, wherein the selected portion of the line bar comprises a keyframe.

68. The computer readable medium of claim 67, wherein the keyframe has a value along the playback-time axis and a value along the content-time axis, wherein when the keyframe is selected, the keyframe has a first content-time value, the computer program further comprising a set of instructions for:

displaying a frame that appears in the video presentation at the first content-time value when the keyframe is selected.

69. The computer readable medium of claim 68, wherein the set of instructions for accepting the user inputs comprises a set of instructions for moving the keyframe along the playback-time axis when the drag operation is along the playback-time axis.

70. The computer readable medium of claim 68, wherein the set of instructions for accepting the user inputs comprises a set of instructions for moving the keyframe along the playback-time axis when:

a) the drag operation is along the playback-time axis; and

b) a particular keyboard key is being pressed.

71. The computer readable medium of claim 68, wherein the set of instructions for accepting the user inputs comprises a set of instructions for moving the keyframe along the content-time axis when:

a) the drag operation is along the playback-time axis; and

b) a particular key on a keyboard is not being pressed.

72. The computer readable medium of claim 68, wherein the computer program further comprises a set of instructions for:

displaying the frame, in the video presentation, that corresponds to the content-time value of the keyframe when the content-time value of the keyframe changes during the drag operation.

73. The computer readable medium of claim 72, wherein a set of instructions for accepting the user inputs comprises the set of instructions for moving the keyframe along the content-time axis when the drag operation is along the content-time axis.

74. The computer readable medium of claim 72, wherein the computer program further comprises sets of instructions for displaying a graphical representation of the video presentation when the keyframe is selected, wherein the set of instructions for accepting the user inputs comprises a set of instructions for:

a) moving the keyframe along the content-time axis when the drag operation is along the content-time axis; and

b) moving the graphical representation along the playback-time axis when the drag operation is along the content-time axis.

75. The computer readable medium of claim 72, wherein the computer program further comprises a set of instructions for:

displaying a graphical representation of the video presentation when the keyframe is selected,

wherein the set of instructions for accepting the user inputs comprises a set of instructions for moving the graphical representation along the playback-time axis when the drag operation is along the playback-time axis.

76. The computer readable medium of claim 65, wherein the set of instructions for providing said GUI graph comprises a set of instructions for representing said line bar as a curve on said GUI graph.

77. The computer readable medium of claim 76, wherein the set of instructions for providing said GUI graph comprises a set of instructions for simultaneously showing said relationship at a plurality of points in the playback-time.

78. A video editing system comprising:

a) a data storage medium for providing a video clip;

b) a video editing application for providing a set of playback speed effect settings through direct modifications of a user selectable line bar on a graph of playback-time versus content-time;

c) an effects manager for;

i) receiving said set of playback speed effect settings from said video editing application;

ii) receiving said video clip from said data storage; and

iii) providing said video clip as individual frames to the video editing application at a rate based on said playback speed effect settings.

79. The video editing system of claim 78, wherein the effects manager is further for providing two sets of frames corresponding to said video clip, wherein a first of said two sets of frames is offset in time from a second set of frames.

80. The video editing system of claim 78 further comprising a frame buffer, wherein said video editing application is for blending two frames of said two sets of frames into a blended frame and sending said blended frame to said frame buffer.