Interactive animation of sprites in a video production

• US 20030146915A1
• Filed: 10/11/2002
• Published: 08/07/2003
• Est. Priority Date: 10/12/2001
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Brook, John Charles, Lawther, Michael Jan, Reeve, Rupert William Galloway, Kowald, Julie Rae, Qian, Lena, Ma, Choi Chi Evelene, Magarey, Julian Frank Andrew
• Granted Patent: US 7,432,940 B2
• Time in Patent Office: Days
• US Class Current: 345/473
• CPC Class Codes: G11B 27/11 by using information not detectable on the record carrier

Abstract

A method of animating a sprite in a video production comprising a plurality of sequential video frames is disclosed. The method comprises the steps of selecting (2004) a feature, with which the sprite is to be juxtaposed, in one of said video frames, applying (2006) a feature tracking process to the video production to thereby output (2008), for a series of said plurality of video frames containing the feature, a temporal-spatial record for the feature across the plurality of video frames, and compositing (2010), with the series of said plurality of video frames, a corresponding series of instantiations of the sprite dependent upon the temporal-spatial record.

Claims

1. A method of animating a sprite in a video production comprising a plurality of video frames, said method comprising the steps of:

selecting, in one of said video frames, a feature with which the sprite is to be composited;

applying a feature tracking process to the video production to thereby output, for a series of said plurality of video frames containing the feature, a temporal-spatial record for the feature; and

compositing, dependent upon the temporal-spatial record, to each frame of the series of video frames, a corresponding instantiation of the sprite.

2. A method according to claim 1, wherein:

the temporal-spatial record comprises a series of feature coordinates, corresponding to the series of video frames; and

the compositing step composites the corresponding instantiation of the sprite dependent on the corresponding feature coordinate in said each frame.

3. A method according to claim 2, wherein the applying step comprises the sub-steps of:

searching, in a current one of the plurality of sequential video frames, for the presence of the feature;

if the feature is present;

i) determining a feature coordinate; and

ii) assigning the feature coordinate to the current frame;

if the feature is not present;

(i) updating a tracking failure attribute;

(ii) checking the attribute against a rule;

(iii) inferring a feature coordinate for the current frame and assigning the inferred feature coordinate to the current frame if the attribute complies to the rule.

4. A method according to claim 3, comprising a further step, if the feature is not present, of:

assigning a TRACKING FAILURE flag to the current frame if the attribute violates the rule.

5. A method according to claim 2;

wherein after the selecting and applying steps, and prior to the compositing step, the method comprises additional steps of;

storing the corresponding series of feature coordinates as a trajectory list referenced by the feature; and

referencing the feature to thereby retrieve the trajectory list from which the feature coordinates are provided to the compositing step.

6. A method according to claim 2;

wherein after the selecting and applying steps, and prior to the compositing step, the method comprises additional steps of;

storing the corresponding series of feature coordinates as a trajectory list referenced by a track object; and

referencing the track object to thereby retrieve the trajectory list from which the feature coordinates are provided to the compositing step.

7. A method according to claim 5, wherein:

the selecting, applying and storing steps are repeated for a number of features, each repetition yielding a corresponding trajectory list associated with a corresponding one of the features.

8. An apparatus for animating a sprite in a video production comprising a plurality of video frames, said apparatus comprising:

means for selecting, in one of said video frames, a feature with which the sprite is to be composited;

means for applying a feature tracking process to the video production to thereby output, for a series of said plurality of video frames containing the feature, a temporal-spatial record for the feature; and

means for compositing, dependent upon the temporal-spatial record, to each frame of the series of video frames, a corresponding instantiation of the sprite.

9. An apparatus according to claim 8, wherein:

the means for applying the feature tracking process is adapted to output the temporal-spatial record in the form of a series of feature coordinates, corresponding to the series of video frames; and

the means for compositing is adapted to composite the corresponding instantiation of the sprite dependent on the corresponding feature coordinate in said each frame.

10. An apparatus according to claim 8, wherein the means for applying the feature tracking process comprises:

means for searching, in a current one of the plurality of sequential video frames, for the presence of the feature; and

, in relation to the case when the feature is present;

means for determining a feature coordinate; and

means for assigning the feature coordinate to the current frame; and

, in relation to the case when the feature is not present;

means for updating a tracking failure attribute;

means for checking the attribute against a rule;

means for inferring a feature coordinate for the current frame;

means for assigning the inferred feature coordinate to the current frame if the attribute complies with the rule; and

means for assigning a TRACKING FAILURE flag to the current frame if the attribute violates the rule.

11. A method of selecting frames for printing from a production comprising video frames which include animation, the method comprising the steps of:

determining relative suitability measures for the video frames dependent upon at least one of (i) meta-data associated with the video frames, and (ii) a production template used to form the production; and

producing said frames for printing dependent upon said relative suitability measures.

12. An apparatus for selecting frames for printing from a production comprising video frames which include animation, the apparatus comprising:

means for determining relative suitability measures for the video frames dependent upon at least one of (i) meta-data associated with the video frames, and (ii) a production template used to form the production; and

means for producing said frames for printing dependent upon said relative suitability measures.

13. A method of animating a sprite in a video production, said method comprising the steps of:

selecting a sprite and a feature in a video frame of the video production in relation to which the sprite is to be animated;

applying a feature tracking process to the video production to thereby output a trajectory for the feature; and

compositing instantiations of the sprite with the video production depending upon the trajectory to thereby form a first animated production.

14. A method according to claim 13, wherein:

the trajectory end points are defined by event bounds comprising a begin frame and an end frame; and

the compositing step is performed in relation to a video frame of the video production falling between said begin and said end frames.

15. A method according to claim 13, wherein:

the trajectory end points are defined by event bounds comprising a begin frame and an end frame; and

the compositing step is performed in relation to a video frame of the video production falling in the neighbourhood of at least one of said begin and said end frames.

16. A method according to claim 13, comprising further steps of:

selecting another feature in a video frame of one of the video production and the first animated production to which the sprite is to be animated;

applying the feature tracking process to said one of the video production and the first animated production to thereby output another trajectory for said other selected feature; and

compositing instantiations of the sprite with the first animated production depending on the other trajectory to thereby form a second animated production.

17. A method according to claim 16, wherein:

the animated sprite in the first animated production has an associated animation time, and the other trajectory for the other selected feature has an associated trajectory time; and

wherein;

if the associated animation time differs from the associated trajectory time for the other trajectory, then the compositing depending on the other trajectory comprises the steps of;

scaling the other trajectory so that the associated trajectory time conforms to the associated animation time; and

compositing instantiations of the sprite with the first animated production depending on the scaled other trajectory to thereby form a second animated production.

18. A computer program for directing a processor to execute a procedure for animating a sprite in a video production comprising a plurality of sequential video frames, said program comprising:

code for selecting, in one of said video frames, a feature with which the sprite is to be composited;

code for applying a feature tracking process to the video production to thereby output, for a series of said plurality of video frames containing the feature, a temporal-spatial record for the feature; and

code for compositing, dependent upon the temporal-spatial record, to each frame of the series of video frames, a corresponding instantiation of the sprite.

19. A computer program for directing a processor to execute a procedure for animating a sprite in a video production comprising a plurality of sequential video frames, said program comprising:

code for selecting a sprite and a feature in a video frame of the video production in relation to which the sprite is to be animated;

code for applying a feature tracking process to the video production to thereby output a trajectory for the feature; and

code for compositing instantiations of the sprite with the video production depending upon the trajectory to thereby form a first animated production.

20. An apparatus for animating a sprite in a video production comprising a plurality of sequential video frames, said apparatus comprising:

a memory for storing a program; and

a processor for executing the program, said program comprising;

code for selecting, in one of said video frames, a feature with which the sprite is to be composited;

code for applying a feature tracking process to the video production to thereby output, for a series of said plurality of video frames containing the feature, a temporal-spatial record for the feature; and

code for compositing, dependent upon the temporal-spatial record, to each frame of the series of video frames, a corresponding instantiation of the sprite.

21. A Graphical User Interface (GUI) system for editing a production having a plurality of media clips, said GUI system comprising:

(i) a clip editing process;

(ii) a GUI comprising;

a graphical representation of a selected one of said plurality of media clips, wherein manipulation of said graphical representation enables the clip editing process to be applied to the selected media clip; and

a presentation means configured to present said one media clip and any editing changes made thereto; and

(iii) a production editing process which is applied to said production to thereby form, dependent upon the selected media clip and said any editing changes made thereto, an edited production, wherein the application of the production editing process is synchronously dependent upon the application of the clip editing process.

23. A GUI system according to either one of claims 21 and 22, wherein the GUI further comprises a graphical representation of the production, said graphical representation being configured to reflect editing changes made to the production.

24. A GUI system according to claim 23, wherein said graphical representation of the production is juxtaposed with the graphical representation of the selected one of the plurality of media clips.

22. A Graphical User Interface (GUI) system for editing a production having a plurality of media clips, said GUI system comprising:

(i) a clip editing process;

(ii) a GUI comprising;

a graphical representation of a selected one of said plurality of media clips, wherein manipulation of said graphical representation enables the clip editing process to be applied to the selected media clip; and

a presentation means configured to present said one media clip and any editing changes made thereto; and

(iii) a production editing process which is applied to an EDL of said production to thereby form, dependent upon the selected media clip and said any editing changes made thereto, an edited EDL, wherein the application of the production editing process is synchronously dependent upon the application of the clip editing process.

25. A method of editing, using a Graphical User Interface (GUI) system, a production having a plurality of media clips said method comprising the steps of:

selecting one of said plurality of media clips;

manipulating, using a GUI, a graphical representation of said selected media clip to thereby apply a clip editing process to the selected media clip;

presenting said one media clip and any editing changes made thereto using a presentation means; and

applying, synchronously with said application of the clip editing process, a production editing process to said production to thereby form, dependent upon the selected media clip and said any editing changes made thereto, an edited production.

26. A method of editing, using a Graphical User Interface (GUI) system, a production having a plurality of media clips said method comprising the steps of:

selecting one of said plurality of media clips;

manipulating, using a GUI, a graphical representation of said selected media clip to thereby apply a clip editing process to the selected media clip;

presenting said one media clip and any editing changes made thereto using a presentation means; and

applying, synchronously with said application of the clip editing process, a production editing process to an EDL of said production to thereby form, dependent upon the selected media clip and said any editing changes made thereto, an edited EDL.

27. A computer program product including a computer readable medium having recorded thereon a computer program for directing a computer to execute a method for editing, using a Graphical User Interface (GUI) system, a production having a plurality of media clips said program comprising:

code for selecting one of said plurality of media clips;

code for manipulating, using a GUI, a graphical representation of said selected media clip to thereby apply a clip editing process to the selected media clip;

code for presenting said one media clip and any editing changes made thereto using a presentation means; and

code for applying, synchronously with said application of the clip editing process, a production editing process to an EDL of said production to thereby form, dependent upon the selected media clip and said any editing changes made thereto, an edited EDL.

28. A computer program for directing a computer to execute a method for editing, using a Graphical User Interface (GUI) system, a production having a plurality of media clips said program comprising:

code for selecting one of said plurality of media clips;

code for manipulating, using a GUI, a graphical representation of said selected media clip to thereby apply a clip editing process to the selected media clip;

code for presenting said one media clip and any editing changes made thereto using a presentation means; and

code for applying, synchronously with said application of the clip editing process, a production editing process to an EDL of said production to thereby form, dependent upon the selected media clip and said any editing changes made thereto, an edited EDL.