Automated visual image editing system

• US 7,362,946 B1
• Filed: 04/05/2000
• Issued: 04/22/2008
• Est. Priority Date: 04/12/1999
• Status: Expired due to Fees

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Kowald, Julie Rae
• Primary Examiner(s): Miller; John
• Assistant Examiner(s): ONUAKU, CHRISTOPHER O
• Application Number:
  US09/543,330
• Time in Patent Office: 2,939 Days
• Field of Search: 386/52, 386 54- 55, 386/46, 386/83, 386/126, 386/95, 386/111, 386/117, 386/125, 360/13, 369/83, 348/239, 348/515, 348/231.5, 348/231.3
• US Class Current: 386/278
• CPC Class Codes: G11B 27/034 on discs G11B27/036, G11B27/038 take precedence
• CPC Class Codes: H04N 5/76 Television signal recording
• CPC Class Codes: H04N 5/765 Interface circuits between an apparatus for recording and another apparatus
• CPC Class Codes: H04N 5/772 the recording apparatus and the television camera being placed in the same enclosure
• CPC Class Codes: H04N 5/781 on disks or drums
• CPC Class Codes: H04N 5/85 on discs or drums
• CPC Class Codes: H04N 9/8205 involving the multiplexing of an additional signal and the colour video signal

Abstract

A method of editing a video sequence comprising a series of clips, in which each clip is formed by video content captured between two points in time. Duration data associated with the duration of each clip is extracted from the sequence. The duration data is processed according to at least one predetermined template of editing rules to form editing instruction data, the template indicating a plurality of predetermined edited segment durations, and the editing instruction data being configured to form output edited segments from the clips The clip is then processed (edited) according to the editing instruction data to form an output edited sequence of output edited segments. Each of the output edited segments has a duration corresponding to one of the predetermined edited segment durations with at least a portion of the clip being discarded by the processing of the clip.

Claims

1. A computer-implementable method of rhythmically editing an input video sequence to form an output edited sequence of shorter duration than the input video sequence without requiring a user to manually edit the input video sequence, the input video sequence comprising at least one input clip, each input clip being formed at least by video content captured between two points in time and thereby defining a duration of the input clip, the method comprising the steps of:

inputting the input video sequence into a computer to implement the method;

extracting duration data associated with the duration of each input clip of the input video sequence;

processing the duration data of the at least one input clip according to editing rules to form editing instruction data, the editing instruction data being configured to define output segments from the at least one input clip of the input video sequence, the editing rules indicating a sequence of the output segments and a plurality of predetermined segment durations including a first duration and a second duration, said processing of the duration data comprising;

(i) identifying, from the input video sequence, the sequence of the output segments, each of the output segment having one of the predetermined segment durations, and being derived from a single input clip of the input video sequence; and

(ii) identifying, from the input video sequence, discardable portions each having at least a predetermined third duration, the discardable portions separating the segments of the rhythmic sequence, there being at least one discardable portion identified from each input clip of the input video sequence; and

processing the at least one input clip of the input video sequence according to the editing instruction data to discard the discardable portions and to form the output edited sequence from the sequence of said segments.

2. A method according to claim 1, wherein the editing rules are contained in a user selectable template.

3. A method according to claim 1, wherein the first duration is between 1 and 8 seconds and the second duration is between 2 and 20 seconds.

4. A method according to claim 3, wherein the first duration is about 4 seconds and the second duration is about 10 seconds.

5. A method according to claim 1, wherein the output edited sequence is formed from a time sequential combination of the output segments based upon said rhythmic sequence according to the first duration and the second duration.

6. A method according to claim 5, wherein the rhythmic sequence comprises alternate first duration segments and second duration segments.

7. A method according to claim 1, wherein said discardable portions comprise an initial interval of a predetermined fourth duration identified from each input clip.

8. A method according to claim 7, wherein the fourth duration is between 0.5 and 2 seconds.

9. A method according to claim 1, wherein said discardable portions comprise an internal interval of a predetermined fifth duration identified from at least one input clip from which at least two of the output segments are to be formed, the internal interval separating portions of the input clip from which the at least two output segments are to be formed.

10. A method according to claim 9, wherein the fifth duration is between 1 and 5 seconds.

11. A method according to claim 1, wherein formation of the output segments comprises cutting a portion from at least one input clip and modifying a reproduction duration of the portion to correspond with one of the first duration and the second duration.

12. A method according to claim 11, wherein the cutting and the modifying are performed when the portion has a reproduction duration within a predetermined range of one of the first and second durations.

13. A method according to claim 12, wherein the predetermined range is from 70% to 200% of the one of the first and second durations.

14. A method according to claim 11, wherein the modifying comprises multiplying the reproduction duration of the portion by a predetermined factor and cutting the modified portion to one of the first and second durations.

15. A method according to claim 1, wherein the editing rules comprise an edited duration during which the output segments are to be reproduced and from which a number of the output segments is determined based upon the predetermined segment durations.

16. A method according to claim 1, wherein each of the plurality of predetermined segment durations is determined using a beat period of a sound track to be associated with the output edited sequence.

17. A method according to claim 1, wherein the duration data comprises data accompanying the video sequence.

18. A method according to claim 1, wherein the editing rules include incorporating at least one title matte as part of the output edited sequence.

19. A method according to claim 18, wherein the title matte is formed and incorporated according to a sub-method comprising the steps of:

examining time data associated with the duration data for each input clip to identify input clips that are associable by a predetermined time function, the associable input clips being arranged into corresponding groups of input clips;

identifying at least one of a beginning and a conclusion of each group as a title location;

for at least one title location, examining at least one of corresponding time data and further data to generate an insert title including at least a text component; and

incorporating the insert title into the output edited sequence at a corresponding title location.

20. A method according to claim 1, wherein the template is selected from a plurality of templates each comprising different combinations of said editing rules.

21. A method according to claim 1, wherein the first duration and the second duration are selected from one of:

(i) about 4 seconds and about 12 seconds respectively;

(ii) about 6 seconds and about 12 seconds respectively;

(iii) about 3 seconds and about 8 seconds respectively;

(iv) about 1 second and about 1 second respectively; and

(v) about 3 seconds and about 3 seconds respectively.

22. A computer readable medium selected from one of a hard drive, a semiconductor memory, a CD-ROM, and a floppy disk, and encoded with a computer program, wherein the program is configured to make a computer execute a method of rhythmically editing an input video sequence to form an output edited sequence of shorter duration than the input video sequence, the input video sequence comprising at least one input clip, each input clip being formed at least by video content captured between two points in time and thereby defining a duration of the input clip, the method comprising the steps of:

inputting the input video sequence;

extracting duration data associated with the duration of each input clip of the input video sequence;

processing the duration data of the at least one input clip according to editing rules to form editing instruction data, the editing instruction data being configured to define output segments from the at least one input clip, the editing rules indicating a sequence of the output segments and a plurality of predetermined segment durations including a first duration and a second duration, said processing comprising;

(i) identifying, from the input video sequence, the sequence of the output segments, each of the output segments having one of the predetermined segment durations, and being derived from a single input clip of the input video sequence; and

(ii) identifying, from the input video sequence, discardable portions each having at least a predetermined third duration, the discardable portions separating the segments of the sequence, there being at least one discardable portion identified from each input clip of the input video sequence; and

processing the at least one input clip of the input video sequence according to the editing instruction data to discard the discardable portions and to form the output edited sequence from the sequence of said segments.

23. A computer readable medium according to claim 22, wherein the editing rules are contained in a user selectable template.

24. A computer readable medium according to claim 22, wherein the first duration is between 1 and 8 seconds, the second duration is between 2 and 20 seconds, and a discardable portion associated with each said input clip comprises an initial portion of the input clip having a duration between 0.5 and 2 seconds.

25. A computer readable medium according to claim 22, wherein said discardable portions comprise an internal interval of a predetermined (fourth) duration identified from at least one input clip from which at least two of the output segments are to be formed, the internal interval separating portions of the input clip from which the at least two output segments are to be formed, the fourth duration being between 1 and 5 seconds.

26. A computer readable medium according to claim 22, wherein formation of the output segments comprises cutting a portion from at least one input clip and modifying a reproduction duration of the portion to correspond with one of the first duration and the second duration.

27. A computer readable medium according to claim 26, wherein the cutting and the modifying are performed when the portion has a reproduction duration within a predetermined range of one of the first and second durations, the predetermined range being from 70% to 200% of the one of the first and second durations.

28. A computer readable medium according to claim 26, wherein the modifying comprises expanding the reproduction duration of the portion by a predetermined factor and cutting the modified portion to one of the first and second durations.

29. A computer readable medium according to claim 22, wherein the editing rules comprise an edited duration during which the output segments are to be reproduced and from which a number of the output segments is determined based upon the plurality of predetermined segment durations.

30. A computer readable medium according to claim 22, wherein the output edited sequence is formed from a time sequential combination of the output segments based upon a predetermined cutting pattern formed using segments of the first duration and the second duration, the sequence comprising one of (a) alternate first duration segments and second duration segments and (b) a pseudo-random selection of first duration segments and second duration segments.

31. A computer readable medium according to claim 22, wherein each of the plurality of predetermined segment durations is determined using a beat period of a sound track to be associated with the output edited sequence.

32. A computer readable medium according to claim 22, wherein the duration data comprises data selected from the group consisting of:

data accompanying the input video sequence; and

data formed by analyzing the input video sequence, the analyzing comprising at least one of time analysis, image analysis, sound analysis, and motion analysis.

33. A computer readable medium according to claim 22, wherein the editing rules include incorporating at least one title matte as part of the output edited sequence, the title matte being formed and incorporated according to a sub method comprising the steps of:

examining time data associated with the duration data for each input clip to identify those of the input clips that are associable by a predetermined time function, the associable input clips being arranged into corresponding groups of input clips;

identifying at least one of a beginning and a conclusion of each group as a title location;

for at least one title location, examining at least one of corresponding time data and further data to generate an insert title including at least a text component; and

incorporating the insert title into the output edited sequence at the title location.

34. A visual image rhythmic editing system for editing an input video sequence to form an output edited sequence of shorter duration than the input video sequence without requiring a user to manually edit the input video sequence, said system comprising:

supply means for providing the input video sequence, the input video sequence comprising at least one input clip, each input clip being formed at least by video content captured between two points in time and thereby defining a duration of the input clip;

extracting means for extracting duration data associated with the duration of each input clip of the input video sequence;

processing means for processing the duration data of the at least one input clip according to editing rules to form editing instruction data, the editing instruction data being configured to define output segments from the at least one input clip, the editing rules indicating a sequence of the output segments and a plurality of predetermined segment durations including a first duration and a second duration, the processing means being operative to (i) identify, from the input video sequence, the sequence of the output segments, each of the output segments having one of the predetermined segment durations, and being derived from a single input clip of the input video sequence, and (ii) identify, from the input video sequence, discardable portions each having at least a predetermined third duration, the discardable portions separating the segments of the sequence, there being at least one discardable portion identified from each input clip of the input video sequence;

editing means for editing the at least one input clip of the input video sequence according to the editing instruction data to discard the discardable portions and to form the output edited sequence from the sequence of said segments; and

output means for outputting the output edited sequence.

35. A system according to claim 34, wherein the supply means comprises a storage arrangement configured to couple the input video sequence to the extracting means, and the output means comprises at least one of a display device by which the output edited sequence is viewable and a further storage arrangement for storing the output edited sequence.

36. A system according to claim 34, wherein the duration data comprises metadata, the extracting means forming a metadata file of the input video sequence based upon each input clip, and the metadata file forming an input to the processing means, and wherein at least the processing means comprises a computer device operable to interpret the metadata file according to the editing rules to form the editing instruction data.

37. A system according to claim 34, wherein the first duration is between 1 and 8 seconds, the second duration is between 2 and 20 seconds, and the discardable portions comprise (a) an initial interval of each input clip of between 0.5 and 2 seconds, and (b) an internal interval of a predetermined (fourth) duration identified from at least one input clip from which at least two of the output segments are to be formed, the internal interval separating portions of the input clip from which the at least two output segments are to be formed, the fourth duration being between 1 and 5 seconds.

38. A system according to claim 34, wherein the editing means comprises means for cutting a portion from at least one input clip and modifying a reproduction duration of the portion to correspond with one of the first duration and the second duration.

39. A system according to claim 38, wherein the cutting and the modifying are performed when the portion has a reproduction duration within a predetermined range of one of the first and second durations, the predetermined range being from 70% to 200% of the one of the first and second durations.

40. A system according to claim 38, wherein the modifying comprises expanding the reproduction duration of the portion by a predetermined factor and cutting the modified portion to one of the first and second durations.

41. A system according to claim 34, wherein said processing means comprises a store of said editing rules, one of said rules comprising an edited duration during which the output segments are to be reproduced and from which the processing means is configured to determine a number of the output segments based upon the first and second durations.

42. A system according to claim 34, wherein the editing means forms the output edited sequence from a time sequential combination of the segments based upon a predetermined cutting pattern formed using segments of the first duration and the second duration.

43. A system according to claim 42, wherein the sequence comprises one of (a) alternate first duration segments and second duration segments and (b) a pseudo random selection of first duration segments and second duration segments.

44. A system according to claim 34, wherein the editing rules comprise incorporating at least one title matte as part of the output edited sequence, and wherein the system further comprises means for forming and incorporating the title matte into the output edited sequence, the means for forming and incorporating comprising:

associating means for examining time data associated with the duration data for each input clip to identify input clips that are associable by a predetermined time function, the associable input clips being arranged into corresponding groups of input clips;

identifying means for identifying at least one of a beginning and a conclusion of each group as a title location;

data examining means for examining, for at least one title location, at least one of corresponding time data and further data to generate an insert title including at least a text component; and

means for incorporating the insert title into the output edited sequence at a corresponding title location.

45. A method of editing an input video sequence comprising a plurality of individual input clips to form an output edit sequence, each input clip being formed by video content captured between a corresponding commencement of recording and a corresponding cessation of recording and distinguished by associated data including at least time data related to a real time at which the input clip was recorded, the method comprising the steps of:

(a) (a1) examining the time data for each input clip to identify input clips that are associable by a predetermined time function, the associable input clips being arranged into corresponding groups of input clips, and duration data for each input clip;

(a2) providing at least one predetermined template of editing rules, the template having at least a plurality of predetermined segment durations and indicating a sequence of output segments;

(a3) processing the duration data of the at least one input clip according to the editing rules of the template to form editing instruction data, the editing instruction data being configured to define the output segments from the at least one input clip, said processing comprising (i) identifying, from the input video sequence, the sequence of said segments, each said segment of the sequence having one of the predetermined segment durations, and being derived from a single input clip of the input video sequence; and

(ii) identifying from the input video sequence, discardable portions each having at least a predetermined discardable duration, the discardable portions separating the segments of the sequence of segments, there being at least one discardable portion identified from each input clip of the input video sequence; and

(a4) processing the at least one input clip of the input video sequence according to the editing instruction data to discard the discardable portions and to form the output edited sequence from the segments;

(b) for each group of input clips, identifying from corresponding time data at least one of a beginning and a conclusion of the group as a title location;

(c) for at least one title location, examining at least one of corresponding time data and further data to generate an insert title including at least a text component; and

(d) incorporating the insert title into the output video sequence at a corresponding title location.

46. A method according to claim 45, wherein the predetermined time function comprises associating any two sequential input clips within a group when a period between a real-time conclusion of one of the sequential input clips and the real time commencement of a following input clip is less than a predetermined (first) duration.

47. A method according to claim 45, wherein the further data comprises user provided data.

48. A method according to claim 45, wherein the further data comprises generated data formed by analyzing a corresponding input clip, and step (c) comprises examining at least one of the time data and the further data to select from a rule based group of alternatives at least one title component from a title database, the at least one title component collectively forming the insert title.

49. A method according to claim 48, wherein the at least one title component is selected from the group consisting of individual words and phrases, the at least one title component being configured for selection in response to a rule based examination of at least one of the time data and the further data.

50. A method according to claim 49, wherein the title database comprises a plurality of typeset configurations applicable to the at least one title component to modify a visual impact of the insert title.

51. A method according to claim 48, wherein the title database comprises a graphical database of graphical objects configured for inclusion in the insert title.

52. A method according to claim 45, wherein the insert title comprises a matte background permitting superimposition of the insert title upon a clip.

53. A computer readable medium, selected from one of a hard drive, a semiconductor memory, a CD-ROM, and a floppy disk, and encoded with a computer program, wherein the program is configured to make a computer execute a method of editing an input video sequence comprising a plurality of individual input clips to form an output edited sequence, each input clip being formed by video content captured between a corresponding commencement of recording and a corresponding cessation of recording and distinguished by associated data including at least time data related to a real time at which the clip was recorded, the method comprising the steps of:

(a) (a1) examining the time data for each input clip to identify input clips that are associable by a predetermined time function, the associable input clips being arranged into corresponding groups of input clips, and duration data for each input clip;

(a2) providing at least one predetermined template of editing rules, the template having at least a plurality of predetermined segment durations and indicating a sequence of output segments and a plurality of predetermined segment durations including a first duration and a second duration;

(a3) processing the duration data of the at least one input clip according to the editing rules of the template to form editing instruction data, the editing instruction data being configured to define the output segments from the at least one input clip, said processing comprising (i) identifying, from the input video sequence, the sequence of said segments, each said segment of the sequence having one of the predetermined segment durations, and being derived from a single input clip of the input video sequence, and (ii) identifying, from the input video sequence, discardable portions each having at least a predetermined discardable duration, the discardable portions separating the segments of the sequence of segments, there being at least one discardable portion identified from each input clip of the input video sequence; and

(a4) processing the at least one input clip of the video sequence according to the editing instruction data to discard the discardable portions and to form the output edited sequence from the segments;

(b) for each group of input clips, identifying from corresponding time data at least one of a beginning and a conclusion of the group as a title location;

(c) for at least one title location, examining at least one of corresponding time data and further data to generate an insert title including at least a text component; and

(d) incorporating the insert title into the video sequence at a corresponding title location.

54. A computer readable medium according to claim 53, wherein the predetermined time function comprises associating any two sequential input clips within a group when a period between a real-time conclusion of one of the sequential input clips and a real-time commencement of a following clip is less than a predetermined (first) duration.

55. A method according to claim 53, wherein the further data comprises user provided data.

56. A computer readable medium according to claim 53, wherein the further data comprises generated data formed by analyzing a corresponding input clip, and step (c) comprises examining at least one of the time data and the further data to select from a rule-based group of alternatives at least one title component from a title database, the at least one title component collectively forming the insert title.

57. A computer readable medium according to claim 56, wherein the at least one title component is selected from the group consisting of individual words and phrases, the at least one title component being configured for selection in response to a rule based examination of at least one of the time data the further data.

58. A computer readable medium according to claim 57, wherein the title database comprises a plurality of typeset configurations applicable to the at least one title component to modify a visual impact of the insert title.

59. A computer readable medium according to claim 56, wherein the title database comprises a graphical database of graphical objects configured for inclusion in the insert title.

60. A computer readable medium according to claim 53, wherein the insert title comprises a matte background permitting superimposition of the insert title upon an output segment.

61. A system for editing an input video sequence comprising a plurality of individual input clips to form an output edited sequence, each input clip being formed by video content captured between a corresponding commencement of recording and a corresponding cessation of recording and distinguished by associated data including at least time data related to a real time at which the input clip was recorded, the system comprising:

associating means for examining the time data for each input clip to identify input clips that are associable by a predetermined time function, and for arranging associable input clips into corresponding groups of clips, and duration data for each input clip;

at least one predetermined template of editing rules, the template having at least a plurality of predetermined segment durations including a first duration and a second duration and indicating a sequence of output segments;

processing means for processing the duration data of the at least one input clip according to the editing rules of a selected said template to form editing instruction data, the editing instruction data being configured to define the output segments from the at least one input clip, the processing comprising (i) identifying, from the input video sequence, the sequence of said segments, each said segment of the sequence having one of the predetermined segment durations, and being derived from a single input clip of the input video sequence, and (ii) identifying, from the input video sequence, discardable portions each having at least a predetermined third duration, the discardable portions separating the segments of the rhythmic sequence, there being at least one discardable portion identified from each input clip of the input video sequence;

cutting means for cutting the at least one input clip of the input video sequence according to the edit instruction data to discard the discardable portions and to form the output edited sequence from the output segments;

identifying means for, for each group of input clips, identifying from corresponding time data at least one of a beginning and a conclusion of the group as a title location;

examining means for examining, for at least one title location, at least one of corresponding time data and further data to generate an insert title including at least a text component; and

editing means for incorporating the insert title into the output segments at a corresponding title location to form the output edited sequence.

62. A system according to claim 61, wherein input clips within each group are sequentially associable by the predetermined time function, and the predetermined time function comprises associating any two sequential input clips within a group when a period between a real-time conclusion of one clip and real-time commencement of a following input clip is less than a predetermined (fourth) duration.

63. A system according to claim 61, wherein the further data comprises user provided data.

64. A system according to claim 61, wherein the further data comprises generated data formed by analyzing a corresponding input clip, and wherein the examining means examines at least one of the time data and the further data to select from a rule-based group of alternatives at least one title component from a title database, the at least one title component collectively forming the insert title.

65. A system according to claim 64, wherein the at least one title component is selected from the group consisting of individual words and phrases, the at least one title component being configured for selection in response to a rule-based examination of at least one of the time data and the further data.

66. A system according to claim 65, wherein the title database comprises a plurality of typeset configurations applicable to the at least one title component to modify a visual impact of the insert title.

67. A system according to claim 64, wherein the title database comprises a graphical database of graphical objects configured for inclusion in the insert title.

68. A system according to claim 61, wherein the insert title comprises a matte background permitting superimposition of the insert title upon an output segment.

69. A computer-implementable method of editing an input video sequence, the video sequence comprising at least one clip, each clip being formed at least by video content captured between two points in time and thereby defining a duration of the clip, the method comprising the steps of:

extracting duration data associated with the duration of each clip of the video sequence;

providing at least one predetermined template, the template having a plurality of attributes including cutting rules comprising at least a first edited segment duration, having a predetermined duration between 1 and 8 seconds, and a second edited segment duration, having a predetermined duration between 2 and 20 seconds;

processing the duration data of the at least one clip according to the cutting rules of the template to form editing instruction data, the editing instruction data being configured to form output edited segments from the at least one clip; and

processing the at least one clip of the video sequence according to the editing instruction data to form an output edited sequence of the output edited segments, each output edited segment having a duration corresponding to one of the edited segment durations of the cutting rules of the template, with at least a portion of the at least one clip being discarded by the processing of the at least one clip.

70. A computer-implementable method of rhythmically editing an input video sequence to form an output edited sequence of shorter duration than the input video sequence without requiring a user to manually edit the input video sequence, the input video sequence comprising at least one input clip, each input clip being formed at least by video content captured between two points in time and thereby defining a duration of the input clip, the method comprising the steps of:

inputting the input video sequence into a computer to implement the method;

extracting duration data associated with the duration of each input clip of the input video sequence; and

processing the duration data of the at least one input clip according to rhythmic editing rules to form editing instruction data, the rules including at least a user selectable reproduction duration for the output edited sequence and a plurality of editing durations including a first duration and a second duration, the editing instruction data being configured to define output segments from the at least one input clip of the input video sequence, said processing of the duration data comprising;

(i) determining from the reproduction duration at least the first editing duration for defining output segments from the at least one input clip and a number of said output segments of the first editing duration to occupy the reproduction duration;

(ii) identifying, from the input video sequence, a sequence of said number of said output segments, each said output segment being derived from a single input clip of the input video sequence; and

(iii) identifying discardable portions from the input video sequence, the discardable portions having at least the second duration and separating the output segments of the rhythmic sequence, there being at least one discardable portion identified from each input clip of the input video sequence; and

processing the at least one input clip of the input video sequence according to the editing instruction data to discard the discardable portions and to form the output edited sequence from the sequence of said output segments.

71. A method according to claim 70, wherein the edited segments of the first duration are compressed in time so that the output edited sequence occupies the reproduction duration.