DETERMINING KEY VIDEO SNIPPETS USING SELECTION CRITERIA

• US 20110293250A1
• Filed: 05/25/2010
• Published: 12/01/2011
• Est. Priority Date: 05/25/2010
• Status: Active Grant

• Current Assignee: Monument Peak Ventures, LLC (Dominion Harbor Enterprises, LLC)
• Original Assignee: Intellectual Ventures Fund 83 LLC (Intellectual Ventures LLC)
• Inventors: Deever, Aaron T.
• Granted Patent: US 8,605,221 B2
• Time in Patent Office: Days
• US Class Current: 386/290
• CPC Class Codes: G06F 16/739 in form of a video summary, e.g. the video summary being a video sequence, a composite still image or having synthesized frames
• CPC Class Codes: G06F 16/786 using motion, e.g. object motion or camera motion
• CPC Class Codes: G06V 20/49 Segmenting video sequences, i.e. computational techniques such as parsing or cutting the sequence, low-level clustering or determining units such as shots or scenes
• CPC Class Codes: H04N 5/275 Generation of keying signals

Abstract

A method for forming a video summary for a digital video having a time sequence of video frames, comprising using a processor to perform the steps of: determining a set of key video frames for the digital video; ranking the key video frames responsive to a predefined criterion; forming key video snippets corresponding to one or more highest-ranked key video frames according to a predefined set of criteria, including a criterion which specifies the total time duration for the video summary and a criterion which specifies the minimum time duration for each of the key video snippets; assembling the key video snippets to form a video summary; and storing a representation of the video summary in a processor-accessible memory.

Claims

1. A method for forming a video summary for a digital video having a time sequence of video frames, comprising using a processor to perform the steps of

a) determining a set of key video frames for the digital video;

b) ranking the key video frames responsive to a predefined criterion;

c) forming key video snippets corresponding to one or more highest-ranked key video frames according to a predefined set of criteria, including a criterion which specifies the total time duration for the video summary and a criterion which specifies the minimum time duration for each of the key video snippets;

d) assembling the key video snippets to form a video summary; and

e) storing a representation of the video summary in a processor-accessible memory.

2. The method of claim 1 wherein each highest-ranked key video frame is included in a key video snippet.

3. The method of claim 1 wherein the digital video is analyzed to determine an importance value as a function of time, and wherein the start and end times for a key video snippet are determined responsive to a criterion which encourages the inclusion of video frames having high importance values.

4. The method of claim 1 wherein the digital video is analyzed to determine a level of audio activity as a function of time, and wherein the start and end times for a key video snippet are determined responsive to a criterion which encourages selecting start and end times corresponding to low levels of audio activity.

5. The method of claim 1 wherein the key video snippets are formed according to a criterion which discourages the formation of key video snippets that have similar content.

6. The method of claim 1 wherein the total time duration for the video summary is user-specified.

7. The method of claim 1 wherein the key video snippets corresponding to highest-ranked key video frames that are separated by time intervals less than a specified threshold are merged to form a single key video snippet.

8. The method of claim 1 wherein the key video snippets are formed according to a criterion which favors starting a key video snippet at the beginning of an encoding group of video frames and ending a key video snippet at the end of an encoding group of video frames.

9. The method of claim 1 wherein the step of determining a set of key video frames for the digital video further includes:

i) analyzing the digital video to determine an importance value as a function of time;

ii) forming a warped time representation by temporal relocation of the video frames responsive to the importance value as a function of time;

iii) subdividing the warped time representation into a set of equal intervals; and

iv) selecting a key video frame for each interval by analyzing the video frames within each interval.

10. The method of claim 1 wherein the step of ranking the key video frames responsive to a predefined criterion further includes:

i) analyzing the digital video to determine a camera fixation pattern;

ii) ranking the key frames responsive to the camera fixation pattern.

11. A digital video camera system comprising:

an image sensor;

an optical system for forming an image of a scene onto the image sensor;

a data processing system; and

a memory system communicatively connected to the data processing system and storing instructions configured to cause the data processing system to implement a method for forming a video summary for a digital video, wherein the instructions comprise;

capturing a digital video using the image sensor, the digital video having a time sequence of video frames;

determining a set of key video frames for the digital video;

ranking the key video frames responsive to a predefined criterion;

forming key video snippets corresponding to one or more highest-ranked key video frames according to a predefined set of criteria, including a criterion which specifies the total time duration for the video summary and a criterion which specifies the minimum time duration for each of the key video snippets;

assembling the key video snippets to form a video summary; and

storing a representation of the video summary in a processor-accessible memory.