RANKING KEY VIDEO FRAMES USING CAMERA FIXATION

• US 20110292229A1
• Filed: 05/25/2010
• Published: 12/01/2011
• Est. Priority Date: 05/25/2010
• Status: Active Grant

• Current Assignee: Monument Peak Ventures, LLC (Dominion Harbor Enterprises, LLC)
• Original Assignee: Intellectual Ventures Fund 83 LLC (Intellectual Ventures LLC)
• Inventors: Deever, Aaron T.
• Granted Patent: US 8,619,150 B2
• Time in Patent Office: Days
• US Class Current: 348/222.1
• CPC Class Codes: G06F 16/70 of video data
• CPC Class Codes: G06F 16/71 Indexing; Data structures therefor; Storage structures

Abstract

A method for ranking key video frames for a digital video having a time sequence of video frames, comprising using a processor to perform the steps of: analyzing the digital video to determine a camera fixation pattern; determining a set of key video frames for the digital video; and ranking the key video frames responsive to the camera fixation pattern.

Claims

1. A method for ranking key video frames for a digital video having a time sequence of video frames, comprising using a processor to perform the steps of:

a) analyzing the digital video to determine a camera fixation pattern;

b) determining a set of key video frames for the digital video; and

c) ranking the key video frames responsive to the camera fixation pattern.

2. The method of claim 1 wherein the camera fixation pattern is determined by global motion analysis.

3. The method of claim 2 wherein the global motion analysis includes analyzing data from an accelerometer.

4. The method of claim 2 wherein the global motion analysis includes analyzing the digital video using a digital motion estimation algorithm.

5. The method of claim 1 wherein the camera fixation pattern is quantized into camera fixation pattern regions.

6. The method of claim 5 wherein each camera fixation pattern region of the camera fixation pattern has an associated fixation value that is a function of the number of video frames having fixation locations in that camera fixation pattern region.

7. The method of claim 5 wherein each key video frame has a corresponding camera fixation pattern region.

8. The method of claim 6 wherein the key video frames are initially ranked by the fixation value.

9. The method of claim 8 wherein the ranking of the key video frames is subsequently adjusted to reduce redundancy.

10. The method of claim 1 wherein step b) includes

i) analyzing the digital video to determine an importance value as a function of time;

ii) forming a warped time representation of the digital video sequence by temporal relocation of the video frames responsive to the importance value as a function of time;

iii) subdividing the warped time representation into a set of equal intervals; and

iv) selecting a key video frame for each interval by analyzing the video frames within each interval.

11. The method of claim 1 further including

d) forming key video snippets corresponding to the highest-ranked key video frames;

e) assembling the key video snippets to form a video summary; and

f) storing a representation of the video summary in a processor-accessible memory.

12. A digital video camera system comprising:

an image sensor;

an optical system for forming an image of a scene onto the image sensor;

a data processing system; and

a memory system communicatively connected to the data processing system and storing instructions configured to cause the data processing system to implement a method for providing ranked key video frames for a digital video, wherein the instructions comprise;

capturing a digital video using the image sensor, the digital video having a time sequence of video frames;

analyzing the digital video to determine a camera fixation pattern;

determining a set of key video frames for the digital video; and

ranking the key video frames responsive to the camera fixation pattern.