Method of selecting key-frames from a video sequence
First Claim
Patent Images
1. A method of selecting key-frames from a video sequence to produce a subset of key-frames for indexing, archiving, browsing and/or searching purposes, comprising the following steps:
- (a) producing an initial set of frames from said video sequence;
(b) introducing each frame of said initial set sequentially into a 3-frame buffer to include a middle frame M, its preceding frame L, and its succeeding frame R;
(c) testing for redundancy each M frame with respect to its preceding L frame and its succeeding R frame; and
(d) selecting as a key-frame for inclusion in the subset those M frames tested to be non-redundant.
4 Assignments
0 Petitions
Accused Products
Abstract
A method of selecting key-frames (230) from a video sequence (210, 215) by comparing each frame in the video sequence with respect to its preceding and subsequent key-frames for redundancy where the comparison involves region and motion analysis. The video sequence is optionally pre-processed to detect graphic overlay. The key-frame set is optionally post-processed (250) to optimize the resulting set for face or other object recognition.
-
Citations
26 Claims
-
1. A method of selecting key-frames from a video sequence to produce a subset of key-frames for indexing, archiving, browsing and/or searching purposes, comprising the following steps:
-
(a) producing an initial set of frames from said video sequence; (b) introducing each frame of said initial set sequentially into a 3-frame buffer to include a middle frame M, its preceding frame L, and its succeeding frame R; (c) testing for redundancy each M frame with respect to its preceding L frame and its succeeding R frame; and (d) selecting as a key-frame for inclusion in the subset those M frames tested to be non-redundant. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12)
-
-
13. A method of selecting key-frames from an initial set of frames to produce a subset of key-frames for indexing, archiving, browsing and/or searching purposes, comprising:
- testing for redundancy each frame of the initial set with respect to at least its preceding frame by;
representing the tested frame as a combination of geometric transformations applied to selected regions in the preceding frame; subtracting the geometric transformation of the preceding frame from the tested frame to produce difference images; and determining from the difference images whether or not the tested frame is redundant, wherein said redundancy test is made by; iteratively identifying motion entities having relative motion between the tested frame and its preceding frame; utilizing the identified motion entities for identifying regions in the tested frame that can be transformed from the preceding frame into the tested frame; transforming said identified regions in the tested frame to the corresponding regions in the preceding frame; measuring the differences between said identified regions and said corresponding transformed regions in the tested frames to produce difference images; measuring the areas of the difference images by counting the number of pixels of an intensity value above a predetermined threshold; and discarding the tested frame when said measured areas are less than a prespecified area threshold. - View Dependent Claims (14)
- testing for redundancy each frame of the initial set with respect to at least its preceding frame by;
-
15. A method of selecting key-frames from a video sequence, which includes overlayed frames having graphic overlays superimposed on live video content, and non-overlayed frames having only live video content, said method comprising:
-
masking out from said overlayed frames the superimposed graphic overlays; testing for redundancy the live video content of neighboring overlayed frames and non-overlayed frames; selecting as key-frames the overlayed frames and non-overlayed frames whose live video contents were tested to be non-redundant, selecting at least one overlayed frame as an overlay key-frame; and merging each selected overlay key-frame with the key-frames selected on live video content.
-
-
16. A method of selecting key-frames from a video sequence for inclusion in a subset of key-frames, comprising:
-
detecting at least one predetermined image in the video sequence; tracking said at least one image through the video sequence to produce a series of image frames; computing at least one image-based metric for said image from said series of image frames; and selecting a key-frame from said series of image frames based on said at least one image-based metric, wherein said at least one image-based metric includes an image self-similarity index and is used to select a plurality of interim key-frames. - View Dependent Claims (17, 18)
-
-
19. A method of selecting key-frames from a video sequence, comprising:
-
selecting one subset of key-frames which represent at least one feature of the video sequence; selecting at least one other subset of key-frames which represent at least one other feature of the video sequence; and combining said at least two subsets of key-frames to produce a combined set of key-frames, wherein said combining of the at least two subsets of key-frame comprises merging said subsets of key-frames to produce said combined set of key-frames; testing pairs of adjacent key-frames in said combined set of key-frames for redundancy; and discarding one of the pair of adjacent key-frames where redundancy is found to be present in a tested pair. - View Dependent Claims (20, 21, 22, 23)
-
-
24. A method of processing a video sequence of frames including graphic overlayed frames having a graphic overlay super imposed on live-video content, comprising:
-
detecting regions of the graphic overlay in said sequence; generating a mask image of said graphics overlay; utilizing said mask image for separating said graphics overlay from at least one overlayed frame in the sequence for also generating thereby a live video content image in addition to said mask image; and selecting at least one key frame from said video sequence based on analysis of said live video content. - View Dependent Claims (25)
-
-
26. A method of selecting key-frames from a video sequence for inclusion in a subset, comprising:
-
detecting at least two frames in the video sequence having a predetermined face image; applying an image-quality metric to said detected frames; and selecting as a key-frame, for inclusion in said subset, the detected frame having the higher image-quality metric; applying prior to said step of applying an image-quality metric an image-similarity metric to select a plurality of interim key-frames; and applying the image-quality metric to said interim key-frames to select the key-frame to be included in said subset.
-
Specification