Method for hierarchical summarization and browsing of digital video

• US 5,995,095 A
• Filed: 05/21/1999
• Issued: 11/30/1999
• Est. Priority Date: 12/19/1997
• Status: Expired due to Term

• Current Assignee: Sharp Kabushiki Kaisha (Hon Hai Precision Industry Co., Ltd.)
• Original Assignee: Sharp Laboratories of America Incorporated (Hon Hai Precision Industry Co., Ltd.)
• Inventors: Ratakonda, Krishna
• Primary Examiner(s): Bayerl, Raymond J.
• Assistant Examiner(s): LUU, SY D
• Application Number:
  US09/316,599
• Time in Patent Office: 193 Days
• Field of Search: 345/328, 345/348, 345/349, 383/4, 383/52, 383/64, 382/225
• US Class Current: 715/255
• CPC Class Codes: G06F 16/739 in form of a video summary, e.g. the video summary being a video sequence, a composite still image or having synthesized frames
• CPC Class Codes: G06F 16/785 using colour or luminescence
• CPC Class Codes: G06F 16/786 using motion, e.g. object motion or camera motion
• CPC Class Codes: G11B 27/034 on discs G11B27/036, G11B27/038 take precedence
• CPC Class Codes: G11B 27/102 Programmed access in sequence to addressed parts of tracks of operating record carriers access by moving the head G11B3/08, G11B5/54, G11B7/085, G11B21/022; by moving the record carrier G11B15/005, G11B17/005, by driving of both record carrier and head G11B15/1816
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence
• CPC Class Codes: G11B 27/326 used signal is a video-frame or a video-field (P.I.P.)
• CPC Class Codes: H04N 5/147 Scene change detection

Abstract

A method of hierarchical digital video summarization and browsing includes inputting a digital video signal for a digital video sequence and generating a hierarchical summary based on keyframes of the video sequence. Additional steps may include computing histograms for the digital video sequence; detecting shot boundaries within the digital video sequence; determining the number of keyframes to be allocated within each shot; locating the actual position of each keyframe within each shot; identifying keyframe locations by the largest consecutive difference criteria; pruning keyframes for an shot without meaningful action; extracting keyframes efficiently in the case of compressed video; and browsing the shots using the hierarchical keyframe summary.

Claims

1. A method of hierarchical digital video summarization and browsing comprising:

inputting a digital video signal for a digital video sequence; and

generating a hierarchical keyframe summary;

developing at least one coarser level summary from a given keyframe summary wherein said developing includes statistical clustering using a pairwise K-means clustering algorithm, of histogram vectors of keyframes.

2. The method of claim 1 where only those keyframes that are consecutive in time are allowed to be included in the same cluster.

3. The method of claim 1 which includes selecting keyframes to represent keyframe clusters and choosing keyframes as those frames within the clusters whose histogram vectors are closest to the centroid vectors of the clusters.

4. The method of claim 1 wherein said selecting includes selecting the keyframes in the second and existing subsequent clusters on the basis of largest difference-from-the-previous-keyframe criterion.