Visual indexing system

• US 6,125,229 A
• Filed: 06/02/1997
• Issued: 09/26/2000
• Est. Priority Date: 06/02/1997
• Status: Expired due to Fees

• Current Assignee: US Philips Corporation (Koninklijke Philips N.V.)
• Original Assignee: Philips Electronics North America Corporation (Koninklijke Philips N.V.)
• Inventors: McGee, Thomas, Dimitrova, Nevenka, Elenbaas, Jan Hermanus
• Primary Examiner(s): Chevalier, Robert
• Application Number:
  US08/867,145
• Time in Patent Office: 1,212 Days
• Field of Search: 386/52, 386/55, 386/4, 386/95, 386/69, 386/68, 386/83, 386/6, 360/13
• US Class Current: 386/243
• CPC Class Codes: G06F 16/739 in form of a video summary, e.g. the video summary being a video sequence, a composite still image or having synthesized frames
• CPC Class Codes: G06F 16/745 the internal structure of a single video sequence
• CPC Class Codes: G06F 16/7864 using domain-transform features, e.g. DCT or wavelet transform coefficients
• CPC Class Codes: G11B 2220/2562 DVDs [digital versatile discs]; Digital video discs; MMCDs; HDCDs
• CPC Class Codes: G11B 2220/65 wherein solid state memory is used for storing indexing information or metadata
• CPC Class Codes: G11B 2220/90 Tape-like record carriers
• CPC Class Codes: G11B 27/102 Programmed access in sequence to addressed parts of tracks of operating record carriers access by moving the head G11B3/08, G11B5/54, G11B7/085, G11B21/022; by moving the record carrier G11B15/005, G11B17/005, by driving of both record carrier and head G11B15/1816
• CPC Class Codes: G11B 27/11 by using information not detectable on the record carrier
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence
• CPC Class Codes: G11B 27/3027 used signal is digitally coded
• CPC Class Codes: G11B 27/326 used signal is a video-frame or a video-field (P.I.P.)
• CPC Class Codes: H04N 19/48 using compressed domain processing techniques other than decoding, e.g. modification of transform coefficients, variable length coding [VLC] data or run-length data motion estimation in a transform domain H04N19/547; processing of decoded motion vectors H04N19/513
• CPC Class Codes: H04N 19/90 using coding techniques not provided for in groups H04N19/10-H04N19/85, e.g. fractals
• CPC Class Codes: H04N 5/147 Scene change detection

Abstract

A video indexing system analyzes contents of source video and develops a visual table of contents using selected images. The source video is analyzed to detect video cuts from one scene to another, and static scenes. Keyframes are selected for each significant scene. A keyframe filtering process filters out less desired frames including, for example, unicolor frames, or those frames having a same object as a primary focus or one primary focuses. A visual index is created from those frames remaining after the keyframe filtering and stored for retrieval. The visual index may be retrieved by a user who may then display the visual index on a display. The user may select one of the frames displayed in the visual index and the source video may be manually (by the user) or automatically advanced to that frame of the source video. Additionally, a user may print the visual index.

Claims

1. A system for creating a visual index for source video, comprising:

a frame grabber for extracting frames from the source video;

a DCT macroblock creator for converting the frames into macroblocks of DCT coefficients;

a significant scene processor for detecting significant scenes and for selecting a keyframe for each detected significant scene, based on the macroblocks of DCT coefficients;

a keyframe filterer for filtering of the selected keyframes to create a visual index; and

a storage medium for storing the visual index.

2. A system as recited in claim 1, wherein said significant scene processor detects static scenes.

3. A system as recited in claim 1, wherein said keyframe filterer filters out unicolor frames.

4. A system as recited in claim 1, wherein said keyframe filterer filters out key frames which have a same object as a primary focus or one of the primary focuses of an image.

14. The system of claim 4, wherein the keyframes filtered out nevertheless exhibit other indicia of scene changes with respect to retained keyframes.

5. A system for creating a visual index for source video comprising:

a frame grabber for extracting frames from the source video;

a DCT macroblock creator for converting the frames into macroblocks of DCT coefficients;

a keyframe filterer for filtering the frames, based on the macroblocks of DCT coefficients, to create a visual index; and

a storage medium for storing the visual index,

further comprising a significant scene processor adapted to detect static scenes.

6. A system for creating a visual index for source video, comprising:

a frame grabber for extracting frames from the source video;

a DCT macroblock creator for converting the frames into macroblocks of DCT coefficients;

a keyframe filterer for filtering the frames, based on the macroblocks of DCT coefficients, to create a visual index; and

a storage medium for storing the visual index,

wherein said keyframe filterer filters out unicolor frames.

7. A system for creating a visual index for source video, comprising:

a frame grabber for extracting frames from the source video;

a DCT macroblock creator for converting the frames into macroblocks of DCT coefficients;

a keyframe filterer for filtering the frames, based on the macroblocks of DCT coefficients, to create a visual index; and

a storage medium for storing the visual index,

wherein said keyframe filterer filters out keyframes which have a same object as a primary focus or one of the primary focuses of an image.

15. The system of claim 7, wherein the keyframes filtered out nevertheless exhibit other indicia of scene changes with respect to retained keyframes.

8. A system for creating a visual index for compressed source video, comprising:

a decoder for converting the compressed source video into frames and DCT macroblocks;

a significant scene processor for detecting significant scenes and selecting keyframes, based on the frames and the DCT macroblocks;

a keyframe filterer for filtering the keyframes, based on the macroblocks of DCT coefficients, to create a visual index; and

a storage medium for storing the visual index.

9. A system as recited in claim 8, wherein said significant scene processor detects static scenes.

10. A system as recited in claim 8, wherein said keyframe filterer filters out unicolor frames.

11. A system as recited in claim 8, wherein said keyframe filterer filters out key frames which have a same object as a primary focus or one of the primary focuses of an image.

16. The system of claim 11, wherein the keyframes filtered out nevertheless exhibit other indicia of scene changes with respect to retained keyframes.

12. A system for video archival and retrieval, comprising:

video archival apparatus for creating a visual index of key frames based on stored or concurrently recorded source video; and

video retrieval apparatus for retrieving the visual index for at least one of display on a display, manual or automatic forwarding to a key frame, and printing of the visual index,

wherein video archival apparatus comprises;

a significant scene processor for detecting significant scene and selecting keyframes to represent the significant scenes; and

a keyframe filterer for filtering the keyframes to create the visual index.

13. A system as recited in claim 12, wherein:

said significant scene processor uses representative values to detect significant scenes and select keyframes; and

said keyframe filterer uses representative values to filter the keyframes and create the visual index.

17. A method for creating a visual index for source video comprising:

extracting frames from the source video;

detecting significant scenes and a respective keyframe for each detected significant scene;

filtering out some keyframes that are associated with significant scenes to create a visual index;

embodying the visual index in a machine readable medium.

18. The method of claim 17,

further comprising, after extracting the keyframes from the source video, converting the key frames into macroblocks of DCT coefficients;

and

wherein the detecting is based on the macroblocks.

19. The method of claim 17, wherein some of the keyframes in the visual index represent scene portions that are static with respect to respective immediately previous keyframes in the visual index.

20. The method of claim 17, wherein filtering includes filtering out unicolor frames.

21. The method of claim 17, wherein filtering includes filtering out keyframes representing significantly different scenes, when those keyframes have a same object of primary focus or as one of the primary focuses of an image.