Extracting key frames from a video sequence

• US 7,046,731 B2
• Filed: 01/08/2001
• Issued: 05/16/2006
• Est. Priority Date: 01/31/2000
• Status: Expired due to Fees

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Wu, Jing, Lennon, Alison Joan
• Primary Examiner(s): LEE, Y YOUNG
• Application Number:
  US09/755,068
• Publication Number:
  US 20010016007A1
• Time in Patent Office: 1,954 Days
• Field of Search: 375/240.16, 348/699, 348/700, 348/701
• US Class Current: 375/240.16
• CPC Class Codes: G06F 16/71 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/786 using motion, e.g. object motion or camera motion
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence
• CPC Class Codes: H04N 19/142 Detection of scene cut or scene change
• CPC Class Codes: H04N 19/179 the unit being a scene or a shot
• CPC Class Codes: H04N 19/87 involving scene cut or scene change detection in combination with video compression
• CPC Class Codes: H04N 5/147 Scene change detection

Abstract

A method of extracting key frames from a video sequence, which video sequence comprises MPEG compressed video data having block motion vectors. The method initially partially decompresses (202) the MPEG compressed video data to obtain block motion vectors and converts (204) the block motion vectors to forward block motion vectors. The method then generates (206) global motion signals and generates (306) dominant global direction clusters. The method then selects (402, 404, 406) potential key frames of the video sequence using the generated dominant global direction clusters. The method lastly decompresses (408) the selected key frames to obtain the extracted key frames.

Claims

1. A method of extracting key frames from a video sequence, wherein the video sequence comprises compressed video data having motion vectors;

the method comprising the steps of;

generating, for each frame of a plurality of frames of the video sequence, global motion signals based on the motion vectors, wherein the global motion signals are representative of a global motion of the frame;

determining for each frame of the plurality of frames of the video sequence, a dominant direction of the global motion of the frame from the generated global motion signals associated with the frame;

clustering the plurality of frames according to their determined dominant directions;

selecting key frames from the clusters of frames; and

decompressing the selected key frames to obtain the extracted key frames.

2. A method as claimed in claim 1, wherein said selecting step comprises selecting key frames from the clusters using a set of predefined rules.

3. A method as claimed in claim 1, wherein said selecting step comprises the substeps of:

selecting potential key frames of the video sequence from the clusters using a first set of predefined rules; and

removing redundant key frames from the selected potential key frames using a second set of predefined rules resulting in the selected key frames.

4. A method as claimed in claim 1, wherein said selecting step comprises the substeps of:

selecting potential key frames of the video sequence from the clusters using a first set of predefined rules;

removing redundant key frames from the selected potential key frames using a second set of predefined heuristic rules resulting in a set of selected potential key frames; and

removing similar and/or repeated key frames from the set of selected potential key frames using a color histogram technique resulting in the selected key frames.

5. A method as claimed in claim 1, wherein said step of generating global motion signals comprises generating, for each frame of the plurality of frames of the video sequence, a pan global motion signal, a zoom global motion signal, and a tilt global motion signal.

6. A method as claimed in claim 1, wherein the dominant global direction is selected from the group consisting of a pan left, pan right, tilt up, tilt down, zoom in, zoom out and global still direction.

7. A method as claimed in claim 1, wherein said determining step comprises, for each frame, the sub-steps of:

generating discrete global motion signals from the generated global motion signals;

removing noise from the generated discrete global motion signals to produce noise reduced discrete global motion signals; and

determining the dominant direction of the global motion of the frame from the noise reduced discrete global motion signals associated with the frame.

8. A method according to claim 1, wherein the motion vectors are block motion vectors, and said step of generating global motion signals comprises the steps of:

decompressing the compressed video data to obtain the block motion vectors;

converting the block motion vectors to forward block motion vectors;

generating global motion signals based on the forward block motion vectors.

9. A method according to claim 8, wherein the compressed video data is MPEG compressed video data.

10. Apparatus for extracting key frames from a video sequence, wherein the video sequence comprises compressed video data having motion vectors;

the apparatus comprising;

means for generating, for each frame of a plurality of frames of the video sequence, global motion signals based on the motion vectors, wherein the global motion signals are representative of a global motion of the frame;

means for determining, for each frame of the plurality of frames of the video sequence, a dominant direction of the global motion of the frame from the generated global motion signals associated with the frame;

means for clustering the plurality of frames according to their determined dominant directions;

means for selecting key frames from the clusters of frames; and

means for decompressing the selected key frames to obtain the extracted key frames.

11. A computer readable medium comprising a computer program for extracting key frames from a video sequence, wherein the video sequence comprises compressed video data having motion vectors;

the computer program comprising;

code for generating, for each frame of a plurality of frames of the video sequence, global motion signals based on the motion vectors, wherein the global motion signals are representative of a global motion of the frame;

code for determining, for each frame of the plurality of frames of the video sequence, a dominant direction of the global motion of the frame from the generated global motion signals;

code for clustering the plurality of frames according to their determined dominant directions;

code for selecting key frames from the clusters of frames; and

code for decompressing the selected key frames to obtain the extracted key frames.

12. An apparatus for extracting key frames from a video sequence, said apparatus comprising:

a storage unit for storing the video sequence comprising compressed video data having motion signals;

a processor coupled to the storage unit and adapted to;

generate, for each frame of a plurality of frames of the video sequence, global motion signals based on the motion vectors, wherein the global motion signals are representative of a global motion of the frame;

determine, for each frame of the plurality of frames of the video sequence, a dominant direction of the global motion of the frame from the generated global motion signals associated with the frame;

cluster the plurality of frames according to their determined dominant directions;

select key frames from the clusters of frames; and

decompress the selected frames to obtain the extracted key frames.