Method and system for segmenting videos using face detection

• US 20070091203A1
• Filed: 10/25/2005
• Published: 04/26/2007
• Est. Priority Date: 10/25/2005
• Status: Active Grant

• Current Assignee: Mitsubishi Electric Research Laboratories, Inc. (Mitsubishi Electric Corporation)
• Original Assignee: Mitsubishi Electric Research Laboratories, Inc. (Mitsubishi Electric Corporation)
• Inventors: Divakaran, Ajay, Peker, Kadir
• Granted Patent: US 7,555,149 B2
• Time in Patent Office: Days
• US Class Current: 348/415.1
• CPC Class Codes: G06F 16/739 in form of a video summary, e.g. the video summary being a video sequence, a composite still image or having synthesized frames
• CPC Class Codes: G06F 16/745 the internal structure of a single video sequence
• CPC Class Codes: G06F 16/784 the detected or recognised objects being people
• CPC Class Codes: G06T 7/215 Motion-based segmentation
• CPC Class Codes: G06V 20/40 in video content extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44
• CPC Class Codes: G11B 27/034 on discs G11B27/036, G11B27/038 take precedence
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence

Abstract

A method generates a summary of a video. Faces are detected in a plurality of frames of the video. The frames are classified according to a number of faces detected in each frame and the video is partitioned into segments according to the classifications to produce a summary of the video. For each frame classified as having a single detected face, one or more characteristics of the face is determined. The frames are labeled according to the characteristics to produce labeled clusters and the segments are partitioned into sub-segments according to the labeled clusters.

Claims

1. A method for generating a summary of a video including a plurality of frames, comprising:

detecting faces in the plurality of frames of the video;

classifying the frames according to a number of faces detected in each frame; and

partitioning the video into segments according to the classifications to produce a summary of the video.

2. The method of claim 1, wherein the segmenting further comprises:

determining, for each frame classified as having a single detected face, one or more characteristics of the face;

labeling the frames according to the characteristics;

clustering the frames according to the labels to produce labeled clusters; and

partitioning the segments having frames classified as single face frames into sub-segments according to the labeled clusters.

3. The method of claim 1, wherein the video is a compressed video.

4. The method of claim 3, wherein the detecting is performed on I-frames of the compressed video.

5. The method of claim 2, wherein the characteristics include a size of a face.

6. The method of claim 2, wherein the characteristics include a location of a face.

7. The method of claim 2, wherein the characteristics include a color histogram of a face.

8. The method of claim 2, wherein the clustering is k-means clustering.

9. The method of claim 2, wherein the clustering is performed using Gaussian mixture models.

10. The method of claim 2, further comprising:

correcting face detection errors using temporal coherence.

11. The method of claim 2, further comprising:

removing sub-segments shorter than a threshold length.

12. The method of claim 11 wherein the threshold length is in a range of 1-3 seconds.

13. A system for generating a summary of a video including a plurality of frames, comprising:

means for detecting faces in the plurality of frames of the video;

means for classifying the frames according to a number of faces detected in each frame; and

means for partitioning the video according to the classifications to produce a summary of the video.

14. The system of claim 13, further comprising:

means for determining, for each frame classified as having a single detected face, one or more characteristics of the face;

means for labeling the frames according to the characteristics;

means for clustering the frames according to the labels to produce labeled clusters; and

means for partitioning the segments having frames classified as single face frames into sub-segments according to the labeled clusters.