Method and apparatus for summarizing and indexing the contents of an audio-visual presentation

• US 7,372,991 B2
• Filed: 09/26/2003
• Issued: 05/13/2008
• Est. Priority Date: 09/26/2003
• Status: Expired due to Fees

• Current Assignee: Seiko Epson Corporation (Seiko Group)
• Original Assignee: Seiko Epson Corporation (Seiko Group)
• Inventors: Chen, Jau-Yuen, Chen, William
• Primary Examiner(s): Saadat; Cameron
• Application Number:
  US10/672,436
• Publication Number:
  US 20050078868A1
• Time in Patent Office: 1,691 Days
• Field of Search: 382/168, 382/171, 382/173, 382/305
• US Class Current: 382/173
• CPC Class Codes: G06F 16/739 in form of a video summary, e.g. the video summary being a video sequence, a composite still image or having synthesized frames
• CPC Class Codes: G09B 5/065 Combinations of audio and video presentations, e.g. videotapes, videodiscs, television systems

Abstract

A method for creating a summary of an audiovisual presentation initiates with segmenting a frame of the audiovisual presentation. Then, a slide region of the segmented frame is identified. Next, a histogram representing lines in the slide region is generated. Next, moving regions associated with successive frames from the histogram are suppressed. A computer readable medium, a system for providing a real time summarization of a meeting and an integrated circuit are also provided.

Claims

1. A method for creating a summary of an audiovisual presentation, comprising method operations of:

segmenting a frame of the audiovisual presentation;

identifying a slide region of the segmented frame;

generating a histogram representing lines in the slide region; and

suppressing moving regions associated with successive frames from the histogram, the suppressing including,

assigning a foreground silhouette a most recent timestamp; and

generating a motion mask, wherein edges located within the motion mask are excluded from the histogram.

2. The method of claim 1, further comprising:

generating a histogram from a stored slide associated with the slide region; and

matching the histogram representing lines in the slide region with the histogram from the stored slide.

3. The method of claim 1, wherein the method operation of identifying a slide region of the segmented frame includes,

determining a shape ratio of the slide region.

4. The method of claim 1, wherein the method operation of generating a histogram representing lines in the slide region includes,

detecting edges of text and figures of the slide region;

and

transforming data representing the edges into a parameter space.

5. The method of claim 1, wherein the method operation of suppressing moving regions associated with successive frames from the histogram includes,

generating the silhouette through differences of luminance channels between frames.

6. The method of claim 1, wherein the method operation of segmenting a frame of the audiovisual presentation includes,

comparing each pixel value of the frame with neighbor pixel values in causal order.

7. The method of claim 6, wherein the neighbor pixel values include five neighbor pixel values.

8. The method of claim 7, wherein the five neighbor pixel values include two previous adjacent pixel values from the frame, a corresponding pixel value from a previous frame, and two adjacent pixel values from the previous frame.

9. A computer readable medium having program instructions for creating a summary of an audiovisual presentation, comprising:

program instructions for segmenting a frame of the audiovisual presentation;

program instructions for identifying a slide region of the segmented frame;

program instructions for generating a histogram representing lines in the slide region; and

program instructions for suppressing moving regions associated with successive frames from the histogram the program instructions for suppressing including,

assigning a foreground silhouette a most recent timestamp; and

generating a motion mask, wherein edges located within the motion mask are excluded from the histogram.

10. The computer readable medium of claim 9, further comprising:

program instructions for generating a histogram from a stored slide associated with the slide region; and

program instructions for matching the histogram representing lines in the slide region with the histogram from the stored slide.

11. The computer readable medium of claim 9, wherein the program instructions for identifying a slide region of the segmented frame includes,

program instructions for determining a shape ratio of the slide region.

12. The computer readable medium of claim 9, wherein the program instructions for generating a histogram representing lines in the slide region includes,

program instructions for detecting edges of text and figures of the slide region;

and

program instructions for transforming data representing the edges into a parameter space.

13. The computer readable medium of claim 9, wherein the program instructions for suppressing moving regions associated with successive frames from the histogram includes,

program instructions for generating the silhouette through differences of luminance channels between frames.

14. A method for creating a summary of an audiovisual presentation, comprising method operations of:

segmenting a frame of the audiovisual presentation, the segmenting including;

comparing each pixel value of the frame with a plurality of neighbor pixel values, the plurality of neighbor pixel values including pixel values from the frame and from a previous frame;

identifying a slide region of the segmented frame;

generating a histogram representing lines in the slide region; and

suppressing moving regions associated with successive frames from the histogram;

the suppressing including, capturing a foreground silhouette;

copying the foreground silhouette;

assigning the copied foreground silhouette a most recent timestamp; and

generating a motion mask, wherein the edges located within the motion mask are excluded from the histogram.

15. The method of claim 14, wherein the plurality of neighbor pixel values include five pixel values.