System and method for skimming digital audio/video data

• US 5,664,227 A
• Filed: 10/14/1994
• Issued: 09/02/1997
• Est. Priority Date: 10/14/1994
• Status: Expired due to Term

• Current Assignee: Carnegie Mellon University
• Original Assignee: Carnegie Mellon University
• Inventors: Smith, Michael A., Mauldin, Michael L., Stevens, Scott M., Christel, Michael G., Wactlar, Howard D., Reddy, D. Raj
• Primary Examiner(s): McElheny, Jr., Donald E.
• Application Number:
  US08/324,079
• Time in Patent Office: 1,054 Days
• Field of Search: 364/419.13, 364/419.19, 360/14.3, 360/72.2, 395/778, 395/793, 395/807
• US Class Current: 715/203
• CPC Class Codes: G06F 16/739 in form of a video summary, e.g. the video summary being a video sequence, a composite still image or having synthesized frames
• CPC Class Codes: G06F 16/7844 using original textual content or text extracted from visual content or transcript of audio data

Abstract

A method for skimming digital audio and video data, wherein the video data is partitioned into video segments and the audio data has been transcribed, is comprised of the steps of selecting representative frames from each of the video segments. The representative frames are combined to form an assembled video sequence. Keywords contained in the corresponding transcribed audio data are identified and extracted. The extracted keywords are assembled into an audio track. The assembled video sequence and audio track are output together. An apparatus for carrying out the disclosed method is also disclosed.

Claims

1. A method for skimming digital audio and video data wherein the video data is partitioned into video segments and the audio data has been transcribed, said method comprising:

selecting at least one representative frame from each of the video segments;

combining said representative frames to form an assembled video sequence;

identifying keywords contained in the transcribed audio data;

extracting those portions of the audio data identified as keywords;

assembling an audio track from said extracted audio data; and

outputting said assembled video sequence in conjunction with said assembled audio track.

2. The method of claim 1 wherein said representative frames are selected using content-independent statistical methods which detect image changes.

3. The method of claim 1 wherein said representative frames are selected to correspond to certain of said extracted portions of the audio data.

4. The method of claim 1 wherein said keywords contained in the transcribed audio data are identified using natural language processing techniques.

5. The method of claim 4 wherein said natural language processing techniques are statistical techniques.

6. The method of claim 4 wherein said natural language processing techniques are expert systems.

7. The method of claim 1 wherein said keywords contained in the transcribed audio data are identified using a term weighting process.

8. The method of claim 1 further comprising the step of compressing said assembled audio track and said assembled video sequence before outputting said assembled video sequence in conjunction with said assembled audio track.

9. An apparatus for skimming digital audio and video data wherein the video data is partitioned into video segments and the audio data has been transcribed, said apparatus comprising:

means for selecting at least one representative frame from each of the video segments;

means for combining said representative frames to form an assembled video sequence;

means for identifying keywords contained in the transcribed audio data;

means for extracting those portions of the audio data identified as keywords;

means for assembling an audio track from said extracted audio data; and

means for outputting said assembled video sequence in conjunction with said assembled audio track.

10. The apparatus of claim 9 wherein said means for selecting representative frames uses content-independent statistical methods which detect image changes.

11. The apparatus of claim 9 wherein said means for selecting representative frames selects said frames to correspond to certain of said extracted portions of the audio data.

12. The apparatus of claim 9 wherein said means for identifying said keywords contained in the transcribed audio data uses natural language processing techniques.

13. The apparatus of claim 12 wherein said natural language processing techniques are statistical techniques.

14. The apparatus of claim 12 wherein said natural language processing techniques are expert systems.

15. The apparatus of claim 9 wherein said means for identifying said keywords contained in the transcribed audio data uses a term weighting process.

16. The apparatus of claim 9 further comprising means for compressing said assembled audio track and said assembled video sequence before said means for outputting.

17. A method of preparing data, comprising the steps of:

partitioning video data into video segments based on content;

time stamping said video segments;

selecting at least one representative frame from each of the video segments;

combining said representative frames to form an assembled video sequence;

transcribing audio data;

time stamping said transcribed audio data;

identifying key words contained in said transcribed audio data;

extracting those portions of said audio data identified as key words; and

assembling an audio track in response to said extracted audio data.

18. An apparatus for preparing data, comprising:

means for partitioning video data into video segments based on content;

means for time stamping said video segments;

means for selecting at least one representative frame from each of the video segments;

means for combining said representative frames to form an assembled video sequence;

means for transcribing audio data;

means for time stamping said transcribed audio data;

means for identifying key words contained in said transcribed audio data;

means for extracting those portions of said audio data identified as key words; and

means for assembling an audio track in response to said extracted audio data.