Multimedia search and indexing for automatic selection of scenes and/or sounds recorded in a media for replay

• US 6,377,519 B1
• Filed: 10/23/2000
• Issued: 04/23/2002
• Est. Priority Date: 06/30/1998
• Status: Expired due to Term

• Current Assignee: GlobalFoundries, Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: Lee, Peter S., Stern, Edith H., Willner, Barry E.
• Primary Examiner(s): DINH, TAN X
• Application Number:
  US09/693,926
• Time in Patent Office: 547 Days
• Field of Search: 369/30.05, 369/30.07, 369/30.08, 369/30.09, 369/30.19, 369/30.21, 369/30.04, 360/15, 360/13, 360/5, 386/55, 386/52, 386/4.5, 386/69, 386/70, 386/75, 386/81, 386/82
• US Class Current: 369/30.04
• CPC Class Codes: G06F 16/7834 using audio features
• CPC Class Codes: G06Q 99/00 Subject matter not provided for in other groups of this subclass
• CPC Class Codes: G11B 2220/20 Disc-shaped record carriers
• CPC Class Codes: G11B 2220/90 Tape-like record carriers
• CPC Class Codes: G11B 27/031 Electronic editing of digitised analogue information signals, e.g. audio or video signals
• CPC Class Codes: G11B 27/032 on tapes G11B27/036, G11B27/038 take precedence
• CPC Class Codes: G11B 27/034 on discs G11B27/036, G11B27/038 take precedence
• CPC Class Codes: G11B 27/107 of operating tapes
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence

Abstract

A multimedia search and indexing system automatically selects scenes or events of interest from any media, i.e., video, film, sound for replay, in whole or in part, in other contexts. The entire audio track of a recorded event in video, film, sound, etc., is analyzed to determine audio levels within a set of frequency ranges of interest. Audio clip levels within the selected frequency ranges are chosen as audio cues representative of events of interest in the track. The selection criteria are applied to the audio track of the recorded event. An Edit Decision List (EDL) is generated from the analysis of the audio track. The list is representative of scenes or sounds of interest as clips for reuse. The clips are reviewed and accepted or rejected for reuse. Once selected, the clips are edited using industry standard audio and video editing techniques.

Claims

1. A multimedia search and indexing system for automatic selection of scenes or sounds recorded in a media for replay in other contexts, comprising:

(a) means for selecting analysis intervals in the media;

(b) means for selecting desired frequency ranges for examination;

(c) means for recording the frequency range, audio level, and an index for each analysis interval;

(d) means for automatically comparing recorded audio level for a selected interval versus a clip level in a frequency range and generating an Edit Decision List (EDL); and

(e) means for selecting clips from the Edit Decision List for replay.

2. The system of claim 1 further comprising:

(f) means for setting parameters by frequency range as clip levels for scenes or sounds of interest.

3. The system of claim 1 further comprising:

(g) means for modifying the parameters and generating a revised Edit Decision List (EDL) for selection of different clips for replay.

4. The system of claim 1 further comprising:

(h) means for generating a start and end index for the selected clips in the EDL.

5. The system of claim 1 further comprising:

(i) means for editing media for selected media clips for re-purposing.

6. The system of claim 1 further comprising:

(j) programmable filter means for selecting frequency ranges of interest in the media clips.

7. In a multi media search and indexing system including a processor, audio analysis means, and selection means for scenes or sounds in a media, a method for automatic selection of scenes or sounds recorded in the media for replay in other contexts, comprising the steps of:

(a) selecting analysis intervals in the media;

(b) selecting desired frequency ranges in the media;

(c) recording the frequency range, audio level and an index for each scene or sound interval;

(d) automatically comparing recorded audio level for a selected interval versus an audio clip level in a frequency range and generating an Edit Decision List (EDL); and

(e) selecting clips from the Edit Decision List (EDL) and editing the selected clips for replay.

8. The method of claim 7 further comprising the step of:

(f) generating a start and end time code for the selected clips in the EDL.

9. The method of claim 7 further comprising:

(g) setting audio parameters by frequency range in the processor as audio cues for scenes of interest from the selected intervals.

10. The method of claim 7 further comprising the step of:

(h) editing media to select media clips for re-purposing, each clip containing audio levels accompany visual scenes.

11. The method of claim 7 further comprising the step of:

(i) selecting frequency ranges of interest in the media clips using a programmable filter means.

12. The method of claim 7 further comprising the step of:

(j) selecting an analysis granularity for the selected media clip.

13. The method of claim 7 further comprising the step of:

(j) selecting time parameters for the media clips wherein a first parameter represents t the time period prior to attainment of an audio level threshold and a second parameter represents the time period after attainment of the audio level threshold in the media clip.

14. The method of claim 7 further comprising the step of:

(k) automatically setting audio cues for selection of scenes and/or sounds of interest in the media clips.

15. The method of claim 7 further comprising the step of:

(l) selecting media clips where the audio level is above a clip level.

16. The method of claim 7 further comprising the step of:

(m) selecting media clips where the audio level is below a clip level.

17. A multimedia search and indexing system for automatic selection of scenes and/or sounds recorded in a media for replay in other contexts, comprising:

a) means for editing media for selected media clips for re-purposing;

b) programmable filter means for selecting frequency ranges of interest in the media clips;

c) means for selecting an analysis granularity for the selected media clips;

d) means for selecting time parameters for the media clips wherein a first parameter represents the time period prior to attainment of an audio level threshold and a second parameter represents the time period after attainment of the audio level threshold in the media clip;

e) means for automatically setting audio cues for selection of scenes and /or sounds of interest in the media clips; and

f) means for selecting media clips where the audio level is above a clip level.

18. The system of claim 17 further comprising:

g) means for selecting media clips where the audio level is below a clip level.

19. The system of claim 17 further comprising:

h) an analog/digital converter for converting an analog signal into a digital counterpart.

20. The system of claim 17 further wherein the selection of the desired frequency ranges is the selection of one frequency range.

21. The system of claim 20 wherein the one frequency range is a human frequency range.

22. The system of claim 20 wherein the one frequency range is an entire audio spectrum.

23. The system of claim 20 wherein the one frequency range is the system capacity.

24. In a multimedia search and indexing system for automatic selection of scenes and/or sounds recorded in a media, a method of selecting scenes and/or sounds for replay in other contexts, comprising the steps of:

a) editing media for selected media clips for re-purposing;

b) selecting frequency ranges of interest in the media clips using a programmable filter;

c) selecting an analysis granularity for the selected media clips;

d) selecting time parameters for the media clips wherein a first parameter represents the time period prior to attainment of an audio level threshold and a second parameter represents the time period after attainment of the audio level threshold in the media clip;

e) automatically setting audio cues for selection of scenes and /or sounds of interest in the media clips; and

f) selecting media clips where the audio level is above a clip level.

25. The method of claim 24 further comprising the step of:

g) selecting media clips where the audio level is below a clip level.

26. The method of claim 24 further comprising the step of:

h) converting an analog signal into a digital counterpart using an analog/digital converter.

27. The method of claim 24 further wherein the selection of the desired frequency ranges is the selection of one frequency range.

28. The method of claim 27 wherein the one frequency range is a human frequency range.

29. The system of claim 27 wherein the one frequency range is an entire audio spectrum.

30. The method of claim 27 wherein the one frequency range is the system capacity.

31. A program medium, executable in a computer system, for automatic selection of scenes and/or sounds recorded in a media for replay in other contexts, comprising:

a) program code in the medium for editing media for selected media clips for re-purposing;

b) program code in the medium for selecting frequency ranges of interest in the media clips using a programmable filter means;

c) program code in the medium for selecting an analysis granularity for the selected media clips;

d) program code in the medium for selecting time parameters for the media clips wherein a first parameter represents the time period prior to attainment of an audio level threshold and a second parameter represents the time period after attainment of the audio level threshold in the media clip;

e) program code in the medium for automatically setting audio cues for selection of scenes and /or sounds of interest in the media clips; and

f) program code in the medium for selecting media clips where the audio level is above a clip level.

32. The program medium of claim 31 further comprising:

(g) program code in the medium for selecting media clips where the audio level is below a clip level.

33. The program medium of claim 31 further comprising:

(h) program code in the medium converting an analog signal into a digital counterpart using an analog/digital converter.