Multimedia search and indexing for automatic selection of scenes and/or sounds recorded in a media for replay by setting audio clip levels for frequency ranges of interest in the media

• US 6,452,875 B1
• Filed: 10/23/2000
• Issued: 09/17/2002
• Est. Priority Date: 06/30/1998
• Status: Expired due to Term

• Current Assignee: GlobalFoundries, Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: Lee, Peter S., Stern, Edith H., Willner, Barry E.
• Primary Examiner(s): DINH, TAN X
• Application Number:
  US09/693,815
• Time in Patent Office: 694 Days
• Field of Search: 369/30.04, 369/30.07, 369/30.08, 369/30.05, 369/30.19, 369/30.21, 369/83, 360/13, 360/5, 386/55, 386/52, 386/4, 386/69, 386/70, 386/75, 386/81, 386/82
• US Class Current: 369/30.04
• CPC Class Codes: G06F 16/7834 using audio features
• CPC Class Codes: G06Q 99/00 Subject matter not provided for in other groups of this subclass
• CPC Class Codes: G11B 2220/20 Disc-shaped record carriers
• CPC Class Codes: G11B 2220/90 Tape-like record carriers
• CPC Class Codes: G11B 27/031 Electronic editing of digitised analogue information signals, e.g. audio or video signals
• CPC Class Codes: G11B 27/032 on tapes G11B27/036, G11B27/038 take precedence
• CPC Class Codes: G11B 27/034 on discs G11B27/036, G11B27/038 take precedence
• CPC Class Codes: G11B 27/107 of operating tapes
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence

Abstract

A multimedia search and indexing system automatically selects scenes or events of interest from any media, i.e., video, film, sound for replay, in whole or in part, in other contexts. The entire audio track of a recorded event in video, film, sound, etc., is analyzed to determine audio levels within a set of frequency ranges of interest. Audio clip levels within the selected frequency ranges are chosen as audio cues representative of events of interest in the track. The selection criteria are applied to the audio track of the recorded event. An Edit Decision List (EDL) is generated from the analysis of the audio track. The list is representative of scenes or sounds of interest as clips for reuse. The clips are reviewed and accepted or rejected for reuse. Once selected, the clips are edited using industry standard audio and video editing techniques.

Claims

1. In a signal processing system including a multi media search and indexing system for automatic selection of scenes or sounds recorded in a media for replay in other contexts, a method for setting audio clip levels in analyzing the media for a set of frequency ranges of interest for replay, comprising the steps of:

(a) selecting an audio clip level for each frequency range as indicative of a scene or sound of interest in the media;

(b) selecting a time interval in seconds leading an audio level exceeding the clip level;

(c) selecting a time interval in seconds following the exceeded audio clip level;

(d) repeating steps (a), (b), and (c) for each frequency range; and

(e) recording and relating each scene of interest exceeding the audio clip level to the index in the media.

2. The method of claim 1 further comprising the step of:

(f) comparing the recorded audio clip level with target clip level.

3. The method of claim 1 further comprising the step of:

(g) determining if the audio clip level was reached.

4. The method of claim 1 further comprising the step of:

(h) recording an associated time code when the audio clip level is reached.

5. The method of claim 1 further comprising the step of:

(j) cross-referencing fixed text word recognition in the frequency range with the audio clip level.

6. The method of claim 1 further comprising the step of:

(k) assigning a time code to the scene of interest exceeding the audio clip level.

7. The method of claim 1 further wherein the selection of the desired frequency ranges is the selection of one frequency range.

8. The method of claim 7 wherein the one frequency range is a human frequency range.

9. The method of claim 7 wherein the one frequency range is an entire audio spectrum.

10. The method of claim 7 wherein the one frequency range is the system capacity.

11. A program medium executable on a computer system for automatic selection of scenes or sounds recorded in a media for replay in other contexts, comprising:

(a) program code in the medium selecting an audio clip level for each frequency range as indicative of a scene or sound of interest in a media;

(b) program code selecting a time interval leading an audio level exceeding the clip level;

(c) program code selecting a time interval following the exceeded audio clip level;

(d) program code in a medium for repeating steps (a), (b) and (c) for each frequency range;

(e) program code in a medium for recording and relating each scene of interest exceeding the audio clip level to an index in the media.

12. A system for automatic selection of scenes or sounds recorded in the media for replay in other contexts, comprising:

(a) means selecting an audio clip level for each frequency range as indicative of a scene or sound of interest in the media;

(b) means selecting a time interval leading an audio level exceeding the clip level;

(c) means selecting a time interval following the exceeded audio clip level; and

(d) means recording and relating each scene of interest exceeding the audio clip level to an index in the media.

13. The system of claim 12 further comprising:

(e) means for comparing the recorded audio clip level with target clip level.

14. The system of claim 12 further comprising:

(f) means for determining if the audio clip level was reached.

15. The system of claim 12 further comprising:

(h) means for recording an associated time code when the audio clip level is reached.

16. The system of claim 12 further comprising:

(j) means for cross-referencing fixed text word recognition in the frequency range with the audio clip level.

17. The system of claim 12 further comprising:

(k) means for assigning a time code to the scene of interest exceeding the audio clip level.

18. The system of claim 12 further wherein the selection of the desired frequency ranges is the selection of one frequency range.

19. The system of claim 18 wherein the one frequency range is a human frequency range.

20. The system of claim 18 wherein the one frequency range is an entire audio spectrum.

21. The system of claim 18 wherein the one frequency range is the system capacity.